Exploring a new AI lexicon

We’re back with another Fully Connected episode – Daniel and Chris dive into a series of articles called ‘A New AI Lexicon’ that collectively explore alternate narratives, positionalities, and understandings to the better known and widely circulated ways of talking about AI. The fun begins early as they discuss and debate ‘An Electric Brain’ with strong opinions, and consider viewpoints that aren’t always popular. Sponsors: RudderStack https://rudderstack.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog – Smart customer data pipeline made for developers. RudderStack is the smart customer data pipeline. Connect your whole customer data stack. Warehouse-first, open source Segment alternative. SignalWire https://signalwire.com – Build what’s next in communications with video, voice, and messaging APIs powered by elastic cloud infrastructure. Try it today at signalwire.com https://signalwirecom and use code AI for $25 in developer credit. Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: A New AI Lexicon https://medium.com/a-new-ai-lexiconA New AI Lexicon: An Electric Brain https://medium.com/a-new-ai-lexicon/a-new-ai-lexicon-an-electric-brain-77a81f3ce446A New AI Lexicon: Function Creep https://medium.com/a-new-ai-lexicon/a-new-ai-lexicon-function-creep-1c20834fab4aA New AI Lexicon: Smart https://medium.com/a-new-ai-lexicon/a-new-ai-lexicon-smart-580484078624A New AI Lexicon: Artificial Identity Cataracts https://medium.com/a-new-ai-lexicon/a-new-ai-lexicon-artificial-identity-cataracts-29f73c32e6bbA New AI Lexicon: Imbrication https://medium.com/a-new-ai-lexicon/a-new-ai-lexicon-imbrication-40b380dafa35Books Free 500+ PDF “Applications of Deep Neural Networks with Keras” book from Jeff Heaton, who is teaching this course at Washington University in St. Louis https://arxiv.org/pdf/2009.05673.pdfUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Daniel Whitenack?

  Maybe it's the scaffolding around those terms. It sets people up for failure if you're saying,

• 0:04Chris Benson?

  It does.

• 0:05Daniel Whitenack?

  I'm going to train my artificial intelligence model on accelerated hardware using an optimization scheme of mini batch gradient descent.

• 0:18Chris Benson?

  Yes.

• 0:18Daniel Whitenack?

  That whole like sentence there

• 0:20Chris Benson?

  Yes.

• 0:21Daniel Whitenack?

  just excludes so many people because they're just like, oh, I give up.

• 0:24Chris Benson?

  You're probably right.

• 0:28Speaker 02

  Big thanks to our partners, Leno, Fastly, and Launch Darkly. We love Leno. They keep it fast and simple. Check them out at leno.com/changelog. Our bandwidth is provided by Fastly. Learn more at fastly.com and get your feature flags powered by Launch Darkly. Get a demo at launchdarkly.com. This episode is brought to you by our friends at RudderStack and we're calling all data engineers to check out RudderStack Cloud and start building smart customer data pipelines. RudderStack is warehouse-first, no more silos. RudderStack builds your customer data lake on your data warehouse, not theirs, enabling all functionality of a CDP with more security and retaining full ownership of your data. It's open source and API-first. RudderStack can be easily integrated into your existing development processes.

• 1:12Speaker 02

  And because they're open source, you can see all their code. So you don't have to worry about vendor lock-in or black boxes. And best of all, they have transparent pricing. Stop paying your CDP a premium to store your data. RudderStack is free up to 500,000 events and pricing scales transparently from there. Learn more and get started at rudderstack.com. Again, rudderstack.com. That's R U D D E R S T A C K dot com.

• 1:47Speaker 02

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We are at Practical AI FM.

• 2:12Daniel Whitenack?

  Welcome to another fully connected episode of Practical AI, where Chris and I keep you fully connected with everything that's happening in the AI community. We'll talk a little bit about some AI news or articles and we'll dig into some learning resources to help you level up your machine learning game. I'm Daniel Whiteknack from S I L International and I'm joined as always by Chris Benson, who is a strategist at Lockheed Martin. How you doing, Chris?

• 2:43Chris Benson?

  I am doing very well, Daniel. Looking forward to today's conversation. I always love our fully connected episodes because we can kind of go wherever we want, whatever captures our interest.

• 2:52Daniel Whitenack?

  Wherever we want, yeah, the no constraints. We can say whatever might pop into our minds. What's on your mind these days? How's work and life?

• 3:00Chris Benson?

  Work and life is good. The little bit of travel lately, which has coincided with our third big wave of COVID as we are recording this and will release shortly. So other than the terror of sitting shoulder to shoulder with a bunch of people on an airplane, a third of which are statistically not vaccinated, other than that and me trying to super glue my mask to my face around the edges, I'm doing great, doing great.

• 3:26Daniel Whitenack?

  Yeah, good, good. Yeah, I feel like this week for me was just a tiring week, but it was like really good. There's a few milestones on various projects that we hit and/or deadlines that came up, but everything went good. But it was just one of those weeks, you know, like you have this backlog of tasks and then everything sort of coincides in one week occasionally. That was this week for me, but thankfully I got a good team and worked through all of that together and was able to release a couple of new releases and train some new models and get some infrastructure things set up. I've been toying around with Clear ML a little bit more, which we had an episode about a while back and doing some

• 4:08Daniel Whitenack?

  experiment tracking and queuing up jobs for GPUs and that sort of thing. And that's been, that's been quite fun to sort of dive into their tooling for that.

• 4:17Chris Benson?

  That does sound like a nice week indeed.

• 4:19Daniel Whitenack?

  Yeah, otherwise I watched Sliced the other night, which we just talked to Meg and Nick from Sliced, which is a live data science competition. And I watched that the other night and that was, that was quite fun. It's just interesting to see you sort of have a stigma of certain people that like, they're like a data science like hero. And when they sit down to code, like they're not, you know, going to Stack Overflow and they're not like doing a thing that just like barfs up a bunch of exceptions on them. But that's definitely not the case, right? So it's, it's just cool to like, kind of sit back and see people live, you know, oh, I'm going to pull up Stack Overflow, I'm going to pull up Plotly documentation.

• 5:03Daniel Whitenack?

  Like, how do I do grouped, you know, bar plot in Plotly? And that's, that's like the same thing I do. It's like whenever I hit one of those things, I just go straight to Google and, you know.

• 5:14Chris Benson?

  Don't we all?

• 5:15Daniel Whitenack?

  Yeah.

• 5:15Chris Benson?

  You know, the same thing on the software development side as opposed to the data science side is even the, the great gods of software development, just as data science, they, they go through the same process.

• 5:25Daniel Whitenack?

  Yeah.

• 5:25Chris Benson?

  They are human.

• 5:27Daniel Whitenack?

  Speaking of human versus AI, a couple of things have happened in, in recent days. Um, one of these is like the next thing in like AI code generation, or I guess not a next different thing, but sort of a new interface to something that we've already talked about, GitHub Copilot. But OpenAI released the individual, you know, access to their Codex model, which is this model that does the functionality of like, I'm going to write in natural language what I want my code to do. And then the model takes in that natural language and generates the code to do the thing. So they released that. That's one thing that's sort of interesting and thinking about like AI creating code

• 6:12Daniel Whitenack?

  is always interesting idea. I don't know if you've had any other thoughts on that recently.

• 6:17Chris Benson?

  I've been having a conversation, an ongoing conversation with a good friend about no-code solutions. That applies to deep learning technologies as well. So I am keenly interested in seeing how these come together. I'm looking at the OpenAI Codex page and where it says, "Join the waitlist." I noticed that it mentions that it is proficient in more than a dozen programming languages. Have you found any place where it actually tells you which ones?

• 6:42Daniel Whitenack?

  I haven't done as much digging on that side of things. I know just from my experience on Twitter and seeing what people are generating. I've seen Python, I've seen JavaScript, but I assume it's more than that. I know when GitHub Copilot released, they talked about all sorts of different programming languages. But yeah, I don't have any other context other than that. But I even saw some sort of interesting scripting and one of the examples I saw was a Microsoft Word document. Maybe I want all of these lines re-indented a certain way or something. So I do it a couple of times and I'm like, "I don't want to do this for like 14 pages of documents."

• 7:22Daniel Whitenack?

  So they like gave a little narrative of what remove spaces in this way, you know, that they said in natural language. And I don't know the plugin to Microsoft Word that enabled this, but a little script was written and then it did the thing and re-oriented things, which is kind of a cool idea.

• 7:41Chris Benson?

  It is.

• 7:42Daniel Whitenack?

  Yeah, especially for like non-technical people. This model, it's like code generation. So I think the one thought is like, oh, this is for coders or technical people to aid them. But that example is actually, you know, has an impact potentially for non-technical people. I'm just using a Word document. And actually, before this thing existed, maybe I wasn't able to write cool scripts because I'm not a programmer. But now I can like have the superpower to like do these sort of tasks over my document or over my data that

• 8:16Daniel Whitenack?

  You know, I didn't have the ability to do before because I didn't know complicated like pivot table map function things in Excel or whatever, however, people do those things that don't code.

• 8:27Chris Benson?

  I think we're kind of coming into this point where we're seeing these capabilities, like what OpenAI is putting out and more and more these types of kind of sophisticated, let us do this for you, let us get that. It's bringing more and more people into this field to take advantage of it. If there's anything that we have seen in the last, this is where the innovation is happening right now. We're not seeing giant new architectures, the way we did a few years ago, where, you know, entirely new paradigms of deep learning architectures. What we're seeing now are novel ways of using it, making it more accessible to people and being able to get more productive and stuff. But just for the record, I'm still waiting for somebody to bring us some novel new architectures as well. I'm ready for some more right now, just, just for the record.

• 9:12Daniel Whitenack?

  Yeah.

• 9:13Chris Benson?

  In case anyone out there is about to release a breakthrough.

• 9:15Daniel Whitenack?

  Yeah, come on the show and talk about it.

• 9:17Chris Benson?

  Absolutely. Do it right here. This is the place to do it.

• 9:20Daniel Whitenack?

  We can release the show the same day as your paper.

• 9:23Chris Benson?

  Excellent.

• 9:24Daniel Whitenack?

  Yeah, that would be cool. It'd be cool to coordinate that at some point. That's a good example, like AI is sort of doing this incursion or infiltration of all sorts of things that maybe we didn't anticipate before, like including Microsoft Word and scripting in Microsoft Word. One of the things that I wanted to bring to your attention and maybe chat through a bit today was this series of articles that the AI Now is putting together, which is a series of blog posts called A New AI Lexicon. And they talk about, hey, AI is having this sort of incursion into all bits of our life.

• 10:06Daniel Whitenack?

  And they talk about needing to generate narratives that can offer both perspectives from other places, but also crucial anticipatory knowledge and strategy. So in my mind, people might not be familiar with the term lexicon, at least before I worked for an organization that did a bunch of language stuff. It wasn't a term that I used very, very much, but I'm just looking at the definition here. And a lexicon is the vocabulary of a person, language, or branch of knowledge.

• 10:38Daniel Whitenack?

  So I think when they're talking about a new AI lexicon, AI would maybe fit into this branch of knowledge category. And we're talking about the vocabulary of this branch of knowledge.

• 10:51Chris Benson?

  For the moment, you know?

• 10:52Daniel Whitenack?

  For the moment, yeah. And I think one of the hardest things for me when I was getting into this field was the jargon and vocabulary, because even from a technical background, I had run across certain things like, you know, whatever it is, ordinarily squared or something. But the way and the jargon in which AI people or data science people talked about these things was different vocabulary and jargon.

• 11:18Chris Benson?

  Yeah.

• 11:18Daniel Whitenack?

  So it wasn't so much that like, the thing was scary or complicated. It was that the vocabulary actually made it scary and complicated. What's your experience there?

• 11:29Chris Benson?

  I agree with you because language is powerful and how we label things is powerful. I have strong opinions there. We can get into as we go forward.

• 11:37Daniel Whitenack?

  Yeah, yeah.

• 11:38Chris Benson?

  Even in the time that we've been doing the show, for those who might have been listening to us since the early days, it's evolved quite a bit. And so I think this is a timely topic personally.

• 11:48Daniel Whitenack?

  Yeah, for sure. Maybe I'll just illustrate what I'm talking about with one of these articles, which I think is a good starting point for this discussion. It's the article called "An Electronic Brain: Naming, Categorizing, and the Futures of AI." That's a guest post by a doctoral researcher at Oxford, Young Au. Sorry if I mispronounce your name. But this article I thought was just fascinating.

• 12:14Chris Benson?

  I did too.

• 12:15Daniel Whitenack?

  I think all of these articles, if I was to put them in a category, it would be like, thought provoking or something. Because I don't know, you know, reading through the bulk of them, I don't think maybe anyone could read through all of them and maybe say, hey, I fully agree with everything that's written here. That's my perspective too. It's different perspectives on the subject of AI, right? So I think it's like, thought provoking. It helps you sort of reformulate how you think about AI and the language that you use around AI. And in this article, "The Electronic Brain," Young, the author, talks about Cantonese terms.

• 12:52Daniel Whitenack?

  And the way that Cantonese terms are are made up is that they're composed of a set of characters and each individual character and its components have some sort of meaning. And so what the author did was look through various terms for things related to AI. So like computer, artificial intelligence, automation, et cetera, and looked at these components in the Cantonese language for sort of insights, maybe about how these terms from the Cantonese perspective might sort of shed light on how we think about AI-related things.

• 13:30Chris Benson?

  And by the way, just as a two-second aside, for anybody that is not familiar with Cantonese out there, it is the second most dominant language in China. Its usage, it lags far behind Mandarin at this point because that is the Communist Party's official language, but Cantonese would be the second most widely spoken language in China.

• 13:50Daniel Whitenack?

  I love language-related things, just a plug, if you if you're wanting some knowledge about like languages and their relationships and populations and all of that, you can look at the Ethnologue is one good place to look for that information, but there's also sort of open versions that are sort of, you'll see these trees of language varieties and how many people speak them and populations and all of that stuff. The author goes through and breaks apart some of these common terms. So for example, for computer, the sort of breakdown of that in terms of the characters that he brought out was called electric brain, which I don't know, when you look at a computer, what what is your, are you thinking electric brain?

• 14:35Chris Benson?

  Indeed, I do not.

• 14:36Daniel Whitenack?

  In what way? I mean, is it the brain part? I mean, it's definitely electric, right?

• 14:40Chris Benson?

  It is electric. I I

• 14:42Daniel Whitenack?

  You don't doubt that?

• 14:43Chris Benson?

  And our brains are electric as well, so.

• 14:45Daniel Whitenack?

  Maybe you're saying that electric brain is redundant because brain implies electric.

• 14:50Chris Benson?

  Actually, that's not where my head was at, but that is in fact true. They both have electrical signals in them that are core to their functionality, but I think that's about the limit of my analogy there.

• 15:02Daniel Whitenack?

  Yeah, so that's computer, which is, I think, pretty interesting. He gives sort of alternatives and breaks down the individual characters. I showed one of my colleagues. The breakdown and he was seeing that, you know, computer could also maybe be like, what do you say, like meet rain or something?

• 15:21Daniel Whitenack?

  Which I don't know what that tells us about how we think about computers. But he goes in and he talks about, you know, these components also of how we frame the term artificial intelligence, which I think is very thought provoking for me. And he draws out this idea that the term itself in Cantonese and the way it's made up talks about, you could think about it sort of primarily as artificial intelligent, but also a sort of parallel to that that's in the language is really more focused on like manual labor or manpower and like able or a system or smart able to do manpower

• 16:03Daniel Whitenack?

  or manual labor, which is much more of a sort of practical idea about artificial intelligence. How does that strike you?

• 16:15Chris Benson?

  It feels more like the feelings that we have when we say machine learning instead of artificial intelligence, you know, the way they do that. I think practitioners tend to think of machine learning as the pragmatic, fundamental, you know, stick with the science kind of mentality, whereas AI is a little bit more glossy marketing, even though that you're speaking of the AI translation, it feels like that's how I would think of it as machine learning in my own context.

• 16:55Speaker 03

  Signal Wire is real time video tech to help you create interactive video experiences previously not possible. It gives you access to broadcast quality ultra-low latency video that's proven and trusted by Amazon, Ring Doorbell, Zoom, and others. See why the future of video communication is being built on Signal Wire. They have easy-to-deploy APIs, SDKs for the most popular programming languages, and expert support from the OGs of software-defined telecom tech. Try it today at signalwire.com and use code AI for $25 in developer credit. Just visit signalwire.com,

• 17:35Speaker 03

  that's signalwire.com, and use code AI to receive that 25 bucks. Once again, that's signalwire.com, code AI.

• 17:56Daniel Whitenack?

  So Chris, we were just getting into some of this discussion about how AI is represented in the Cantonese context and starting to talk about how there is embedded within the term this idea of manual labor or something very practical.

• 18:13Chris Benson?

  Yes.

• 18:13Daniel Whitenack?

  One of the things that's mentioned in this article is thinking about, you know, what if these systems, what if AI systems or automated systems or augmented systems, what if these systems were seen less exceptional or very sophisticated or whatever that term is, and more ordinary or less exceptional?

• 18:36Chris Benson?

  You don't want to take me down this path because I have strong opinions on this one. You don't want to do that, do you?

• 18:41Daniel Whitenack?

  Yeah, so there's like, the things that he draws out are more ordinary and less exceptional, or more human and less machine, more labor and less enchantment. I thought that was a cool, like, a pairing of terms. So like, There is this perception, maybe not by the practitioner like you were talking about, but definitely by people at large that AI systems have some sort of enchantment around them.

• 19:09Chris Benson?

  Yeah.

• 19:09Daniel Whitenack?

  When I'm teaching classes, I think oftentimes when I get into like, okay, well, if we think about most AI systems, and there's certainly exceptions that don't fit into this pattern that I'm about to say, but if we think about most AI systems, what we're talking about is essentially an automated process of trial and error. It's actually a very dumb process, right? In the sense that we have a set of examples, we're trying to recreate the data transformation from that input to an output, and the way in which we do it is we try a bunch of times. So we put the examples in, we try to make the prediction based on some encoded parameters,

• 19:49Daniel Whitenack?

  and when it's not good, we change the parameters, and then we just do that a bajillion times until we get the right set or the quote best set of parameters. And then out comes the model, which so many people see as enchantment, or like you put fairy dust on the computer in the corner. And, you know, I see some faces in training sometimes where it's like, oh, it's a little bit disappointing for people. In other ways, it's like maybe reassuring because they came into it thinking it's harder to get into this practice than they thought because there's some like fundamentally subtle and mysterious thing that they need to understand in order to do the work.

• 20:31Chris Benson?

  Daniel, they hear it's a black box. It's a black box, it's magical, this is their way into Hogwarts.

• 20:37Daniel Whitenack?

  There's another term, black box, right?

• 20:39Chris Benson?

  It is.

• 20:40Daniel Whitenack?

  What does that terminology imply about how we think about?

• 20:44Chris Benson?

  At least, I don't know how people are thinking about it now, but early on it was, it's unknowable. And that was really before the rise of explainable AI as a field. Yeah, I mean, that's what black box was intended to be is this is unknowable what we're doing here, which of course is a little bit silly.

• 21:00Daniel Whitenack?

  Yeah, and there's also this very unfortunate connotation with black just meaning it does have a negative connotation in the context of like, most of the times, these terms that people have used through time, like a blacklist or something, which now we're sort of trying to get away from those terms because we definitely don't want that connotation in many cases with like, black shouldn't have this sort of pre-existing negative connotation. And so when you say black box, there's like this part of, part of that term is like this unknowingness about it. But from the start, at least in our culture, you also get this sense of maybe you're trying

• 21:40Daniel Whitenack?

  to say something, something negative about the AI thing, that it's black box, it's unknowing, but maybe in an unknowing, threatening sort of way.

• 21:51Chris Benson?

  There has always been kind of a mysticism around the term artificial intelligence or AI. And frankly, there have been many marketers out there who have taken great advantage of that. But what we do, the deep learning techniques that we do is just mathematics. And it's very, very pragmatic. It's very down to earth and there's nothing mystical about it. So I think that's one of the places I've arrived. When we started the show and named our show Practical AI, I was probably pretty high on the term AI then, but as we have really delved into this and the black box nature of the math has changed, it's hard for me to think of it as, and I certainly don't think of it as intelligent, it's hard for me to even think of it as AI at this point.

• 22:35Chris Benson?

  So I think my term AI has gotten more aspirational as we've gone on, meaning that we have not yet achieved it.

• 22:41Daniel Whitenack?

  Yeah, it's this system or machinery like was talking about in that sort of breakdown of the Cantonese. The system or machinery that is doing some kind of man or woman related power or labor. And so, yeah, I think that's a very practical way to look at it. And I know in the past, you know, just speaking of lexicons and how we talk about AI, even this recent episode where we chatted with the people from SLICE, so Meg and Nick, I've already mentioned, I've been watching that. The sort of premise of that show was their concept around Chopped, the cooking show.

• 23:22Daniel Whitenack?

  And I found it very interesting that that's also in my lexicon of AI terms. That's very much how I think about actual AI development. And the practicalities of AI development is much more like cooking than it is this sort of like pure research topic of people standing at a chalkboard and doing various things. And so that in the spirit of what AI now is trying to do, you know, I would think that, for example, like a recipe is a good term to have in an AI lexicon where I agree. It's not, because even like if you think about a model, model has certain connotations,

• 24:03Daniel Whitenack?

  especially for people that aren't like deeply rooted in maybe a scientific discipline like physics or chemistry or mathematical simulations. They may think that that model term is also quite intimidating. But this is another maybe earth shattering thing for people when I'm doing trainings is, you know, a model. What is a model? Well, maybe I'll ask you, Chris. If someone was to ask you, what is an AI model at its core? Like when you get your magnifying glass and you look at the AI model, what is it composed of? I guess maybe that's another way.

• 24:41Chris Benson?

  Well, the composed off threw me off a little bit. I mean, I think of a model in a generically, and this isn't specific necessarily to deep learning, but as a representation of an entity that is real in some sense. That doesn't mean it necessarily is physical.

• 24:56Daniel Whitenack?

  Yeah.

• 24:56Chris Benson?

  It represents the characteristics and stuff. But really, I mean, when we talk about a model in the context of deep learning and we're producing a model at the end, we love to talk about, you know, the word inference and such, but I think of it as a filter, honestly.

• 25:10Daniel Whitenack?

  Yeah.

• 25:10Chris Benson?

  Type of filter. And so you give it an input and it is arrived as a set function that produces an output based on that input. And until you change the model with further training, it's a filter that gives you a particular set of outputs. And that's down to earth, the way I think about it.

• 25:27Daniel Whitenack?

  I like the filter idea. I think that's something that people can grab on to in terms of their lexicon. Like everyone's interacted with a coffee filter, right?

• 25:37Chris Benson?

  Yeah.

• 25:37Daniel Whitenack?

  Or at least in most cultures, there's some idea of that sort of filtering or straining. And also like a data transformation, I think is a less scary terminology than model. But when I am teaching, oftentimes I'll ask this question and there's a bunch of ideas about like, oh, it's like a thing that mimics human brains, or it's like this, like very enchanted ideas of models, right? And eventually I break them down and I say, here's what a model is.

• 26:12Chris Benson?

  Oh, I thought you were breaking the person down for a moment there. You're breaking the model explanation. I was like, here comes Daniel, you know?

• 26:18Daniel Whitenack?

  Yeah. So eventually if you get down to the roots of it and you say, hey, so you have a Jupiter notebook, right? And you trained a model. Which bit of that code is the model? Well, I mean, there's some definitions. There's a function or a class that defines your model, right? But then at the end, you quote, save your model, right? Cause you want to use your model later. So if you save your model, what is saved? It's not sort of like spoken into reality with some incantation, right?

• 26:50Chris Benson?

  Yeah.

• 26:50Daniel Whitenack?

  Like you're saving numbers, right?

• 26:52Chris Benson?

  You are.

• 26:53Daniel Whitenack?

  What you're doing is you're serializing a bunch of parameters and the model in my mind is The parameters that you save, which are just numbers.

• 27:02Chris Benson?

  It's a static snapshot.

• 27:04Daniel Whitenack?

  Yeah, like a class or a function that defines how to use those numbers to either filter something like you're talking about, to filter it, to transform data, to produce certain types of outputs. It's really just a parameterized function, right? And the parameters plus the definition of that function is your model. There's no sort of like beyond that in my mind. Now, there's certainly a like

• 27:29Chris Benson?

  I agree with you.

• 27:30Daniel Whitenack?

  Other terms like these terms like meta learning. Oh, that's like super meta, right?

• 27:36Chris Benson?

  How many thousands of people though? You just took the enchantment right out of it, the whole mysticism.

• 27:41Daniel Whitenack?

  Yeah, I've destroyed lots of

• 27:43Chris Benson?

  There are thousands of listeners out there that are going to turn away from this field now. You've demystified it. You have taken the magic from it. It's no longer Hogwarts. We're all muggles still.

• 27:54Daniel Whitenack?

  Maybe we need some terminology from Harry Potter and our AI lexicon.

• 27:59Chris Benson?

  Excellent. I could do with that.

• 28:01Daniel Whitenack?

  I don't know if that crosses all cultures, but that gets to the idea that some of these terms, just from the start, like artificial intelligence model, gradient descent, and that's not a bad term for the method of optimization, right? It's actually a very appropriate term.

• 28:18Chris Benson?

  Very accurate, actually. I don't have a problem with that one because I think it describes what you're doing.

• 28:22Daniel Whitenack?

  Right. It definitely is a, can be a scary term though. It's like stochastic gradient descent or mini batch gradient descent. So maybe it's the scaffolding around those terms that like it sets people up for failure. If you're saying, I'm going to train my artificial intelligence model on accelerated hardware using an optimization scheme of mini batch gradient descent.

• 28:51Chris Benson?

  Yes.

• 28:51Daniel Whitenack?

  That whole like sentence there is like just excludes so many people because they're just like, oh, I give up.

• 28:58Chris Benson?

  You're probably right.

• 28:59Daniel Whitenack?

  Yeah.

• 28:59Chris Benson?

  And then on the opposite thing, without going that way, going for the cheap thing that doesn't represent it well. I think maybe, maybe the term, the description that just makes me cringe more than any other is the use of the word cognitive.

• 29:14Daniel Whitenack?

  Oh, cognitive. Yeah.

• 29:15Chris Benson?

  That's really gotten popular. Oh, and I just, oh.

• 29:20Chris Benson?

  If you're going to talk to me about-

• 29:21Daniel Whitenack?

  Yeah, the electronic brain sort of idea.

• 29:24Chris Benson?

  Yeah, if we're talking deep learning, don't talk to me about cognitive because I will turn off right away. I'll be like, this person doesn't know what they're talking about.

• 29:33Daniel Whitenack?

  I also listened to the NLP Highlights podcast from the Allen NLP Group.

• 29:38Chris Benson?

  Yeah.

• 29:39Daniel Whitenack?

  And their recent episode was really relevant to this topic because they were talking about language models and how they process language data versus like a human cognitively processes language. And actually, this is like very much like a huge area of research, which is not very well defined at this point. And even just thinking about the guest on the show, her name is Lisa Baneborn. And she did this work where she tried to match up certain layers of a neural network or operations in a neural network like attention with cognitive signals from doing

• 30:19Daniel Whitenack?

  imaging of brains to see if there was sort of a similarity with how a brain processed this information versus like a computer processes this through a modern language model. And she talks about how it was very hard for her to find any signal that had that sort of correlation.

• 30:39Chris Benson?

  Yeah.

• 30:39Daniel Whitenack?

  And eventually she found some like metrics within the world of machine learning interpretability that matched up a bit with the cognitive signals, but it was very, very far from clear cut. So I would recommend people, you know, look at, we'll include the link in our show notes, but if you're interested in hearing that discussion, yeah, this idea of matching up neural networks with cognitive things is very interesting and definitely an area of active research. We've talked about a number of these terms in our AI lexicon, and that's definitely one way to think about what AI now is doing with their series in this AI lexicon.

• 31:21Daniel Whitenack?

  But they're also kind of exploring various subjects that are thought provoking around how AI models behave, not just how we describe them, but how we describe their behavior and also think about how they behave and interact with humans. For example, they talk about those words, "smart," like we talk about like a smart fridge or a smart home or a smart city. What does it fundamentally mean that a thing is smart? I don't know, maybe that's another question to you. What?

• 31:52Chris Benson?

  Oh, that puts it right up there with cognitive with me.

• 31:54Daniel Whitenack?

  Yeah, yeah.

• 31:55Chris Benson?

  It's one of those words that's alluding, it's a marketing thing. It's alluding to this thing, as you just pointed out before the break, doesn't often show itself to be a reality.

• 32:05Daniel Whitenack?

  Yeah, and they talk about, you know, this is maybe driven by claims of technology companies to drive marketing in various ways. And in some ways, it's harmful because it, people want their stuff to be smart, right? But they don't think about like, where is this data going? What risk does this expose me to? Which we've definitely seen some issues with that over time in terms of IoT and smart devices and security and all of those things.

• 32:35Chris Benson?

  You know, that's a great point is that when you see smart something, you know, the something was already an existing thing and then they add "smart" to it. And it rather implies that something has changed. And usually that something that has changed is the use of data in some way, which begs all these questions that not only are we asking now, but we've talked about over many episodes. But the consumer isn't necessarily data savvy. They're not necessarily familiar with these technologies that we and our listeners are talking about week in and week out. So there's a danger there. There's a danger of what are you opening yourself to in the name of "smart"?

• 33:11Daniel Whitenack?

  I think a similar thing, maybe related to the smart discussion, they talk about function creep.

• 33:17Chris Benson?

  Yep.

• 33:17Daniel Whitenack?

  I know also people use the term model drift and other things. That's a kind of separate idea where the function you're, so model drift is when the functionality that you're doing with your model initially Matches the distribution of the data that you're putting into your model. But eventually that distribution of data actually changes. And so your model may behave in sort of interesting weird ways. But also you could think about function creep or function drift, which is hey, we created this facial recognition model to do X thing. Maybe it's to like ensure that people, you know, are identified securely when they enter a building or something, right?

• 34:02Daniel Whitenack?

  But then that's pretty easily, you know, COVID happens and then you can say, well, it's also a really good way to see if people are wearing their masks. Or, you know, do some type of more analytical analysis of this in a sort of surveillance type of way.

• 34:20Chris Benson?

  And that happens a lot as well.

• 34:23Daniel Whitenack?

  Yeah. What do you, what do you think that's driven by? What is the thought process and the discussions behind the scene when that happens?

• 34:29Chris Benson?

  I think things that I've observed is people go, well, it costs us nothing extra. The model we have for purpose A will actually serve purpose B as well. The decision to ever create the model was only analyzed against the circumstances around A. And so it's a way of circumventing many of the due diligence processes. In some cases, maybe even ethical considerations that are there.

• 34:52Daniel Whitenack?

  Yeah.

• 34:52Chris Benson?

  I think that happens quite a lot.

• 34:53Daniel Whitenack?

  Also, there's a lot of companies that provide AI tooling, but they're not the end user of that. Right? So let's say that I'm a company and I'm doing facial recognition technology. And I provide an API or some tool for companies to do facial recognition. And I release this to them. How could I ever understand how they're actually using this? Or how my technology is actually being used and maybe in terms of this function creep idea?

• 35:24Chris Benson?

  Yeah.

• 35:24Daniel Whitenack?

  Is there any mechanism to monitor that or to help understand how people are using the things that you're creating? Specifically AI models?

• 35:34Chris Benson?

  Yeah, I don't think there's any standard approach that I'm aware of at least. This reminds me, it's not exact, but it's close. A year or two ago, we were talking about how AWS was offering facial recognition services to law enforcement agencies. And there were some civil liberty concerns around that. And I believe AWS ended up pulling back from that as offering it. But the point is, is that AWS may not have felt as a service provider that they needed to evaluate that. But if you're talking about a tool that can be used in ways that may not be strictly ethical in every context, then there's a big question mark hanging over that entire thing is what are your responsibilities when you release such a capability to the public?

• 36:19Daniel Whitenack?

  I mean, at the end of the day, I, you can't have total control over what your end users do with, with their thing. Because at the end of the day, humans are tricky and deceptive and, you know, all of those things in, in various scenarios. I think you can, you know, maybe this gets to the principles and how you structure your licenses and that sort of thing as a provider. For example, I know there's well-accepted and widely-used guidance from the United Nations around malicious use of technology to oppress indigenous populations, right? And so this is language that you can include in like licenses that you sign with, with companies.

• 37:02Daniel Whitenack?

  And certainly like a license and their company signing, you know, a license on that. That doesn't mean that they won't do that or oppress some population or something like that.

• 37:11Chris Benson?

  It's a self-regulatory kind of approach.

• 37:14Daniel Whitenack?

  Yeah, but at least there is a precedent to where like, if you are aware that something like this is going on, then, you know, you have in your license, an agreement that allows you to, you know, terminate things or recoup costs that maybe you could push back to that population or what, you know, whatever the thing is. There could be a mechanism around it, but if you don't have that, then like, hey, you're just not thinking about it upfront. That's maybe a poor excuse. Like, I didn't think about the scenario where this could be used in this way.

• 37:48Chris Benson?

  Yeah, I think that draws inspiration from the software world because, you know, often we see software licenses, including open source licenses that start with a particular intent. And it's not until you figure out whether or not that's appropriate for the situation and, and there's some learning there.

• 38:04Daniel Whitenack?

  There's a whole variety of these articles which I would recommend, you know, people check out. There's new terms in there that, you know, me as a technical person and not, you know, not having taken a, an English class since high school, I wasn't familiar with, but it was good to learn about those things. And glad to see people writing these sort of thought provoking articles in, in this way. So we'll include links to a number of these and, and the series in our show notes. So definitely check it out. But one of the things that we like to do on these fully connected shows is share with people some learning resources that we've run across. Because as you mentioned, Chris, things are

• 38:45Daniel Whitenack?

  moving so rapidly. People use all sorts of fancy jargon, which is confusing. So having a good set of, of resources to turn to is, is really important. One of the ones that I found, actually, I, I don't know if it was yesterday or today, was a new free book, PDF book that's posted onto archive from Jeff Heaton, who's teaching a course at Washington University in St. Louis, entitled Applications of Deep Learning. And you can just go straight to the archive and download the PDF. I was really impressed with my initial inspection of this resource. So it's Applications of Deep Neural Networks with Keras. And it seems

• 39:30Daniel Whitenack?

  like there's this sort of very consistent and practical treatment of everything from kind of preliminaries of how to read in a CSV file, through to like, how do I interact with TensorFlow to do various things, all the way down to like reinforcement learning and setting up environments for reinforcement learning. And all of this is treated seemingly, you know, very consistently with Keras, and very much in a code-first sort of way. So as soon as you start scrolling through the book, you'll see Very explicit sections which are, hey, here is the Python code that does this thing.

• 40:13Daniel Whitenack?

  And here is the expected output from that Python code. And of course, there's, you know, equations and figures and stuff in there as well. But I like that there's this very clear call out of how this is code based, how it's all with Keras. I haven't gone through the whole thing, obviously, since I just found it in the last couple of days. But I know I'll use it as a reference, but I think it would also be a good alternative for people that are out there, maybe thinking, hey, what is that sort of comprehensive resource that I could turn to to really get into this field? And maybe, starting with a little bit of Python knowledge, work into some of these more complicated topics like adversarial networks or convolutional neural networks or other things with TensorFlow and Keras.

• 40:59Daniel Whitenack?

  I think this would be a really good one to check out and maybe include in your list of things to work through.

• 41:04Chris Benson?

  I agree, and I was not aware of this until you mentioned it just now. And I downloaded it as you were talking and looking through it. It's very current. It's fall of 2021 and we're in August of 2021 as we're recording this. So it's a recent resource and I'm looking through it and it looks fantastic.

• 41:21Daniel Whitenack?

  Yeah, it's like 500 pages.

• 41:23Chris Benson?

  Oh, yeah, it's huge and it covers so many great topics. So yeah, thank you. I'm going to be going through this after, after we stop recording this episode.

• 41:30Daniel Whitenack?

  There's definitely other ones that take this approach. Like I'm thinking of this dive into deep learning resource that is very comprehensive, but also very much code based and provides code examples. And I think this is really useful because you get your hands on very quickly and are able to explore things very rapidly. So yeah, definitely check it out.

• 41:50Chris Benson?

  August 10th, two days, as we record this, it was released two days ago.

• 41:54Daniel Whitenack?

  Oh, look at that. I'm, I'm really on top of, really on top of Twitter here. People must have liked it on Twitter. It got into my feed. So thank you, Jeff Heaton. If you're, if you're listening, come on to the show sometime and talk to us about how the class went.

• 42:08Chris Benson?

  Absolutely.

• 42:09Daniel Whitenack?

  We'd love to hear about it. If you're interested in these sorts of resources or topics that we talk about on the show, maybe check out our community at Practical AI. You can join a Slack channel where there's discussions on these sorts of resources and other things. You can find that at changelog.com/community. We're on LinkedIn and Twitter and other places. So find us and let us know what you're thinking about the show and start some discussion around various AI topics. I really enjoyed today's conversation, Chris, talking about words and what they mean.

• 42:41Chris Benson?

  I did too. It was fascinating and for me is, is participant. And I'm just absorbed already in this book. You've, you've lost my attention, Daniel, because I'm

• 42:49Daniel Whitenack?

  I totally lost you. Yeah.

• 42:51Chris Benson?

  I'm totally into the book now. So yeah, go download it, everyone. It's free out there on the archive. So, uh, cool. Great show.

• 42:58Daniel Whitenack?

  Yeah. Thanks, Chris. See ya.

• 42:59Chris Benson?

  See you later.

• 43:03Speaker 03

  Thank you for listening to Practical AI. We have a bundle of awesome podcasts for you at changelog.com, including our brand new show, Ship It with Gerhard Lazo, a podcast about getting your best ideas into the world and seeing what happens. It's about the code, the ops, the infra, and the people that make it happen. Yes, we focus on the people because everything else is in implementation detail. Subscribe now at changelog.com/shipit or simply search for Ship It in your favorite podcast app. You'll find it. Of course, the galaxy brain move is to subscribe to our master feed. It's all changelog podcasts, including Practical AI and Ship It in one place. Search changelog master feed or head to changelog.com/master and subscribe today. Practical AI is hosted by Daniel Whitenack

• 43:48Speaker 03

  and Chris Benson with music by Breakmaster Cylinder. We're brought to you by Fastly, OnStarfleet and Linode. That's all for now. We'll talk to you again next week.