Explainable AI that is accessible for all humans

We are seeing an explosion of AI apps that are (at their core) a thin UI on top of calls to OpenAI generative models. What risks are associated with this sort of approach to AI integration, and is explainability and accountability something that can be achieved in chat-based assistants? Beth Rudden of Bast.ai https://bast.ai has been thinking about this topic for some time and has developed an ontological approach to creating conversational AI. We hear more about that approach and related work in this episode. Sponsors: Fly.io https://fly.io/changelog – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog https://fly.io/changelog and check out the speedrun in their docs https://fly.io/docs/speedrun/. Fastly https://fastly.com/?utm_source=changelog – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorshipTypesense https://cloud.typesense.org/?utm_source=changelog – Lightning fast, globally distributed Search-as-a-Service that runs in memory. You literally can’t get any faster! Featuring: Beth Rudden – LinkedIn https://www.linkedin.com/in/brudden, X https://x.com/ibethruddenChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Bast.ai https://www.bast.ai/Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:07Speaker 00

  Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related technologies are changing the world, this is the show for you. Thank you to our partners at Fastly for shipping all of our pods super fast to wherever you listen. Check them out at fastly.com. And to our friends at Fly, deploy your app servers and database close to your users. No ops required. Learn more at fly.io.

• 0:42Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I'm a data scientist with S I L International, and I'm joined as always by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris? Doing well. How are you today, Daniel? Oh, man, so much better than last week. As you know, I was sick last week when we were supposed to record, so sorry for the skip of the week, but I'm happy to be back here and with a super relevant topic around AI systems and explainability, the delivery of AI systems that are explainable. We have with us today Beth Rudden, who is CEO at Bast AI. Welcome, Beth.

• 1:25Speaker 02

  Thank you for having me.

• 1:26Daniel Whitenack?

  Yeah, yeah. We were just talking before the show about like this craziness that we're experiencing around the hype of these AI systems, which maybe are just like a webpage that connects to OpenAI. But you were talking about how you've been thinking for quite a while about explainability and accountability of AI systems. I'm wondering if maybe you could start out by just giving us a little bit about the journey you took to landing in that space. Like, how did you get interested in those topics specifically?

• 2:01Speaker 02

  I think that a really good place to start is, you know, understanding 2012. The Harvard Business Review said that the data scientist would be the sexiest job of the 21st century.

• 2:13Daniel Whitenack?

  Yes, which is why some of us on the call are data scientists.

• 2:20Speaker 02

  So I was working at IBM and I was a pleaser. So I went after the Information Architect certification. But I had a couple of friends that were mathematicians and statisticians and, you know, really data engineers and, you know, DevOps people and software engineers. And, you know, they didn't really want to be an information architect. They didn't want to be an IT architect. So we, you know, rinsed and repeated experiential certification to be able to say, well, how, how can we make sure that we're making data scientists that actually know how to use data in the scientific method to solve business problems?

• 3:02Speaker 02

  And so we started that and it took us about six, seven years. It's accredited through Open Group, anybody can get it. And it's hard because you actually have to submit, like, you know, depending on your level one, two, or three, you have to submit a, you know, several projects that say, you know, here's how I took my problem and put it into a hypothesis that could be tested. And then here's how I negotiated with my business stakeholder. And then here's how I like, you know, showed my results. And then, you know, as you get further and further along, here's how I integrated my model into production. And I think that's when things get a little crazy and people are like, wait, what? How do I do that? You know, here's my Jupiter notebook.

• 3:47Speaker 02

  Isn't that great? But I've been writing or, you know, looking at how to deliver AI, but I've been doing a lot more on the linguistics or the semantics side and for probably about 15, 20 years. And if you look at, you know, how the NLP work is really, you know, a lot of people are like, oh, hey, I pulled down this thing from Spacy. I can write NLP, right? You know, I just called this like cosine similarity kind of, you know, model. I'm, I'm good to go. And I was an archaeologist in, you know, the late 90s. That's what I actually got my degree in. I did Greek and Latin and spent some time in Italy.

• 4:29Speaker 02

  And when you're learning languages, you know, you're learning declensions and you're learning, you know, etymology and you're learning stemming and lambitization and tokenization and, you know, all of the text preprocessing. So I was always the squishy human data scientist. I was the one that was studying languages and doing semantics. So I think it was twenty fifteen, Andrew Ng's like, oh, hey, if we use GPUs or graphical processing units, we can process all this structured data like really, really fast against these statistical models. And so a lot of people, I think, forgot about entity extraction and, you know, ontologies and the semantic web. And, you know, I use the OWL, which are formal knowledge graphs.

• 5:14Speaker 02

  And I hopefully am not speaking great to a lot of people, but really looking at the language side as opposed to the machine learning side. And that understanding of semantics is like, it's really put me in a great position now because all of the statistical models. NLP has three, I kind of chunk them into three things. NLU, natural language understanding, NLC, which is classification, which is your prediction, and then NLG, which is your generation. And prior to having access to GPT, we were generating language old school. And it's super hard. I mean, it's really, really hard. Like, if you think of like pronouns and, you know, trying to say, you know, John was, John

• 5:59Speaker 02

  was the guy, you know, who referred to him and then he took the notch or the wrench from Joe, et cetera, et cetera. I mean, just, it's so complicated. But now that we have a really good natural language generator, I'm like kicking butt because I have the semantic side, so I have really, really high NLU or natural language understanding. Because how an ontology works, and I try to remind people of this, an ontology is the study of the nature of your reality based on the language that you use. So when you use language, I'm understanding your reality. And so I map that into a knowledge graph and then take that language to understand whatever

• 6:43Speaker 02

  you're going to say against it. So I have a map. So when the natural language generator generates all the variation that I need in order to put into my machine learning models, and then I have a map to make sure that I understand what kind of things you actually want to talk about. I can create conversational AI that has lineage and provenance, that has the source of where you're starting from. And then I use all of the GPT generators to effectively generate the fluff or the syntax around whatever entities that I extract about whatever you want to talk about. If you guys could help me break that down into better English, that's what I've been trying to do for the last couple of months.

• 7:27Daniel Whitenack?

  That's awesome. I would love to, so one reference point in my ontology of this space is people sort of talking, and we even talked about this in our last episode with Brian McCann from you.com about grounding, right? Like, some people are thinking about, okay, like, I can generate text or a response to a user query by looking at some external knowledge and grounding my response in that sort of context. How would you compare? So like, one way you could do that is to say, okay, I'm gonna go, I'm gonna pull an article and like, insert a paragraph from that article in a prompt, a natural language

• 8:10Daniel Whitenack?

  prompt to a language model, and that's the way I'm inserting knowledge into like, my response. Here, you have this sort of concept of a knowledge graph and entities and like ontology. Would you consider that grounding? Or is it slightly different in terms of like, what you're talking about with this ontology and kind of bringing that external knowledge into the generation?

• 8:34Speaker 02

  You know, the nice thing about software is it's multidirectional. So how the conversational AI, the product in, just a quote here, so we named our company, or I named my company VAST AI. It is after the Egyptian cat goddess, and we build conversational AI technology. So we build cats.

• 8:55Daniel Whitenack?

  Ah, nice.

• 8:56Speaker 02

  And we really wanted to distinguish from bots. And, you know, having the word cat makes a lot of sense, at least in my mind, because I've been, you know, overriding the bot. But the idea is that the conversational AI technology includes a data pipeline. So bring your own data. Let me take a book that you have written. And then the cat through the data pipeline ingests that book and then puts the entities into the ontology. And that can be done both supervised and unsupervised. I I'm a big proponent and I hope we can get into this a little bit of AI is is to augment human beings. It's really to help us understand and I'm always like, why can't we carry a pocket

• 9:41Speaker 02

  brain like we carry a, you know, a pocket GPS? So the cat ingests the book and then those entities go into the knowledge graph and then that those entities are in a concept hierarchy. And they carry, well, it was from this paragraph on this page in the book. So that carries the actual provenance of where that entity was and the understanding of that entity within the concept hierarchy. Then when the cat, you are interacting with the cat with the conversational interface, the cat will be able to respond using those entities. And so it will show you where it got its response from.

• 10:23Speaker 02

  And so it's predicting the response based on your question because I have that high degree of NLU. So I take your question and I do text preprocessing and I match it against the entities in that ontology. Or if it's not in that ontology, we have a series of orchestration to send it to um a couple of various different places that we had to create. The way that we handle uh toxicity is something that I'd love to talk about too, just because I think the way that we're handling that is very, very elegant and fun. And the idea is that we wanted to have fully explainable AI. We wanted to show people how they could ingest a paragraph.

• 11:07Speaker 02

  And then be able to communicate with the conversation or communicate with the AI to understand how that paragraph is being understood in the relation of what the person is asking for.

• 11:19Daniel Whitenack?

  Yeah, and maybe just to give an example of this, I love the the way that you frame this in terms of like reaching out to ontology that's hierarchical and you can kind of ground citations in that as well. So like, just to give an example, let's say that I had a book, maybe it's a novel. I'm reading right now. The, I don't know if you ever read it, The Cuckoo's Egg by Cliff Stoll. It's about like a hacker at the Lawrence Berkeley Labs way back in the day. It was really interesting. Anyway, like, let's say I have that book and I put it through this data pipeline. So I've got my ontology and I've forgotten like, oh, did Cliff like reach out to the NSA?

• 12:03Daniel Whitenack?

  Or was it the CIA when he was talking? And I asked the question, you know, did Cliff reach out to the CIA or the NSA? What would happen next in that? Like, how would, how would that query, the processing of that query differ in that example as compared to maybe like other types of, of ways of handling this?

• 12:23Speaker 02

  So instead, I mean, in that case, we would have, you know, the direct path where we could say, okay, well, Cliff is a character in the book and it was the CIA, not the NSA. So I know that for a fact in the ontology. So then, you know, I could do it two ways where I could answer you directly. And we, I don't know if you call it cheating a little bit, but we have a corpus. So anything that's answerable, that is really, really easy answerable, we stick into OpenSearch, which is just a form of elastic search. So then we just pull that, give you the answer. We know with 100% certainty that it was the CIA. Done. And all of these scores, we have about 50-ish

• 13:04Speaker 02

  different models, depending if you, how you count them, that is run through our orchestration system. So each one of those models has scores, you have targets, configurability, you can expose different hyperparameters, et cetera, et cetera. Or we could take the Cliff entity and the CIA and NSA and we could do prompt engineering. And we can give you, like, if you wanted to say, give me Cliff's reply and tell me, was it the CIA or the NSA? And then you could go like, you know, ask it to generate a response for you. And one of the things that we're really playing around with is conversations should be interactive. So we want the cat to also engage the person.

• 13:46Speaker 02

  So we could say, um, oh, Cliff was part of the CIA and you wanted me to generate like a response that Cliff would give. Here you go. And what else would you like me to do? Would you like me to generate like some books that Cliff would have written in the style of Cliff? So you can kind of really start to do that engagement too. And you know, I I use the word lineage and providence, um, but it's really attribution. And when you're starting to attribute things to the right, to the right source system, everything changes. And anytime I like, you know, show some of the cats to people. One of their first responses is like, let me put my own data in it.

• 14:29Speaker 02

  And that's exactly what I want to instigate with humans is not having the black box algorithm do the answering. Just have the black box algorithm do the generating. And I know that a lot of people are super excited about using them. I would really caution about creating them. Because with the generative AI, it's just gonna generate based on syntax, not based on understanding. And I think that that's like the biggest thing that I want people to hear. There's no sentient, there's no sapiens, there's no consciousness. And I think that that's all a distraction for the amount of compute that these models really take.

• 15:14Speaker 02

  So I'm like, can we make it like a utility and everybody uses it sort of like a dictionary? And then, you know, we're good to go or like a thesaurus.

• 15:25Speaker 02

  And I just, I I really do think that when you're using the generative transformer to generate transformations. That's the big difference that I'm trying to get people to see is like with the ontology as the map. And, um, actually GPT gave me this analogy and it's very good at analogies and metaphors because of how it's built. And, you know, clustering and all of the things that happen be- behind the scenes. So when you're using an ontology with our conversational AI or with our cats, and you own a toy store, if you want to ask that cat about any toy in your toy store, you'll have that ontology to tell you about any toy in your toy store. If you ask the cat about a toy that's not in your toy store,

• 16:10Speaker 02

  it will tell you it's not in your toy store. If you do the same thing to ChatGPT, it's going to tell you about a toy that doesn't exist.

• 16:27Chris Benson?

  So Beth, that was a fantastic explanation, and I'm learning a lot. Daniel is quite the expert in this area himself, but I'm not. But you, as we were coming into the break, you made a point that has been weighing on me for the last moment or two as you were finishing up. And that was, as you are a user and you're interacting with this capability, remarkable capability that's really taken all of our attention this year. You pointed out that it's really important for that user not to infer intelligence, not to infer a consciousness in that. I would argue that that, for the typical user out there, someone who's not in the AI space and doesn't have an understanding of these, that's a really hard ask.

• 17:11Chris Benson?

  And I'm definitely, as you know, with this year with GPT, everything chat, GPT-4 has been out this last week and as we're recording this. And I'm talking to a lot of people and I think they're really struggling with that. So you, you kind of gave that direction, but I think that's easy for us. I think that's a tall order for people not in the space. And a lot of people, a lot of our audience are people kind of coming into this and learning about it. Can you provide some guidance in how you keep that separate? And what it means and how you should use this new capability as people are now engaging? Because it's changing the way we're all, you know, operating day to day.

• 17:52Chris Benson?

  Even non-technical people who have never really done any AI are now going to ChatGPT and things. And we're really at a moment where people need to understand how to appropriately engage this brand new technology.

• 18:06Speaker 02

  I think that it's a combination of things, but you know, I think that TLDR, go out and use it and use it as much as possible and ask it about yourself. Ask it about things that you know. And, you know, the way to really understand how something works and, you know, remember, we're in the realm of cognitive science that binds philosophy, psychology, and computer science. And I think that really, when you understand that it is a generative transformer, it generates transformations. It does not have the understanding to have consciousness or sentience. It doesn't understand what you're saying.

• 18:50Speaker 02

  It's a stochastic parrot. It mimics language. So it mimics what you're saying based on the syntactical rules of that language. And it's incredibly good because it's been fed a huge amount of data.

• 19:07Chris Benson?

  So if I'm having a conversation at work with somebody who's not a technologist.

• 19:12Speaker 02

  Sure.

• 19:12Chris Benson?

  They might be in a marketing department or something like that. How does that change how they're engaging in terms of how they should be thinking about it? Because I would argue that that's a tall order to actually get, you can say that to someone and they'll say.

• 19:27Speaker 02

  Right, yeah. Ask it about a product that you are not selling, that you're not marketing. You know, ask it about something you know.

• 19:34Chris Benson?

  Okay.

• 19:35Speaker 02

  And you know to be true. And ask it to say, you know, I, I would like you to market a blue tomato. We have blue tomatoes. Blue tomatoes grow on trees. Could you market that for me? And it will give you marketing for blue tomatoes that grow on trees. And so I really want people to come from a space of abundance, not scarcity. I really want people to think about what do they have right now? And what every human being has is their own experience. And what this AI has been trained on is a very small number of people's experience who have been on the internet and been writing on the internet.

• 20:19Speaker 02

  And so my opinion is that every single human being is already impacted by AI. They should be using it. And they should be using it in a way that, you know, I used it to help my daughter come up with an analogy on reciprocals. I asked it to come up with a good metaphor in order to explain, you know, what an ontology does. You know, there's so many different things that you can use it for. And I, I really think that the best way that people can do is go out and use it and ask it about things that you know. Many authors are like, oh, so I wrote ten books, not four.

• 21:01Speaker 02

  Ha, you know, I mean, people are starting to see that it's going to generate the next proper noun, the next predicate, the next syntactically correct, and, and I'm, you know, doing it in a sentence, but it's so smart because it has been fed so much data. But here's a myth. You know, I talked a lot technically about using ontologies and knowledge graphs and concept hierarchies and, you know, all these things. But here you go. All of what I just talked about can run on something that was like the very first iPhone one. The myth, you do not need big data and big honking machines to create AI.

• 21:46Speaker 02

  And I would ask, who does that myth serve? And if everyone could understand how to use the data that they have that is special to them, that they understand that they, like, if you take your grandfather's journals and put them into an AI so that you can have a conversation with your grandfather, this is what the technology is enabling us to do. And we want it to be distributed to every human on earth. Because every human on earth is being impacted by AI.

• 22:17Daniel Whitenack?

  Yeah, and I don't know if you can talk at all. One of the things, I love how you brought out this element of people being able to bring their own data to the table, kind of combined with the fact of them being able to run this, maybe even on their own hardware. How shocking would that be? And I love that also because like, I think that there's a real concern that I've been having over time of just how Western and English focused most of this conversational AI is. And the fact is that like, we come to the NLP table with these biases that are say like, oh, wouldn't it be great if every language community in the world had a translated version of Wikipedia?

• 23:02Daniel Whitenack?

  And that concept makes sense to us, but the reality is some language communities, they don't want that. It's explicitly not how they use their language and they want to use their language in a community setting, maybe for storytelling, or maybe for like, whatever it is. And they would rather bring a different kind of data to the table. So I think that that also helps in this regard. I don't know if you've also seen like, in the ontology space or the knowledge graph space, how do you think about like, I guess bias and like, the availability of data? Because that's a big topic with these large language models, right?

• 23:42Daniel Whitenack?

  Is like, if you're just using them for generation, they come loaded with what they're trained with, right?

• 23:48Speaker 02

  You know, I was stunned at how quickly the models are able to statistically generate the language. And, you know, just, we used to make fun of like, you know, natural language processing statistically generating language, like, you know, it's a corant and so it's such an oxymoron in so many ways. There's actually a really great article by Karen Hayo about the Maori people and what they're doing with artificial intelligence. And I'd love to link that just because it was from MIT and it's fantastic review and really speaks to what you were saying about that. As far as bias, again, I'm going to go back to cognitive science, you know, philosophy, how do you know what you know?

• 24:33Speaker 02

  Psychology, how do you make sure that you are not using your powers to manipulate humans? Seriously, like just put some ethics there. And then computer science. So when you're talking about bias and you know, there's a cognitive codex and it's like 188 cognitive biases and counting. And one of the best ways that I did this when I was still in IBM is I started the Trustworthy AI Center of Excellence and many of my peers are still, they're so strong and what they're doing is amazing. But what I wanted to do with bias is we did

• 25:12Speaker 02

  some modeling on the Titanic. And we did some predictions on whether somebody was going to live or die based on their class, in order to show the social bias of the time, because the person in steerage would never have gotten a lifeboat. And I use that explicitly to talk about bias. And what you said about the very small, you didn't use the word homogeneous, but like Western kind of culture that we're sort of codifying into this AI, we have got to have a wider variance. We have got to have more diversity. And that's why we really need to be able to give everyone the ability to build their own without having

• 25:56Speaker 02

  to build their own generative model.

• 25:59Chris Benson?

  Could you talk a little bit about like how to do that? Because this is a topic that we've talked about in different ways over a number of episodes. It's very hard to get it out there. It is definitely not an equal world.

• 26:11Speaker 02

  That's right, access.

• 26:12Chris Benson?

  Yeah. Can you talk a little bit about access and how you create that and how that becomes possible?

• 26:17Speaker 02

  Um, well, shameless plug, I'm looking for funding and investment.

• 26:23Speaker 02

  But, but you know, I think that the ability to use the combination of like knowledge graphs and semantics and you know, being able to access these generative models. And one of the things that I did with the orchestration and the reason that I use sort of the corpus driven and dump a bunch of stuff into OpenSearch is to make it small enough and accessible by as many people as possible. So just use the access to the generation to generate all the variants that you need, but eventually you're done. You don't need anymore and you have that stored in a corpus. So you can access that as much as you want.

• 27:03Speaker 02

  And so it's really about like, how do we use the generative AI more as a utility that everybody uses instead of creating, I call them cheese graters because that's how I think of the generative model is they cheese grate the text and then they sort of glue it back together or stitch it back together with duct tape or you know, whatever. But I mean, it's codifying so much of our Western notion of ideas. If you go to Aboriginal societies, their construct of time is entirely different. Like if you're facing the West or the North or the East, their concept of time is different. And that's expressive in their language.

• 27:47Speaker 02

  So to think that we have created a generative model that can encompass all of our world is, is not correct. We can do so much more if we have a wide variant. Have you guys heard of um, the diversity prediction theorem or the wisdom of the crowd?

• 28:06Daniel Whitenack?

  Yeah.

• 28:07Speaker 02

  To me, it's the secret of the universe. Like the wider your variants, the more standard your mean. Like the closer to truth that we want to get to, the more diverse human neocortexes that we need to get there. So I'm a big proponent of, you know, to really answer your question, Chris. We need to make the generative model something that is is accessible and and OpenAI has done a really great job of like making it accessible to a wide range of people. But I was talking to, you know, my parents today and they were like, we don't even know how to access that, but I think I went to Google and it might have done something because it gave me a weird response. So I shut it down and then I tried to go to the other

• 28:52Speaker 02

  thing. I was like, oh, good. So we we need to be better about really making sure that people understand that they are accessing just a generative model. That's it.

• 29:04Chris Benson?

  I think that's one of the one of the challenges I see is we're here in this AI community, you know, we're a tiny little slice of that in this episode as we talk. But at the same time, I participate in other communities that are not technical at all. And the other participants in those communities are not technical. And I think that's the challenge is trying to do exactly what you said with people who otherwise not just don't have access, don't even know it exists in a lot of cases. So.

• 29:32Speaker 02

  Yeah, I used to say, and a friend of mine reminded me of this a long time ago, but there's no hand waving in math. So if somebody is not explaining how they got to the prediction, or how the model works, or saying it's proprietary, or shoving a bunch of data into a neural net to have it guess the features, you know, engineering, they're probably hitting the easy button. They're not doing the work. And I come by that honestly because I think that people need to understand there's no hand waving in math. We need to stop thinking that just because it's AI or just because it's statistics. You've heard the Mark Twain quote?

• 30:14Chris Benson?

  Is it a statistics quote?

• 30:15Speaker 02

  Uh huh.

• 30:16Chris Benson?

  I probably should have.

• 30:17Speaker 02

  Lies, damn lies on statistics.

• 30:19Chris Benson?

  Yeah, that's right.

• 30:20Daniel Whitenack?

  Yes, yes.

• 30:22Speaker 02

  So it's just statistically, 2013 ish, 10 years ago, goodness, Jennifer Gold Black, social scientist, gets on the TED stage, tells the world that she can statistically predict whether you have done drugs or not based on five of your likes on Facebook. And I was like, hallelujah, everybody understands, everybody sees it, right? Everybody understands that like, we can now predict these things statistically if we have enough data. And no, I I I still don't think that people can get that and we need to teach. Like I taught my kids probability through poker.

• 31:05Speaker 02

  Like we we can teach this to people so that they understand that. It's only statistically accurate to a certain probability. So if it's ninety seven percent accurate. What does three percent look like? What's your test reliability for that three percent? Is that three percent gonna give you that same answer every single time? And if not, it's not science.

• 31:34Daniel Whitenack?

  I love so much about this conversation and one of the things that I was thinking about in my own context is like my own tendency to not give users enough credit. So like one of the things that happens when like we anthropomorphize AI and like talk about it in these different ways is. You know, there's a tendency to like maybe think it's always right or it has more like you're talking about more intelligence than um it really does have. But I've also found where it's whether it's like. Family members in my own life who aren't involved in the AI world and they're using chat GPT. A lot of times they interact with it. More responsibly, I feel than some colleagues in the AI world in the sense that like my.

• 32:19Daniel Whitenack?

  Like my brother in law, Jack, I don't know if you're listening, hey, uh if out there like he um. We were talking last night over tacos and he had like used chat GPT to write up. Some like speech or something that he was giving at work. And I'm like, oh, so you like wrote that with chat GPT? And he's like, yeah, like I used it, but I what I do is I don't like just. Have it generate it for me, I'll just like type as fast as I can and just have it rephrase it into something good that's grammatically correct. And I'm like, wow, that's like, yeah, go for it. That's really good. Like, that's awesome, because like, that's a great way or like I'm thinking of um teams that we work with in in India in my day job, I was talking to someone and like.

• 33:03Daniel Whitenack?

  Some people would say like, oh, we can't like just output machine translations because like they won't post edit them and like make them. Good or like look for corrections and in fact like. You know, translation teams are working with in India. They know it's a machine translation. They're just happy they don't have to type as much because typing is really difficult in their language, right? Like, they're fine to post-edit it. So, yeah, I'm wondering if you've seen this as well, and like, if you have any recommendations, specifically after working with users in conversational interfaces, which can seem kind of like human-like, like, it's like you're having a conversation. How do you set up an interface? How do you set up a system such that it, like, produces useful

• 33:47Daniel Whitenack?

  behavior and, like, promotes the right type of usage?

• 33:51Speaker 02

  You know, I started playing with very earlier versions of GPT. And so we strung them up in Slack. And we did that on purpose because we didn't want to deal with identity access management and all the other stuff. And Slack's a great interface for plugging things in. But it's also really something that, you know, I was the anthropologist, and when we installed Slack in the largest enterprise in IBM, I watched, you know, the people going, "Oh, my gosh, what's the protocol?" And really, when you deal with, like, really senior executives, they're like, "Wait, this is persistent? This is kept forever? What do, you know, what are we doing?"

• 34:27Daniel Whitenack?

  How do I respond to this gif?

• 34:29Speaker 02

  That's right, that's right. And so, like, you know, I had, I had a lot of what you would call training in trying to get people to understand this new modality of communication. So, we were playing around with bots, and we wanted the bots to talk to each other. And so, we use the cats now to test out what we're doing. And, you know, we talked a little bit earlier about, like, setting up an iframe or a webpage that just, like, you know, strung up to OpenAI, and you can ask it any question you want. But if that AI doesn't give the answer that you like, and it causes your customer to not trust you, that's a big deal. So you really want it to be tested.

• 35:13Speaker 02

  And so we use the, we use the cats to test the cats, or the cats to make kittens, or the cats to test the kittens, or the cats to test the intense, or, you know, and, and this is the, the joy of having some of this automated. And when we were, you know, back in the day when you used to do like conversational AI, and that you, you would do like, you know, either dialogue flow or corpus driven, it was always the IT group that had to do, like, give me a hundred variations of how somebody would say this. Give me a thousand variations of how somebody would say that. Give me sixteen synonyms for this. Give me seventeen synonyms for that. So, we're using it all the time. Because again, it's great at generating variations. One of the ways that I used it with, um, teachers and students,

• 35:58Speaker 02

  and I get to work with this amazing university, Maryville University, that is truly transforming education, and they're doing such amazing things with my friend Phil Kumarni, who used to be SVP of Innovation at Salesforce. We're doing great things there. And I got to do a fantastic, um, workshop with all of the teachers and the faculty, and they came to my session really kind of skeptical. And they left my session going, oh my gosh, I now understand how to use this. And what I did is I had ChatGPT list out fifty things that a teacher does all day. And then I had ChatGPT list out fifty things that a student does all day.

• 36:40Speaker 02

  And then I had the teachers and the students, I had a hundred and thirty-eight people on a Miro board, working together for fifteen minutes. And I'm like, what are you gonna eliminate? What are you gonna raise up? What are you gonna reduce? And what are you gonna create? And so give people an understanding of what the technology does, and then the messy middle between the skills that you have and the title, or the role that you have. It's what do you do all day? Like, what do people do all day? I'm like Richard Scary, 1968. But like, when you're doing that, you're like, wait a second, I don't, and, oh gosh, when I was a mom, or like a mom of younger kids, I guess I'm still a mom, even though

• 37:21Speaker 02

  my child is 18, or one of them is, anyway, I digress. Like, I would sit there and I'm like, oh my gosh, my head is hurting, what do I do for dinner? Chat GPT, give me a recipe with chicken and broccoli. You know, like, it can be so useful for so many people to just generate what the idea needs, especially when you're tired, or you're exhausted, or I definitely wouldn't want to fire your entire marketing team, but, you know, you, you, because you want to keep it on, like, on cue, but you really can use it to really augment your business and augment what you're doing. Just keep it in the realm of fiction and creativity,

• 38:03Speaker 02

  and, you know, those kind of things. And, and we haven't even talked about, you know, some of the, some, some of the art and the, the creative expression. And then, I'm a huge fan of, like, you know, especially for people who don't code, I'm like, go ahead, program in it, like, get it to render some code for like a graph is, so that you can see like a visualization. And code's great, because it kind of works or not.

• 38:30Daniel Whitenack?

  True. We've talked a lot about this sort of idea of knowledge graphs and ontologies as a reference that's domain-specific and known in combination with generative models. Do you think there's a parallel in the sort of image, vision, audio space where like,

• 38:49Speaker 02

  I hope so.

• 38:50Daniel Whitenack?

  I, I imagine, you know, groups are like, hey, I need to generate a new design for a webpage, or I need to fill this empty room with furniture, and, you know, I could generate a couch, but it'd be really nice if I knew that this couch existed and I could order it online, and it's an actual couch that exists, because otherwise I could sell this design to my client, and now, like, I can't source the couch for my room. What are your thoughts on that in terms of maybe extending some of your ideas about combining knowledge graphs with generative models to more modalities than just text?

• 39:31Speaker 02

  We're in the realm of like, most human beings can't see the difference between 4K and 8K. But you ask any artist to kind of look at, and I, I've done a ton of work with my, you know, just playing around too. It's like, it's kind of off and you don't know why. So, but it'll, it will get better and better and better. So I think that what I hope will become more valuable is attribution. And AI that can give actual attribution. And so you could do that with anything visual, you could do that with video, you can do that with, um, you know, any sort of, you know, as you were doing it with like, kind of like a design or shopping. I was once having a conversation and again, this is the, the geek in me, but I'm like,

• 40:16Speaker 02

  an AI can produce its own architecture and its own architectural diagrams or its own ERD. And that's where I think we should be going is for the AI to explain how it works in and of itself. And, you know, that's where we're gonna potentially get to, you know, cognitio ergo soon. But, um, I, I think that it would be really cool if AI can start to think in terms of that, that three dimensionality. And I think that if you can get the AI to design how it's functioning and how it's working within itself, that's going to be far more valuable and again, far more trusted because that's something that you're gonna wanna actually build a relationship with.

• 41:00Speaker 02

  The big problem that I see with the generative transformers that are pre-trained right now is that they're pre-trained on data that was harvested without people's consent. Which means that that data was potentially put there, you know, I've been, I thought it was my job to lie to all of the search engines for the last, you know, at least 10 years. So I mean, how good is that data? And my hypothesis, and I've played it out many times, is like when people interact with my cats, where it's AI that they trust and AI that they know where the data comes from, it's an entirely different experience than, you know, back to your initial question, Chris.

• 41:44Speaker 02

  Like when you're interacting with something that you don't know how it works, and you don't know where the data comes from, that is, you know, you're being told that it's Fed data. You're like, what? You know? What? It's such a different experience when you can interact with something you trust.

• 42:00Daniel Whitenack?

  Yeah. It does change your perception coming into an interface, knowing like, oh, this is, I'm searching against my company's knowledge base, or I'm searching against like a PDF of this book, or you know, whatever it is. As we kind of draw to a close here, I'd love to hear maybe like, you've talked a lot about things that you're actively working on, and that's all really exciting. As you look to the future, or what are you most excited about in terms of maybe what's going to be possible in the coming year that we aren't quite there yet, but that is really on your mind? It could be something that, you know, you're actively working on, or just something in the industry as a whole.

• 42:42Daniel Whitenack?

  What, as a positive kind of closeout, what are you most excited about in terms of like, trends that are happening, or things that you're working on or thinking about?

• 42:52Speaker 02

  I have a couple. I do think that with the technology that I have and with an ontology, I can take like a paragraph of your text and understand something about your existing mental model. So if I can relate new information to your existing mental model, like I found out you live in Indiana. If I say something about Indiana that makes you not only trust me, it makes me give you new information in a way that reduces your period of disequilibrium. We can make people learn faster and that's truly exciting to me. And I think that human beings, given trusted, evidence-based, tested, artificial intelligence,

• 43:34Speaker 02

  if they had that, you know, I think that it opens up this entire world of visual thinkers. And shout out to Temple Granon and her new book, Visual Thinking, all about how we've been living in a verbally dominant society. Well, guess what? Words just became very cheap, very much of a commodity. So the engineers, the tradesmen, the plumbers, you know, the, the people who are doing the things with their hands, the artists, the fuzzies, you know, this is our time. And I think that you're opening up an entirely new market for everyone to be able to create. And that, to me, is super exciting.

• 44:18Daniel Whitenack?

  That's awesome. Yeah, I think that's a great way to close out on a great thought. And I now know my next book after I finish the cuckoo's egg. So, um, thank you so much for joining us, Beth. It's been a real pleasure to talk through things. Recommend people check the show notes for links that we'll include there and hope to chat with you again soon, Beth. Thanks so much.

• 44:39Speaker 02

  You're welcome. Thank you.

• 44:49Speaker 00

  Thank you for listening to Practical AI. Your next step is to subscribe now if you haven't already. And if you're a long time listener of the show, help us reach more people by sharing Practical AI with your friends and colleagues. Thanks once again to Fastly and Fly for partnering with us to bring you all Chang Talk podcasts. Check out what they're up to at fastly.com and fly.io. And to our beat freak in residence, Breakmaster Cylinder, for continuously cranking out the best beats in the biz. That's all for now. We'll talk to you again next time.