Pod Insights MarTech Podcast ™ // Marketing + Technology = Business Growth Navigating GenAI's Hallucinations About this episode Senior Product Marketing Manager for AI at Progress, Philip Miller, delves into the world of GenAI's capabilities and how marketing executives can effectively navigate through the hype to understand its true potential. Discover the ethical considerations that come into play when integrating GenAI solutions into marketing strategies and how this technology can revolutionize consumer behavior prediction. Gain insights into successful GenAI integration in marketing campaigns and learn about common misconceptions that marketing executives should be aware of before adopting this cutting-edge technology. Show NotesConnect With: Philip Miller: Website // LinkedInThe MarTech Podcast: Email // LinkedIn // TwitterBenjamin Shapiro: Website // LinkedIn // TwitterSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
Transcript 0:00 Speaker 00 The Martech Podcast is a proud member of the I Hear Everything Podcast Network. Looking to launch or scale your podcast? I Hear Everything delivers podcast production, growth, and monetization solutions that transform your words into profit. Ready to give your brand a voice? Then visit iHereEverything.com.
0:25 Speaker 01 From advertising to software as a service to data, across all of our programs and clients, we've seen a 55% to 65% open rate.
0:36 Speaker 02 Getting brands authentically integrated into content performs better than TV advertising.
0:42 Speaker 03 Typical lifespan of an article is about 24 to 36 hours.
0:46 Speaker 01 If we're reaching out to the right person with the right message and a clear call to action, then it's just a matter of timing.
0:53 Speaker 00 Welcome to the Martech Podcast, a member of the I Hear Everything Podcast Network. In this podcast, you'll hear the stories of world-class marketers that use technology to drive business results and achieve career success. Here's the host of the Martech Podcast, Benjamin Shapiro.
1:15 Speaker 04 Welcome to the Martech Podcast. I'm Benjamin Shapiro, the executive producer of the Martech Podcast, and today we've got a special episode for you, which is going to be guest hosted by Juan Mendoza, the author of the Martech Weekly Newsletter. Juan is a recovering martech consultant turned creator who writes an amazing weekly newsletter about the martech industry, and I'm thrilled to invite him and some of his friends to take the mic and share their knowledge with you, our loyal Martech Podcast listeners. All right, here's a special episode of the Martech Podcast, guest hosted by Juan Mendoza, the author of the Martech Weekly Newsletter.
1:51 Speaker 05 Welcome back, MarTechers. My name is Juan Mendoza, your guest host on the MarTech Podcast and the CEO of the MarTech Weekly, a MarTech nerd who is excited to talk about AI and how it's hallucinating in the marketplace right now. Joining me today is Philip Miller. He's the Senior Product Marketing Manager for AI at Progress. Progress is innovating at the edge of disruptive change in generative AI, large language models, and the real sort of bleeding edge of technology in using machine learning and other AI models. Relied upon by over 4 million developers and technologies worldwide, Progress empowers organizations
2:32 Speaker 05 to develop, deploy, and manage AI-powered applications and experiences with agility and ease. So Philip Miller and I are going to discuss navigating generative AI's hallucinations. I'm sure you've seen the memes. You've seen Google recommending that the best way to stick cheese on a pizza is to glue it down, or random stuff that comes out of a chat GPT session. Today, this episode is kind of to think about what do we do with that from a marketing standpoint? How much can you actually trust these tools when they don't get it right? How should we treat them? Should we treat them like a coworker, a boss, or should we treat them like an intern that thinks they know it all, but they really don't? So Philip and I are going to dive into this. He's the Senior Product Marketing Manager for AI at
3:17 Speaker 05 Progress, and we're excited to have him. Philip, thanks for joining us.
3:20 Speaker 06 Juan, thanks very much for having me. I'm really excited to be here today. I'm really looking forward to discussing, like you say, hallucinations and how to kind of navigate that in this space. It's a new field to a lot of people, but interestingly, as we've been defining AI in our organization, we've been pulling out some sort of foundational subsets of AI, like machine learning, like expert systems, like retrieval augmented generation now that's new in that space that can really help in this arena. So yeah, really interested to have a chat with you today.
3:51 Speaker 05 I am really excited to, I've been dying to do an episode on this topic, on AI hallucinations and this whole problem around AI getting stuff wrong. And I think that if we take a step back, is hallucinations the right word here? Because when you say someone's hallucinating, it means that they're either being somewhat knocked unconscious or they're mad, like they literally have a mental disorder. But we've kind of used this word to explain what happens when these large language models get it wrong. But maybe you can kind of start there with the definition of what is a hallucination? And what does it actually mean in the context of large language models?
4:27 Speaker 06 Well, it's kind of in our nature to humanize the things around us, to recognize patterns and put shapes in faces. So hallucinations fit the bill, but underlying the hallucinations is really maths. So when you prompt a gen AI, it will then go and look at its model, which come from its training, and then go, I need to answer this question. Okay, what's the most likely next token, not necessarily whole word, but next token to come up in my response to this prompt? And then it will fill out that until it has basically recognized the prompt and then answered that question. Now, because it's maths and essentially I like to say it's like a weighted number generator.
5:11 Speaker 06 So it's producing these random sequences, but it's saying, actually, I know that this should be here. So sometimes if it doesn't have the answer, but it has to answer the question, it's been told it's got to answer this question, then it will then hallucinate, it will generate the next word in that thing and can diverge quite quickly from what the answer should be. And that's for businesses, for organizations, for NGOs, whatever it might be. That is a problem because either they're relying on these systems internally for efficiency sake and things like that, or they're pushing this out to customers and that's the way their customers are engaging with them using these systems. So if things go wrong, then we need to look at that and how do we mitigate that?
5:55 Speaker 05 It's interesting because the metaphor I use is your typical kid logic, like your toddler logic. So one story that which I love to retell, which is a father brings a daughter to the airport for the first time. It's the first time she's this little girl has seen an aeroplane, right? Let's say she's five years old, right? So she's sitting there and she's watching a plane take off. Then she turns to her dad and she said, Daddy, why, when people travel or when people get on planes and they go somewhere else, is it that they get really small? Why is that? Why do they get really small? And then he had to explain, no, that's just because they're moving into the distance really fast. But she thought that she made the wrong inference there. Just because the plane looks like it's getting smaller, it means that that's what travel is, is getting really small. And I think that's a good metaphor for AI hallucinations, which is
6:38 Speaker 05 it's pattern matching, but sometimes it's just matching on the wrong things. Like I mentioned before, cheese and glue. Well, they seem to be sticky type substances that join things together. There's this sort of pulse positive on things that match together that really don't, which seems to me more of an experience question than like a learning question as opposed to creating random things out of nowhere. It seems to be pattern matching, but just on the wrong sort of categories or the wrong edges.
7:03 Speaker 06 There's two things really. AI has been trained on publicly available data. This is data that it has, and I'm not going to go into the ethics of this in terms of how they've got this data and things like that. But I will say that it's been trained on this publicly available data. But there's a whole other dataset that is underneath this that it's never seen before. That is either inaccessible to it for whatever reason, but it's that proprietary private data. It's the data in your organisation, about your organisation, about the products, the services, the domain you exist in, how you describe things in your business. The AI has never seen that.
7:44 Speaker 06 So when you start to get specificity coming into place in these gen AI solutions, I it's not asking a generalist question. It's asking specific questions and you're relying on those giving specific answers. You need to provide it with your data. You need to share with it your view of the world. Now, we could and people have. In a very basic implementation of what they call retrieval augmented generation, one solution that can help reduce hallucinations. You could just send all of your data to the Gen AI and say, have a look at this, learn about my business, learn about my world, everything's there, just have a look at it, right? And the AI that can then make inferences, but again, it's based on the Gen AI's model
8:27 Speaker 06 that has been trained on that other generous set. What we've done and what our customers are now deploying is we use a multi-model semantic RAG approach. And these are all fancy words to basically say, we take your data and we run it through our progress data platform, which includes two kind of fundamental technologies. One is the Mark Logic server, which is a multi-model server that will ingest the data as is. It will curate and harmonize that data, put it into like a single model of your data for like downstream applications. You can then search that with our multi-model query engine to return results and that's great. We also have another feature that works in conjunction with our sister technology
9:12 Speaker 06 in the platform, which is Semaphore, which is this knowledge management tool. It's the knowledge modeling tool. It identifies data about the data and then it puts it into a model of your world, of your business and can take in external models like industry standard taxonomies or ontologies, again, big words, but basically these are models of the world that they exist in. And then what you can do is you're not just providing raw data to the Gen AI. What you're providing is connected contextualized data that essentially you are telling the AI, you know, you exist, great, but you exist in this world and this world is really important
9:56 Speaker 06 that we make sure that we know that this is part of this, this is part of this. Is that maybe look at it as like the tree of life or other taxonomy models or ontological models of the world, but it's basically putting a lens on on your data. And what that allows people to do is the AI then Has more context, has more information, but it has correct knowledge of your business. And it's informed by, and this is what we try and do with our customers and how we implement these things. It's informed by having that human in the loop. It's informed by the dad in that conversation. Yeah, you're kind of right, but it's going further away, but that's perspective and not what actually happens.
10:37 Speaker 06 So it's taking those subject matter experts within your business, business experts, whether they be technical or non-technical. It's taking the industry knowledge of that world with those external ontologies or taxonomies. It might be standard, say, for instance, or legislation, something like that. And it's applying that to that data so that you really have that data in context and that the AI can understand that data. And that's where we're seeing, when we get more accurate answers, we get specificity to the answers. Because it's not just saying like standard Google searches, you Google something and then you go, it gets to the top page and you have a look and then you have a look down. It doesn't give you necessarily the answer, it just gives you where it found that sequence of words.
11:21 Speaker 06 And what we're saying is, I know, in that document or in that webpage or in that, whatever it might be, in paragraph seven, there is the answer to your question. And so not only are we getting specificity, but we're getting trust because it's trust but verify, that kind of intelligence world approach to everything. So you are helping the end user quickly gain insight and trust into the answers that the G A I is delivering.
11:49 Speaker 05 I think that there's some interesting angles on this, which is there's this like this semantic approach where we're starting to see this. And this will be out in tomorrow's episode, when hopefully it'll be back with us, Phil. But we're talking a lot more about integrating generative AI into the enterprise. But I think a big part you called out is that semantic layer, right? Which is a taxonomy, which is basically fancy words for a bunch of data and a bunch of information about your specific context in which you want the generative AI tool to operate in, whether that be on a specific data set within your marketing or a specific bunch of email campaigns or customer feedback. It's not general intelligence, it's. Like very good specialized intelligence that you can kind of build now with these toolsets.
12:30 Speaker 05 So I appreciate you calling that out because that's one way to kind of put the guardrails around hallucinations. The smaller the world, in theory, the less rate of error when working with clean data and clean information for that model. You can kind of see as companies start to look at this, our marketers start to pick it up. There's this opportunity to go, well, how do you minimize that error rate, that hallucination? It's probably making a smaller world for the AI to play in because a lot of what obviously has made so much noise in the industry is ChatGPT. When it came out back in November in 2022, which feels like forever now. But when it came out, it was trained on just the slop of the internet, just everything. And yes, a lot of the information is actually pretty good. It's pretty high quality.
13:14 Speaker 05 But on the other side, there is a lot of stuff that's not great in it as well. And so you can kind of think through this and go, okay, well, what model application do I actually need? And then maybe that's one way to tackle hallucinations. Another way I've been thinking about recently, and I would love your thoughts on this, is how much responsibility should you give an AI model? If it's getting some things wrong and you can't trust it to be absolutely correct, well, you can't really trust an intern to be absolutely correct on everything either. But the difference, and maybe I'm wrong here, but the difference is that an intern can at least, well, sheepishly, explain how they got wrong. Or explain what they did to get to these outcome that was incorrect. How do you approach this? Is this a co-pilot? Is this an intern that is kind of sometimes wrong and you laugh it off?
13:58 Speaker 05 How do you see the way in which a marketer should think about the error rate with these platforms in their own business and the risks they should take with it?
14:06 Speaker 06 Content creation has always been a collaborative effort in any organization. You have editors, you have final checks, you have people looking at the content. I have people, it makes me a better writer, it makes my message be more on point, or I can use different words and things like that. And we accept that as human beings. So why is that not the same for an AI agent? Where you're asking it, can you create me a piece of content? Now, you can give it better prompts, you can then. Include data into those prompts to say, oh, and take this from here. So you're taking that rag approach, yeah, that retrieval or generation. You're searching your data, you're providing it context, and then you're saying, generate that response.
14:51 Speaker 06 But it's the back end as well. When it comes back and that response, you should be looking at that. Now, I can tell you, because of the platform that we use, what happens to your data for ninety percent of its journey? I can tell you when that data was created, if that data is talking about other timelines within the document itself. So it might be talking about stuff that happened two weeks ago, and I can create timelines from that. I can tell you whether it's compliant with your internal controls and governance. I can tell you whether it's compliant with legislation. I can classify and I can tell you why I classified that data, why those facts were important, how they're connected, why we connected them, right? I can tell you right up until the A I. So I could do ninety percent of the journey. That ten percent is in that A I. It's that black box.
15:36 Speaker 06 And explainable A I. Is a field of study, but it's not something that is going to be solved quickly, if ever. Because the mentions that A I. Are dealing with are much more complicated than we view the world. I like to say, like, we have a model of the world in our heads. And we only use our perception, the five senses more than that. But the five senses that we use, only two percent of that data actually informs how we view the world, roughly about that percent. The rest of it, the model. So I do it like this, walk into the living room, look around, and then you go, this is fine. You don't actually use the eyes and sounds and smells and things like that. Most of that information is from the model because you expect the living room to be the same.
16:19 Speaker 06 If you open the door and there's a dinosaur there, it goes to the model and goes, should this dinosaur there? No, quick, fly, fly, right, get out of the door. So you won't necessarily be able to explain why the A I. But what you can then do is with the response, take that and then look at your knowledge base. Look at that contextualized data that you have of your thing. Does it exist? Yes, it does. Okay, here's the right answer. And that comes down to that human in the loop. Human AI collaboration that we see not only for successful classification and connected and contextualized data, but we also see it in the output as well. So my ideas are one thing we're really interested in. We have a lot of publishing customers. We have a lot of standards organizations that publishing adjacent.
17:04 Speaker 06 We have a lot of content use cases in our business right throughout the content value chain. We have a smart content series coming out, but essentially the first two are out and another one is next week, dealing with this topic, this next gen application, including the application of a G A I. But it starts with the data, like any AI solution starts with your data. Bad data in, bad data out. Good, clean, contextualized, connected data, you've got to get a better response. Will we make completely hallucinations? No, because every time it does it, there are a lot of AI people will be very cross with the standards, but it's rolling the dice, seeing what the response should be, and then putting that response in there. That's why that human in the loop on both ends of the spectrum, the prompt and the
17:49 Speaker 06 response, that's why that's so important for organizations.
17:52 Speaker 05 There's this thing that you're touching on, which is a very interesting philosophical concept, which is this mental model of the world. And every person's mind has, you walk into your living room, on the way there, you're predicting what it's going to look like. And your attention on certain things fades into the, into the distance as well. You might be just focused on one thing, like your car keys in your living room, not the table, not the lounge, not the light switches, not all the other things that are going on. Your mind focus is actually very selective on ignoring most of the information that's in your domain and selecting the things that are the most important to what you want to achieve or accomplish. And I think that's kind of an interesting way of thinking about generative AI, because in a way, it's got billions and billions of data sets that it's at its fingers, right?
18:35 Speaker 05 And then it has to sift through all of that in order to find the one focal point, the one needle in the haystack that is actually the correct answer, that is formulated the right way in that it makes sense for a human. Now, that is a technological feat itself, and that is something worth celebrating. And I think that's kind of where we're heading here. Yes, there's problems. Yes, there's hallucinations. But there are ways, explainable AI, as you mentioned, there are ways to actually overcome this, and there are ways to sort of minimize that error away by giving a better focal point for what you want to achieve with AI. Instead of taking ChatGPT off the shelf, looking to apply to specific domains of business and knowledge would just be a far more sound way of thinking about integrating this technology. So Phil, this has been a
19:17 Speaker 05 fabulous conversation, very fascinating, very interesting, a lot of new things happening in this space, but I do appreciate your expertise in navigating generative AI hallucinations. We could talk about this all day long, but we can't because you have to head off and we'll be catching you again tomorrow. So thank you to Philip Miller from Progress for joining us. In our second interview tomorrow, Philip and I are going to discuss developing and deploying AI-powered apps successfully in the enterprise. If you can't wait until our next episode, you'd like to learn more about Philip, you can find a link to his LinkedIn profile in our show notes, or you can visit his company website at progress.com. Phil, thank you so much for joining us.
19:57 Speaker 06 Thanks very much for having me. Cheers.
19:59 Speaker 04 Okay, that wraps up this episode of the Martech Podcast. Thanks to our guest host, Juan Mendoza, the author of the Martech Weekly Newsletter. If you'd like to get in touch with Juan, you can find a link to his LinkedIn profile in our show notes, or you can contact him on Twitter. His handle is JuanMendoza, but it's spelled crazy pants. It's J U four N M E N D zero Z four, or it's a little easier to just visit his company's website, which is themartechweekly.com. Just one more link in our show notes I'd like to tell you about. If you didn't have a chance to take notes while you were listening to this podcast, head over to martechpod.com where we have summaries of all of our episodes and contact information for our guests. You can also subscribe to our weekly newsletters and you can even send us your topic suggestions or your marketing questions, which we'll answer
20:44 Speaker 04 live on our show. Of course, you can always reach out on social media, our handle is martechpod, M A R T E C H P O D, on LinkedIn, Twitter, Instagram and Facebook, or you can contact me directly. My handle is Ben J. Shap, B E N J S H A P. And if you haven't subscribed yet and you want a daily stream of marketing and technology knowledge in your podcast feed, we're going to publish an episode. Every day this year, so hit the subscribe button in your podcast app and we'll be back in your feed tomorrow morning. All right, that's it for today, but until next time, my advice is to just focus on keeping your customers happy.
21:43 Speaker 00 Thanks for listening to the MarTech Podcast and I Hear Everything production. Looking to launch or scale a podcast like this one for your brand, then visit iHearEverything.com.
Previous Successfully Tracking Gen Z Online Habits Next Developing & Deploying AI-Powered Apps Successfully Page updated November 7, 2026 · transcript and summary are machine-generated
