Cybersecurity in the GenAI age

Dinis Cruz drops by to chat about cybersecurity for generative AI and large language models. In addition to discussing The Cyber Boardroom, Dinis also delves into cybersecurity efforts at OWASP and that organization’s Top 10 for LLMs and Generative AI Apps. Sponsors: Speakeasy https://speakeasy.com/ – Production-ready, enterprise-resilient, best-in-class SDKs crafted in minutes. Speakeasy takes care of the entire SDK workflow to save you significant time, delivering SDKs to your customers in minutes with just a few clicks! Create your first SDK for free! https://speakeasy.com/Fly.io https://fly.io/ – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun https://fly.io/speedrun/ to get started in minutes. Shopify https://www.shopify.com/practicalai – Sign up for a $1/month trial period at shopify.com/practicalai https://www.shopify.com/practicalaiFeaturing: Dinis Cruz – Website https://diniscruz.blogspot.com, GitHub https://github.com/DinisCruz, LinkedIn https://www.linkedin.com/in/diniscruz, X https://x.com/DinisCruzChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: The Cyber Boardroom https://www.thecyberboardroom.comOWASP - Top 10 for LLMs and Generative AI Apps https://genai.owasp.org/llm-top-10OWASP - Generative AI https://genai.owasp.orgUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Speaker 00

  Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related tech is changing the world, this is the show for you. Thank you to our partners at fly.io. Fly transforms containers into micro VMs that run on their hardware in 30-plus regions on six continents, so you can launch your app near your users. Learn more at fly.io.

• 0:36Speaker 01

  Well, our friends over at Speakeasy have the complete platform for API developer experience. They can generate SDKs, Terraform providers, API testing docs, and more. And they just released a new version of their Python SDK generation. It's optimized for anyone building an AI API. Every Python SDK comes with PyDantic models for requests and response objects, and HTTPx client for async and synchronous method calls, and support for server sent events as well. Speakeasy is everything you need to give your Python users an amazing experience integrating with your API.

• 1:16Speaker 01

  Learn more at speakeasy.com/python. Again, speakeasy.com/python.

• 1:36Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I am the founder and CEO at Prediction Guard. And I am joined, as always, by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. It's good to record one together, Chris. There've been a couple where we've been apart.

• 1:56Chris Benson?

  That's right.

• 1:58Daniel Whitenack?

  Yeah, it's good to have the band back together, I guess.

• 2:02Chris Benson?

  Absolutely. I took a brief respite. I had some lower back surgery. And since I'm always thinking about technology and AI, I was trying to imagine... What what the surgery might be like today versus if I had done this, if we were in a slight time warp and gone to the future.

• 2:19Daniel Whitenack?

  Did they have a robot arm?

• 2:21Chris Benson?

  Exactly. If they did, you know, reinforcement learning combined with, you know, kind of a chat interface and the whole thing. So it was kind of a novel, a novel mind experiment there that I had about what would it be if I was, you know, this was a little bit different time. But yep, all good for here.

• 2:39Daniel Whitenack?

  Yeah, well, I would hope that in our, in all of our futures, as we have medical procedures done increasingly in, by AI or AI assisted and other things like that, that those things are very secure as they happen, that that would be very important, I think.

• 2:58Chris Benson?

  Yes, absolutely.

• 2:59Daniel Whitenack?

  And on that front, I'm really excited because Chris, on this show, actually a couple of times, I referenced the OWASP top 10, Gen AI top 10 sort of risk, white paper breaks down security and privacy related risks into sort of different categories and helps people think about it. This is a collaborative thing with with multiple organizations involved. But today we've got with us Dennis Cruz, who is a founder at the Cyber Boardroom, but also has been involved in OWASP in various capacities over the years and is aware of all of this that's going on and contributing to it. So we're just super excited to have you with us,

• 3:42Daniel Whitenack?

  Dennis. This is one I've been been really looking forward to.

• 3:45Speaker 04

  Thanks for inviting me. And this is a topic that I'm very interested in, but I because I think there's a lot of potential. There's there's a lot of dangers too, but I think it's very exciting times for us as an industry in all sorts of levels.

• 4:00Daniel Whitenack?

  Yeah, well, like I mentioned, you've been involved in OWASP in different capacities. How did the kind of, I guess, give us a little bit of background and how the, first off, what what OWASP is and what that means for those that aren't familiar, but also as you kind of started seeing this Gen AI stuff come about, how did it strike you from that standpoint of being involved in OWASP for over the years?

• 4:25Speaker 04

  Okay, so OASP is the Open Web Application Security Project. And you can say that OASP is the people around the world that kind of cared about application security, that found that sort of intersection of the world moving to apps versus networks and then the security elements of it, right? And I think it's definitely one of those organizations has grown from nobody really cared about it to, you know, some more people caring it. So now it's been referenced left, right and center, right? And it has really, you know, changed the world in very, very nice ways. And it's always attracted, I think, the people that want to do better. I think it's a lovely community. It's very open, you know, by its nature, but it's also, I think it's been a great hubbed of innovation, right?

• 5:09Speaker 04

  In terms of the first OASP top 10, the testing guide, a lot of amazing tools come out of it, OASP Jab, you know, dependency checker, there's billions of tools. I developed a couple too, and it's a great, great community, right? And I think GenAI, it's an interesting evolution on our technology, even from a security point of view, because I think for the first time, we have a technology that kind of understands intent, right? And I think that's quite different. Now, I want to say ah, the bad, but I think what GenAI is doing, ChatGPT and everything, is actually making most of what we always talked about in OASP even more important, right? Because if you think about it, you know, GenAI is an API, right?

• 5:53Speaker 04

  It's fundamentally an API that you send some data in, you get some outputs out, and it is there to even connect to internal systems. And in a way, it's your most vulnerable and your most dangerous API at the same time. And I would say that the most thing that is quite unique with GenAI is the fact that you're now sending English or Portuguese, right? Or French or whatever language or Klingon, right? Whatever language you want to talk to, that is now code, right? And that's quite different, you know? For the ones who've been doing security for a while, we always talked about securing code and data, right? Like if you can separate code and data, we know what code is, we know what data is, you actually can secure things quite well. Now you're in a world where data is code.

• 6:37Speaker 04

  So that introduces a huge amount of interesting challenges. And I think it's one of those technologies that at one end, he's going to create a lot of problems. But on the other end, I think he has the potential to solve some of the biggest challenges of scale that we always had in security and in engineering, right? And in development with that technology too. So I think it's going to add up to both sides of the universe there.

• 7:01Chris Benson?

  It's interesting. I actually want to go back to something you said right as you were starting off a second ago and build on it a little bit that, like you kind of alluded to, for a long time, you know, over the years, security was always kind of a, you know, the redheaded stepchild of the, of application developer, you know, there was a lot of folks that just didn't care too much. And obviously, we've progressed along and we're talking about Gen AI and things like that, what we're doing now. But going back, I'm just curious, as we're talking about security in general in app dev, even before we get to AI being introduced into it, what has motivated people to get into the security side in your view as you've watched this? Because I too have seen that the security concern go from being kind of a backseat thing

• 7:46Chris Benson?

  to being very front and center and very important in a lot of organizations. And that's been an evolution. And as you've been in the middle of this ecosystem, how have you seen that happen? And why? And where's that progression come to?

• 7:58Speaker 04

  Well, so I would say from personal experience, and I think from most of my peers, there's, you know, there's a thing of it's nice to make the world safer, right? There's definitely a shank of like, it's nice to do something that the outcome is to make people safe. And I've always liked to stay on the sort of the fence, you know, yes, a bit offensive, but I'm making things systems secure, right? So I think AI intellectual is really cool. I think from a technology point is really cool. There is a thing of hacking and breaking into stuff, which is also quite, you know, quite interesting, right? There's a little bit of a James Bond, you know, hacking into the big system and getting there and all that jazz, which I think is part of the evolution, right? So there's, I think, is a really cool industry. It's very open, it's very welcoming, right? It's also very pragmatic, right?

• 8:42Speaker 04

  It's in the beginning is like, can you exploit or not, right? Can you do this? Can you do that, right? And I have to say, we need to also be pragmatic to say, for a long time, it didn't really matter. Right? Like, in a way, there was a period where you knew who's being attacked because they were the companies who had the best security teams. Literally, there was a one-to-one correlation. That team does security very well, cool, they've been attacked, right? They're on the evolution, right? And I've now been a CISO, I've worked a lot of companies. And there's definitely, I tend to break into three different worlds, right? There is the nation-state, you know, the high-level espionage. There is the ideology attacks, right? And there's the commercial attacks. The reality is that this one is a completely different level. If you're dealing with nation-state, you know about it, right? You have hell of a teams.

• 9:26Speaker 04

  And we can do it, right? But it's a very different game. The ideology, it tends to depend on the industry you're in, right? Some industries are much more toxic, they tend to attract that. Most of it is to do with the business model of the attackers. So what has happened in the last, you know, ten, fifteen years is that the more we move our society into digital, the more the business model of the attackers evolve. So in the past, you can get away with a lot of things. You could get away with crazy secure applications out there, and the probability of you being attacked was quite low, right? Now, you kind of can't, right? So I think there's been that evolution of, in a way, the stakes are much higher. Right now, you know, even the recent CrowdStrike problem, right? You know, which I still think is a security problem, right? We can talk about it if you want.

• 10:09Speaker 04

  But that just shows how one problem in technology can bring down, right? Large chunks of society, right? And in the old days, it was the NIMDA, right? The ISS worms that brought down a chunk of technology. But at the time, the impact was limited. Now, it really makes a difference, right? Like, now a cybersecurity problem in an organization of a medium size can break the company, can really severe disruption, right? Even a ransomware incident in one of our suppliers can actually cause a lot of problem in your world. So I think, you know, the application security and all that and everything, we have, in a way, matured with the kind of... The evolution of the market, right? Although I have to say that I still feel that most companies still it's still a marketing exercise.

• 10:51Speaker 04

  I'm not, it's a controversial view, but I think we still don't have a good way to measure the cybersecurity preparedness of companies. So there's still a lot of companies that kind of roll the dice and go, well, I hope it goes okay. Right? Although much less than before, but I still feel there's a lot of maturity that we need in our own industry. Right? To do that. And this is the final point, the thing I also learned a lot was that cybersecurity is not an isolation. Cybersecurity is a side effect of engineering and business practices. Right? So there's a moment where if you want to fix cybersecurity problems, you want to see, you know, fix application security problems, you have to fix engineering problems. Right? And that leads to that, right?

• 11:36Speaker 04

  If you do really good engineering, if you do really good development, really good practices, half of what we talk about in application security is not needed because you just do it. Right? It's just good practices. The problem is a lot of companies, a lot of teams don't do that. So the side effect is the security problems.

• 11:53Daniel Whitenack?

  I'm wondering, when you're talking about this, this sort of business practices, the engineering, where these, you know, security vulnerabilities pop up and that sort of thing. Part of what comes to my mind on the gen AI side is the fact that it brings the domain experts a lot closer to the actual technology in the sense that like often there are tools or there are interfaces that people are using to create sort of chains of reasoning just with prompts and other things, you know, just with natural language. So they're not, to your point, I think now this sort of natural language is kind of like the code of programming these models. How do you think that that shifts the dynamic between like the engineering side and the business side?

• 12:37Daniel Whitenack?

  Cause if from my perspective, a lot of these domain experts and business people are coming much closer to the actual kind of core functionality of an application.

• 12:46Speaker 04

  And I think that's the insane opportunity, right? So first of all, I don't view that the real power of chat GPT is that it can create things. Right, the real power of expertise is that it can understand context, it can understand data, it can create mappings and relationships, right? And what is in practice, you know, the way I I visualize what you described is that in the past, everything you want to do had to be coded, right? It had to be solidified in bits of code, which is also where vulnerabilities were created, but also where, in a way, we were locking down the business logic into code, microservices, trucks, etc. And then we got bogged down by the complexity of that, right? And then we couldn't understand the side effects and then things ground to a halt.

• 13:29Speaker 04

  And then, again, security vulnerabilities, vulnerabilities appear in those gaps, right? It's those gaps, it's those misunderstanding of how APIs work, or an API that was created over here in a secure state that is now plugged to the internet, you know, maybe not the best move, right? And but I think what where we're now entering a phase is that you can have a layer where the business logic is now described in prompts, right? And what happens when you describe the prompt, you start to describe the intent of what you want to see happening. And instead of that intent being locked in code, which is really hard to change, it has a huge amount of things, you know, side effects, you can have that in prompts. Now, we need to be better understanding the prompts. I'm doing a lot of work on provenance deterministic AI

• 14:14Speaker 04

  to make sure that we actually have AI outputs that are deterministic. But it's very powerful to be able to start to describe that, right? Even I'll give an example, like if you go to any organization, like when we do threat modeling, so one of the practices in security is threat modeling, right? Threat modeling is just social engineer the other side to give us an architecture diagram, right? That is up to date. Because once you have that, you'll find vulnerabilities, right? Because you start asking questions, what about this, what about that, what about that, what about that, right? You know, in a way, the hidden elephant is most organizations have no idea how the app works, right? They have no idea, you know, how actually things connected, the guys who developed it have gone, right? Like, you know, now imagine a world where we start to use G N A I to explain how it works.

• 14:57Speaker 04

  Imagine, you know, a code commit that goes to a G N A I layer that I can ask the question. Did this change the attack surface, right? Did the attack surface of our organization change because of this? And they say, oh, if you did, then we need to do a review, right? If you don't, cool, go all the way to production, right? Now, these questions in the past were impossible to ask, right? Or you had ridiculous static analysis that, you know, just about didn't work. Now we can start to codify a lot of those business logic questions, a lot of those intents of what I want to do, right, in, you know, in, in basically in simple language instead of code. Because imagine like, if you say, I want to do a change that allow anybody from the internet to access

• 15:39Speaker 04

  every record from this company, right? You probably go, oh, hold on, I'm not sure that's, that's a good idea, right? Or, you know, I, I want this to allow, you know, an attacker to run and control JavaScript on my clients, right? Ooh, maybe not, right? I want my secrets to be put on the source code, right? Maybe not. But at the moment, all those things get locked in to code, right? And it's, and it's hard to understand the side effects. Because nobody arrives one day and says, I'm going to create a security vulnerability, right? They, you know, unless they're malicious, right? But apart from those, right? It just, a lot of times it's just genuine mistakes, right? So I think what Gen AI gives us is the ability to start thinking a lot more like three dimensional, right?

• 16:23Speaker 04

  In our questions, but also in our applications where we start to describe the intent of what we want to do. And that can be analyzed, in fact, that can be analyzed by other Gen AIs. So we can start to get a much better sense of actually what is happening.

• 16:49Speaker 01

  Okay, friends, I'm here in the breaks with Annie Sexton over at Fly. Annie, you know, we use Fly here at Change, but we love Fly. It is such an awesome platform and we love building on it. But for those who don't know much about Fly, what's special about building on Fly?

• 17:06Speaker 05

  Fly gives you a lot of flexibility, like a lot of flexibility on multiple fronts. And on top of that, you get, so I've talked a lot about the networking and that's obviously one thing. But there's various data stores that we partner with that are really easy to use. Actually, one of my favorite partners is Tigress. I can't say enough good things about them when it comes to object storage. I've, I never in my life thought I would have so many opinions about object storage, but I do now. Tigress is a partner of Fly and it's S3 compatible object storage. That basically seems like it's a CDN, but is not. It's basically object storage that's globally distributed without needing to actually set up a CDN at all. It's like automatically distributed around the world.

• 17:50Speaker 05

  And it's also incredibly easy to use and set up. Like creating a bucket is literally one command. So it's partners like that, that I think are this sort of extra icing on top of Fly. That really makes it sort of the platform that has everything that you need.

• 18:05Speaker 01

  So we use Tigress here at ChangeLog. Are they built on top of Fly? Is this one of those examples of being able to build on Fly?

• 18:12Speaker 05

  Yeah, so Tigress is built on top of Fly's infrastructure and that's what allows it to be globally distributed. I do have a video on this, but basically the way it works is whenever, like, let's say a user uploads an asset to a particular bucket. Well, that gets uploaded directly to the region closest to the user. Whereas with a CDN, there's sort of like a centralized place where assets need to get copied to. And then eventually they get sort of trickled out to all of the different global locations. Whereas with Tigress, the moment you upload something, it's available in that region instantly. And then it's eventually cached in all the other regions as well as it's requested. In fact, with Tigress, you don't even have to select which regions things are stored in. You just get these regions for free. And then on top of that, it is so much easier to work with.

• 18:57Speaker 05

  I, I feel like the way they manage permissions, the way they handle bucket creation, making things public or private is just so much simpler than other solutions. And the good news is that you don't actually need to change your code if you're already using S3. It's S3 compatible, so like whatever SDK you're using is probably just fine. And all you gotta do is update the credentials, so it's super easy.

• 19:19Speaker 01

  Very cool. Thanks, Annie. So Fly has everything you need. Over 3 million applications, including ours, here at Change, like multiple applications have launched on Fly, boosted by global anti-cast low balancing, zero configuration, private networking, hardware isolation, instant WireGuard VPN connections, push-button deployments that scale to thousands of instances. It's all there for you right now. Deploy your app in five minutes. Go to fly.io. Again, fly.io.

• 20:12Daniel Whitenack?

  So Dennis, you said something that was pretty intriguing to me, which I think is maybe a kind of distinction that is maybe interesting to draw out and get your thoughts on, which is this idea that like when you think of kind of the intersection of cybersecurity and AI, you could come at it from two perspectives. So you could come at it from how can we use AI to help us in our cybersecurity tasks or to create new tools for cybersecurity? And then the other side would be, well, how do we operate AI systems in a secure way? So there's probably some interaction between these two things,

• 20:54Daniel Whitenack?

  but could you give us a sense from your perspective as a, as an expert in this field and also seeing a lot of things so far? How do you see the kind of maturity of these two sides of that coin? Anything you'd want to highlight on either side of that in terms of how both, both things are progressing, at least at the state of where we are now?

• 21:15Speaker 04

  So you're saying that the difference between using AI to sort of build systems and do things and then using one of those outputs is the cybersecurity analysis, right? Of what you have.

• 21:27Daniel Whitenack?

  Yeah, I could imagine there's ways I could use AI to fight cybercrime, for example, or to prevent malware, or like you just said, to help explain applications. So that's using AI to help you create more secure systems. Whereas there also could be just your AI system is insecure in and of itself, right? And how you've deployed it and run it.

• 21:51Speaker 04

  Yeah, and just on the second one, I think if you're not careful, most AI deployments are ridiculously insecure, right? Like, in fact, we have to take into account that we still don't have a good understanding for how the models work. So the reality is there's nobody today that can tell us that these models don't have ridiculous backdoors in there. They don't, even not intentional, right? Even maybe, you know, just the way it works. When we started this, people thought that a string copy was okay, right? People thought that a little catch between a memory copy in the OS was okay. And then we realized that you can drive buffer overflows, ridiculous exploits through it, right? So I think we're in a nation state at the moment, like in the early days of understanding everything you can do with a model.

• 22:36Speaker 04

  So my kind of view on this is that models that you want to use on that, you know, how to use models to cure should be read-only, should not learn. You don't want them to almost bring any content. You want to give you the content. You run them in complete isolation and you assume that whatever you put on it is already exposed and you verify the hell out of what comes out of it, right? And I think there's a lot of companies who are rushing into pushing models. The problem is that they're not taking into account that the models themselves are ridiculously powerful. And this is why we want to imagine that somebody can put a payload that is then executed by a model. And that model sits now in the middle of your organization, in your cloud, in your environment.

• 23:21Speaker 04

  Who probably has access to APIs or other assets, right? That is ridiculously dangerous, right? And that's what we're doing, right? So I think in one hand, I think we need to be very careful in putting models in line in how we actually validate the inputs and the outputs, which is kind of why I view them in multi-tier sort of flows. And then on the other hand, when we use them in a safe way, they're ridiculously powerful because going to your first form, how to use them for cybersecurity. What I really like is that I always felt that the way, the model for cybersecurity is a model based on the attacker making a mistake. It's not about you protecting everything. It's about you almost, you want the attacker to make a mistake.

• 24:04Speaker 04

  I make a call that was not supposed to happen, make a download, make a connect or connection, you know, access the application in ways that no user will access it, call web services that completely out of sequence, right? In the past, again, it was impossible to model this, right? We tried recent technologies, even people that we created ridiculous, you know, installations of seams and technologies, et cetera. They really struggled that. But I think we now have a good chance of doing that. So that means that we can now create much more, I would say, hostile environments for attackers because we force them to follow the paths of the users, which by the way, they don't know, right? What those paths are unless they're already in your system, right? So I think we have a chance of using that.

• 24:48Speaker 04

  But what we need is we need models that are really, really reliable. So OAS has an amazing top 10 for applications, has a really good top 10 for GNI models, right? Et cetera, LLMs. What I think about is most of that is trying to deal with the fact that the models can learn and the models can actually be, you know, don't have deterministic outputs. And I like the idea of actually turning the tables around and say, hey, I don't want my model to learn. I want the data that my model has access to be completely determined by the session and the state that that request comes in, which is normal app sec, right? And ideally, I don't even want the model to have knowledge, right?

• 25:33Speaker 04

  I want to give the model the knowledge that it's going to use so I control hallucinations, right? Like, you know, Chris, you know, you talk about your operation, right? Like, you don't want the GNI doing the operation on your back to slowly go off piece, right?

• 25:47Chris Benson?

  Definitely not.

• 25:48Speaker 04

  An operation on your leg, right? So in this theory would be that you want, for example, the GNI model that is facilitating your operation to only know about back stuff, or maybe to know general things about the body, understand that, but the main knowledge that it has should be laser sharp focus to the situation that you're in, and then that's how you control hallucinations, right? So I think the fundamental problem that we actually went backwards in security is that we now don't have a separation between code and data, right? And that's, I don't think we speak enough about this, because for me, that's a massive problem, right? It's a massive problem because we really need to be able to distinguish what is code, what is data, what's an input, what's a command?

• 26:34Speaker 04

  So I'm doing a lot of stuff where I go from JSON to JSON, and the latest model do this better, where I almost want an API coming in. I give that API, which I can form nicely with data validation and stuff, to the model, and then the model output itself is an API that is completely strongly typed, so I understand the output, if that makes sense.

• 26:55Chris Benson?

  It does, it does. And just on the side, I'll just say I might actually need that operation on my leg too, but I would prefer it was a separate operation that we

• 27:04Speaker 04

  Exactly.

• 27:05Chris Benson?

  planned out, just to note it. It's funny, as you were, as you were taking us through that, I have a whole bunch of different pages up here relating to things we're talking about, including on the Ooff site, that top 10 for LLMs and generative AI apps. And ironically, you were going through that, I was like, wow, you know, they already have this amazing list, which kind of addresses these things you were talking about. And then you referenced it explicitly. I was wondering, could you kind of take us through, like, how was that generated? What's the thinking? Because it looks, based on everything you were saying, it looks like almost a roadmap of the things that one needs to be thinking about when going through the process. Could you take us through that a little bit?

• 27:45Speaker 04

  Well, I think you just nailed it. I feel that what you have there is, you know, the team who did it, and I wasn't very involved in it. I was a little bit on the outskirts of that project, because I thought they were doing amazing work, right? Is that they had a use consultation period, right? They talked to a lot of people. They basically, they listed a lot of the stuff that goes wrong. What I think is interesting about that is I think that that whole list has a bias for the teams are kind of deploying their own solutions, right? And it kind of covers a lot of those things, right? I kind of feel that a lot of that needs to be addressed by the people that provide the models, right? And I think more and more, I almost, you know, it gets to the point where you don't want to build your own cryptography, right? You want to use cryptography models that are very robust. It gets to a point where some of these, I'm like, you shouldn't be building your own model.

• 28:30Speaker 04

  It's really like, look, unless you have a hell of a team, and you really know what you're doing on that area, right? Most organizations, I don't think should be building models, right? Because the big paradigm shift for me was when the prompts is where the action is, right? And even if you look at things like Claude and the recent, now the people are sharing the prompts for those things, you see how much the prompt is actually impacting, right? The stuff. So go back to top 10. I think it's a great roadmap. People who are deploying their models to go, do I have to care about this? This is applied to me, like, how do I answer this in an effective way, right? Because I think that's very important.

• 29:28Speaker 01

  You know, when we started podcasting back in 2009, an online store was just the furthest thing from our minds. Now we have merch changelog.com and you can go there right now and order some T-shirts and that's all powered by Shopify. What did we do before Shopify? I'll tell you, we did nothing. We couldn't sell. There were other ways, of course, but they were very hard, very difficult. Shopify let us build out an entire front end, obviously branded like changelog is. It's amazing. Merch.changelog.com. And our favorite feature is we use their API to generate a new coupon code, a personalized coupon code for every guest that comes on our podcast and they get a free T-shirt from our merch store. And that's so cool.

• 30:11Speaker 01

  They choose the shirt they want, they use the coupon code, it arrives free of charge to them and life is amazing. But also you can go there right now too. Merch.changelog.com and buy some threads yourself. And that's awesome as well. So upgrade your business and get the same checkout we use with Shopify. Sign up for your $1 per month trial period at Shopify.com/PracticalAI. All lowercase. Go to Shopify.com/PracticalAI to upgrade. You're selling today. Again, Shopify.com/PracticalAI.

• 31:08Daniel Whitenack?

  Well, Dennis, I'm really fascinated by this concept that you brought up about separating the model and the data. You phrase that in various ways. It sounds like you've been thinking about this concept, you know, a lot. I'm wondering if you could bring that to a practical level, maybe for those out there that are kind of wondering like maybe in both cases. So I'm using a closed model provider like, you know, OpenAI or Anthropic or something like that. There's that scenario. There's also people that are hosting their own model or even running it locally on their laptop with a local model server. From your perspective, what are the interactions between kind of model and data?

• 31:49Daniel Whitenack?

  Or like, as you put it, knowledge and model that are relevant in those scenarios to create either goodness or badness in each of those scenarios?

• 31:59Speaker 04

  So I think the first very important thing that is very relevant today, that wasn't, I would say, six months ago, is that we need to move from this idea that you have one model, right? What you have now is you have an ecosystem where you have multiple models, right? And they will go from probably some of the most, you know, commercial, if that fits your model that you want to use, to the open source one, but also from the most powerful to the least powerful, right? Because what you want is this mode where you start with, I want to do X, right? And with X, you want to start figuring out what is the best model? And sometimes what is the best combination of models that will give me that output, right? Because in a weird way, the best deterministic way to do something is code

• 32:44Speaker 04

  or to have the least amount of moving parts in there. So, and because also remember, there's a cost issue here, right? So the more you use the models, you want a situation where you're firing these model analysis all the time. Now, if every one of those is hitting an OpenAI endpoints, that will get very expensive very fast. But it's not just that, right? Sometimes you don't want that whole package. You don't need all of that. If you just want a summary or you want a validation, you want this. There's now a lot more models and there's models who are specializing in specific things, who have certain biases that you want to have those biases, right? In terms of that. And so I think it's important to start thinking not just of one model, but as a sequence of models, but also what is the best model that you have?

• 33:27Speaker 04

  And the open source models, the reason why they are getting changed, right? Is because suddenly you can now run models, let's say with Olama. On a desktop, CPU with decent speed, right? And this is speed might not be like, you know, the real time now we now get to chat GPT and Claude three, etc. But maybe even how chat GPT was a year ago or two years ago. But what it means, it means that if that's on your pipeline, you now have a pipeline that is run off CPU, right? You don't even need GPUs now. You can, if you of course, if you have them and you can afford them, it fits a model. But it's CPU level that can run a model that is completely isolated from the internet. And I think it's very important.

• 34:08Speaker 04

  I think it's very important you have a design that has those workflows because you start to introduce them as part of your workflows. In a way, the key answer to your question is, people need to pick up a use case, right? It doesn't need to be ridiculous complex, but pick a use case and then try to do that with a Gen AI workflow, right? And that workflow where in the past you had to code now is a workflow that has multiple L L Ms, it has multiple sequence. I have things where sometimes you create same question to three models, then you use a fourth one to analyze it and on that pipeline. And what I'm trying to do with the cyber boardroom is fundamentally try to address in cybersecurity, how to communicate, how to translate cybersecurity knowledge to board members or executives,

• 34:53Speaker 04

  but also how to get those executives to ask good questions and to translate what they care about. So a really cool use of technology and that is to think about translation, right? So for example, if in the past, as a CISO, I produced reports and briefings for a lot of people, but I didn't customize them because it didn't scale, right? Now I have the ability to create a customized version for Daniel, right? And take into account your culture, your language, your your context, your level of interest. Do you care? Don't you care? What's your focus? You know, and I have another version for Chris, right? So maybe Daniel, you're a lot more focused on the financial element. Chris might be more focused on the strategic element of it.

• 35:36Speaker 04

  So I now have the ability to translate a bit of knowledge into very specific domains of one, right? Because I can feed the knowledge, I can feed what the background information, I can feed the audience, and then I can say chat GPT or Claude or Lama or, you know, Gemini, translate this, right? And they are really good at that and they don't tend to hallucinate. At that level, because you you create the parameters. So that's a good example of a use case, right? Which is very laser sharp, but adds a lot of value. Because, you know, imagine, these are not just execs. Imagine you have the project manager and you have a program manager and you have the lead developer and you have the QA and you have the marketing person and you have the executive. You can now create briefs for every single one of them.

• 36:20Speaker 04

  That puts into context why they care. Why is this important? Why this, you know, cybersecurity stuff means that when the marketing team does a campaign, the website doesn't block your users because you just have 50% more traffic. This is a real story, by the way. It's like, you know, we we were attacked by our marketing department around the prime time TV ad. Great, but if you knew about it, right? We could have planned, right? So there's all this lack of communication, and I think is really interesting to do in organizations. And in the past, this didn't scale, right? So if you now take into account that you now have multiple models with different level of capabilities, you almost want to think, why is the most cost effective? What's the most deterministic way for me to chain this?

• 37:05Speaker 04

  Where, you know, you maybe use some cheaper models to do some stuff and then maybe use the last more expensive model to actually do some kind of Uber analysis and make sure that you still, it all makes sense, right? And of course, this should all be calibrated by the humans who start to calibrate the inputs and the outputs that go into the system. But that's why I feel that it's very, very exciting now that we're having this super competitive race between the different models because we now have a huge amount of models to choose from and you can now start to pick which is better for each capability, right? And that's why I I I want to see models that have no content, right? I want to see models that just have understanding and logic, right?

• 37:46Speaker 04

  It's almost like we need to find a way to strip away, you know, some of the content so that I can say, this is my policy, right? This is what I care about. This is my world. Because remember now, we now have big context windows, right? So you can now feed quite a lot of data in a prompt that goes into the model.

• 38:06Chris Benson?

  It's really fascinating, you know, with what you're doing at the Cyberboard Room in terms of kind of optimizing, using the right models in the right way. And then I actually want to reach back a little bit to something you said a few minutes ago, where you were saying, you know, from a security concern, you wouldn't want most organizations to, for instance, build their own models from the ground up, you know, use these foundational models for which you have strong security basis in that you can trust, and then optimize in the way that you were just discussing. I think you've really hit on something because I think a lot of organizations are really struggling with how to approach the different workflows to maximize their productivity and that output while staying secure and not exposing themselves.

• 38:51Chris Benson?

  So you seem to have a really good grasp of this workflow that maximizes the productivity and the efficiency for the different audiences while not getting them into trouble by doing something they're probably not well-suited to do. Is that fair? Would you say that that's a fair way of?

• 39:08Speaker 04

  Yeah, it's nice compliment, thank you. But that's what I'm trying to do, right? I kind of call it deterministic, gen AI. And people go, well, but you know, it's not supposed to be deterministic. I'm going, yeah, but that's a problem, right? Like, for sure, there's a cool side of it for creativity. Great, we already have that, right? What I think is interesting is to leverage that ability to understand context and to write it in English or write in Portuguese or write whatever language you want and to do that translation layer, but to be very deterministic, which also means that we need to be much better at provenance, which is basically that path of this, that, this, this. But also it's a way to scale, right? Because if you start to have provenance on good sources of information,

• 39:51Speaker 04

  then you don't have to do this all the time, right? You kind of, you can build your knowledge base, right? You can build your workflow base and you can build your confidence. And then it's about creating these microservices that do one thing really well. And we think about like, you don't want a microservice that changed behavior next week when a new model comes along, right? Like dude, like it's like, we want deterministic stuff, right? Because we built things on top of that, right? So we need to start getting to these building blocks again, components, right? That they do one thing, they do it really well, they are super reliable. Yes, there might be a model in the middle, but that means that the thing has this size versus that size, right? In terms of the capabilities.

• 40:31Speaker 04

  And more importantly, I think that you mentioned this before, that this allows us to go to the business owners and let them be in a driving seat. Because think about in the past, we had gurking, if you guys know what that is, when you write stuff in quasi-language. If, when I do this, then I do that, or given this, given that. But that was always a hack because it was like fucking hardcore to the back end. Now we can actually have the business describe the intent. They can describe the workflows. They can describe what the experience they want for the user, for the data, even data transformations. We can start to describe what I want to get from here to there, right? And then it's about how can you lock it down in the most reliable piece of code and system, right?

• 41:16Speaker 04

  That can do that, which is why having models that run offline are very important. Because that allows you to lock in version control that thing. And then know that it will still go in two months, two years, five years, it will still do the same thing. And I think that's very important. So from a security point of view, what I've learned was every time you have a system that behaves like a black box, you have vulnerabilities. Right? Like it's literally, you know, so we creating the Uber black box, right? For this stuff. So what can go wrong, right? So in the, in the, in the past, I knew that the less the team tested, the less the team had the architecture diagrams, the least they understood how a part of the application work, the more vulnerabilities I was going to discover. Because they couldn't test it. It's almost like, how can the developer

• 42:00Speaker 04

  understand the side effects of what they're doing if they don't see it, right? So I think we have to be very careful by creating these black boxes, right? That we don't understand the behavior and how it works.

• 42:12Daniel Whitenack?

  Something that has maybe brought into a little bit more clarity for me as you were just describing what you just described with the the black boxes. We had a few episodes ago, Chris, you remember we had the episode. I chatted with the NATO when I was in London at with secure. And one of the things that he brought up, I'm curious of your perspective on Dennis is like these very, very large models, especially the closed ones, have a huge attack surface. Like the vulnerabilities, like they're so general, there's so much knowledge kind of embedded that it would be sort of impossible to think that you could kind of fully explore the space of behavior and prevent things like jailbreaks

• 42:55Daniel Whitenack?

  or or things like prompt injections, that sort of thing. Whereas sort of the smaller model you bring the data to is either going to perform really good if you bring the right data to the table because it's not embedded in the model, or it's going to be complete trash output, which maybe is better because you're operating in a regime where the data distribution isn't what you expect. I'm curious if that tracks.

• 43:20Speaker 04

  And you can ask, you can verify. Yeah, I think that's spot on, right? Look, it tells you something that we still, it's almost like somebody had a cool analogy, like it feels like we're back in the navigation. Like Portuguese has a great history of discovering the world, right? So we, at school, we learned how the Portuguese, right? And the Spanish and everybody else were like, what's out there? Let's send a boat, right? And then, hey, look, we discovered a country or we discovered, you know, something, right? It feels a lot of these models are like that, which in a weird way is ridiculously scary. Like, imagine like.

• 43:50Daniel Whitenack?

  There'd be dragons.

• 43:51Speaker 04

  You know, yeah, imagine you're running up, you do a web service, you're shipping, go, hey, do you know that thing knows chemistry? It's like, well, well, you know, that thing speaks 20 languages? Well, so I think we, it's the fact that we still have, we're talking about emergent properties. And the fact that. People still talk about what the models do by probing it, right? Like from the outside as a ridiculous black box. That shows you how immature we still are in that level, right? Like, yes, software is complex, but we can actually understand kind of what it does, right? Like, we can actually, you know, okay, given the time and money, we could actually reverse engineer even the most complex piece of software and go, yes, this thing is not going to know chemistry, right? This thing is not going to do beyond this.

• 44:36Speaker 04

  It might do some bugs, but it has a limited operation space, right? I think the models, in a way, it's a good thing in some aspects, but they have all these properties because they build these really big three-dimensional or multiple-dimensional views of it. But that's not what you want for mission-critical systems. And that's what I was talking about exploits. My prediction is there's going to be a number of exploits, backdoors and secrets of instructions. My crazy ASCII characters, crazy XML characters, whatever, math numbers, whatever, that will trigger the models to go into a place that they do crazy stuff. But we don't know that because we don't understand like the models, right? Like, people hack a model almost in English, oh, pretend that you're now writing ASCII art and you do this.

• 45:20Speaker 04

  Like, that is, if you think about it from an exploit point of view, that is very basic, right? Like, there's, there's, there's going to be way more ridiculous, complex, but interesting, well, I guess from a security point of view, type of exploits that people will have once you start to understand how it works inside, right? Which is why I think we also need to start measuring almost like what, what is the behavior of models, right? Like, how do they arrive at those conclusions? And then even have models that can only do those bits? Does that make sense, right? So, so again, I want deterministic models. I want models that I can start to vouch for how they work and how they do, even if maybe sometimes they're a little bit less efficient, but you earn the explainability.

• 46:06Chris Benson?

  So I want you to actually, as we're starting to wind up here, I would like to ask you to even extend that a little bit in terms of kind of where you think that's going to go because you've already touched on, you know, trying to get to more deterministic models and some of the things you're expecting. If you were to blow that out in a slightly longer time horizon, maybe several years, where do you think all this is going to go? And how do you think it might get there speculatively, like recognizing that we're just doing the crystal ball and asking you to tell us, kind of when you go to bed at night and you're thinking about this, what do you think's going to happen?

• 46:43Speaker 04

  Okay, there's multiple areas, right?

• 46:44Chris Benson?

  Sure.

• 46:45Speaker 04

  I feel on the whole fake news, creation of crazy content, using AI for the attack, that's going to grow, right? There's business models around it. I actually think that's going to force us to have deterministic improbance on news. So I think that's a good thing, right? It's going to be bad, but it's going to force us to address that problem, right? There is a level here that we're going to have to control it, just like we control nuclear weapons, right? Other things. I think there's a level here that we have to be careful, right? Not to create things that go completely out of control. And we might have a couple, but that's, that's a bigger problem, right? That needs to be addressed. Where I think where we're going to have the biggest impact is like, I think Kevin Kelly had a great phrase in one of his books. He talks about AI even before ChatGPT, but he talked about AI will

• 47:29Speaker 04

  become like electricity, right? You'll become embedded in all these little things, but it's not a massive thing. It's little bits, right? It's little things that you slowly start to have that introduce a level of intelligence that we don't have today, right? So if you think of most of our interactions, they don't have a lot of intelligence, right? They don't learn. They don't, you know, but, and I think as, again, the models become small to run in your phones, as the model becomes small to run in microservices, I think what will be very powerful is the creation of these little, lots and lots and lots of little use cases that really make a difference. And then you start to trust it, right? And then you compound them on each other, right? And my instinct is that anything that relies on a black box eventually will blow up, right? And when it blows up, people push back and

• 48:14Speaker 04

  going, whoa, whoa, whoa, we can't have that. It's okay for proof of concepts, but that cannot be doing stuff. Also, because technology now, I think had got to a point where we would drown in complexity, right? We would drown in all sorts of things that, you know, we don't fully understand, it don't connect the dots, and even the application systems, and they're so complex, and companies are so complex, right? I think understanding them in the smallest way, that will dramatically change. That's how you change society. You change society by literally introducing something that becomes really powerful, really useful. And the final point I want to make here is I think there's a ridiculous opportunity for reframing education, right?

• 48:55Speaker 04

  And to finally create a learning environment that individuals can be learned in the best way for them, right? So I think we can change away how we learn, continuous learning to be a thing, but also how we change the education from being like a memory kind of exam-based stuff, right? To actually be about learning, right? And now it's about creating customized learning paths to the individuals, right? And but also, I guess, just a final point is that this means that the human is literally the one that is ridiculously important here. This is a tool to help, you know, the human to be even more productive, the same way that we use the internet, the same way that we use the hammer, the same way we use electricity. It's just a different one, right?

• 49:37Speaker 04

  That we had before because he can understand language and we never had that before, right? So I think it's an insane opportunity. Um, you know, but the attack side, yes, it will go, but we didn't need, we didn't need Gen AI for people to create ridiculous attacks, right? But on the defense side, I think it changed the nature of the game, right? So I think it's very exciting about. I hope that answered your question.

• 50:01Daniel Whitenack?

  It was a great answer. It's awesome. Yeah, thank you so much for taking time, Dennis. This has been great. I really have been looking forward to this and love the conversation, love this idea of kind of thinking about knowledge and data and model and how those are connected or not connected. Please continue your great work. It's a great contribution and yeah, thank you so much for taking time. It's been a pleasure.

• 50:23Speaker 04

  My pleasure. Great talking to you guys.

• 50:29Speaker 00

  Thank you for listening to Practical AI. You know what's cool? Free stickers. During the month of September, we're mailing out Change Log sticker packs to everyone who leaves us a thoughtful five-star review or blog post about our pods. Simply email proof of your review to [email protected] alongside your address, and we'll mail out the goods anywhere in the world. Once again, that's [email protected]. Picks or didn't happen only in the month of September. Let's do this. Thanks again to our partners at fly.io, to our beat freak in residence, the one and only Brickmaster Cylinder, and to our longtime sponsors at Sentry.

• 51:13Speaker 00

  Use code CHANGELOG when signing up for a new Sentry team plan and save $100. That's all for now. We'll talk to you again next time.