Serving deep learning models with RedisAI

Redis is a an open source, in-memory data structure store, widely used as a database, cache and message broker. It now also support tensor data types and deep learning models via the RedisAI module. Why did they build this module? Who is or should be using it? We discuss this and much more with Pieter Cailliau. Sponsors: DigitalOcean https://do.co/changelog – The simplest cloud platform for developers and teams Whether you’re running one virtual machine or ten thousand, makes managing your infrastructure too easy. Get started for free with a $50 credit. Learn more at do.co/changelog https://do.co/changelog. Brain Science https://changelog.com/brainscience – For the curious! Brain Science is our new podcast exploring the inner-workings of the human brain to understand behavior change, habit formation, mental health, and being human. It’s Brain Science applied — not just how does the brain work, but how do we apply what we know about the brain to transform our lives. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Featuring: Pieter Cailliau – Website https://redislabs.com, GitHub https://github.com/K-Jo, LinkedIn https://www.linkedin.com/in/pietercailliau, X https://x.com/cailliaupChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Redis https://redis.io/Redis modules https://redis.io/modulesRedisAI https://oss.redislabs.com/redisai/Hangar https://buildmedia.readthedocs.org/media/pdf/hangar-py/latest/hangar-py.pdfHangar on GitHub https://github.com/tensorwerk/hangar-pyUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at Change Law because of Rollbar. Check them out at rollbar.com. And we're hosted on Leno cloud servers. Head to leno.com slash change log.

• 0:16Speaker 00

  This episode is brought to you by Digital Ocean, the simplest cloud platform out there. And we're excited to share they now offer dedicated virtual droplets. And unlike standard droplets, which use shared virtual CPU threads, their two performance plans, general purpose and CPU optimized. They have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, C I C D, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers, or active front end web servers that need to be full duty CPU all day, every day. Then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very competitive, starting at 40 bucks a month.

• 1:01Speaker 00

  Learn more and get started for free with a $100 credit at do.co slash change log. Again, do.co slash change log.

• 1:21Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slide with us around various topics of the show at change.com slash community. Follow us on Twitter. We're at Practical AI FM. And now into the show.

• 1:46Daniel Whitenack?

  Welcome to Practical AI. This is Daniel Whitenack. I am a data scientist with S I L International. And I'm here with my co-host, Chris Benson, who is the Chief AI Strategist at Lockheed Martin. How are you doing, Chris? Doing great, Daniel. How's it going today? It's going great. You know, I finally got a break in the heatwave that's been. All over the US here. I know you're a little bit further south than me, but I stepped out yesterday and I almost thought I needed a jacket because the difference in temperature was so drastic.

• 2:21Speaker 02

  Oh, I haven't gotten the benefit of the cold weather down here. I'm in Atlanta, and so it's still quite toasty and humid here. You know, last week I was up in Boston the whole week, and I was really counting on nice, cool Boston weather, and it just didn't happen. It was only about five degrees cooler up there than it was down here. So it was sad.

• 2:42Daniel Whitenack?

  Yeah. Well, I have a bit of story. So when I first joined, or my first job as a data scientist was with this startup. And, you know, at the time, everybody was into microservices and all of this stuff. And, you know, we started talking about microservices. I learned what those were, but they kept saying things like Redis this and Redis that, and we're going to use Redis for this and Redis for that. And eventually I learned that Redis is this cool data thing that can be used for like message passing and data caching and data store and all this stuff. And recently I saw that Redis released this project called Redis AI. So that's going to be the topic of today's conversation.

• 3:27Daniel Whitenack?

  We're joined by Peter Caio, who is a Senior Product Manager at Redis Labs. Welcome, Peter.

• 3:33Speaker 03

  Hi, guys.

• 3:34Daniel Whitenack?

  Yeah. So Peter, could you just give us a little bit of a background about how you got into software and how you ended up at Redis?

• 3:42Speaker 03

  Oh, that's a, well, it's quite a long story, but I'll try to keep it short. So effectively, I was, when I was a kid, I was into astronomy, and I wanted to always kind of like motorize my telescope, so it could effectively follow the turning of the sky so to have long exposure of photos.

• 4:03Daniel Whitenack?

  Right. That sounds awesome.

• 4:04Speaker 03

  Yeah, that brought me into electromechanical engineering and then afterwards, computer science. My first job was at TomTom. Well, sometimes in the US, they say TomTom. So it depends a bit, but. I was working on the, to the portal navigation devices, right? I was working on the map back in, in, in Belgium, in Ghent. Map quality assurance was, is a difficult problem, right? So Tom Tom tries to differentiate in how, how well the quality of this, is of these maps. And they have many rules. And one of those rules is a very expensive rule. It's, is the map still connected, right? Can you still travel from everywhere to everywhere? And you could imagine, for example, you have a continent, you have an island, there's a bridge in between.

• 4:49Speaker 03

  And the map editor says this bridge is now one-way traffic. So in the morning you go to the island and the evening you want to go back, but your device says, no, sorry, can't do.

• 4:57Daniel Whitenack?

  That would be a problem.

• 4:59Speaker 03

  That would be a problem. And so for that, effectively, we, we were using a graph database. Uh, we were effectively using Neo4j. And that rolled, well, I rolled into, to, uh, to the NoSQL database world. Uh, because I joined Neo4j, um, worked there for three, three and a half years. And, and afterwards I, I joined Redis, um, first as a solutions architect. And then afterwards they said like, hey, we've got the, one of these modules here is, is graph related. And I joined effectively the product management team to, to initially first, um, support the, the Redis graph module. Uh, but now, uh, I have seven of these modules, uh, under my, uh, what I call it supervision or umbrella.

• 5:37Daniel Whitenack?

  Oh, wow, that's great. Yeah. How many in total, like how, what is the ecosystem of modules around Redis? I imagine there's, there's quite a few.

• 5:45Speaker 03

  There, there is quite a few, right? So I don't, I don't know exactly, but I think we have 2025 open source modules. May, maybe it's a bit good for the audience that I introduce what, what these modules are and effectively introduce what Redis is, because that, that make, will make the story nicer.

• 5:59Daniel Whitenack?

  Yeah, that's no problem.

• 6:01Speaker 03

  So, so Redis is a, is a key value store. Um, it's an in-memory database, it's a NoSQL database. The most typical in-memory database that most people know is Memcached. And in Memcached, the value is always a string. In Redis, however, you can have different data types. It can be either a string, but it goes also be a numeric value, a list, a sorted set, a hash. So you have many data types, you can effectively see this memory as a service. So if you're coding, you could say, okay, I have this data structure that is residing within a distributed database.

• 6:29Daniel Whitenack?

  So could you just give an idea, so you've kind of called out Redis as this key value store and a NoSQL database, and you also mentioned like graph databases. Could you just kind of generally describe, you know, how the NoSQL databases are separate from what people might have in mind that they execute like select statements on?

• 6:51Speaker 03

  Sometimes confusing, right? So NoSQL stands for not only SQL. So it was the movement that, well, we have, we're living in the, some buzzwords, right? So there is big data, there is much, much, much more data.

• 7:04Daniel Whitenack?

  Yeah, we have a, we have a podcast about AI, so we totally get the buzzwords.

• 7:12Speaker 03

  Nothing but buzzwords. Nothing but, but yeah. So the database had to consume or to ingest higher volumes of data. And, and, and there were some performance issues. So, so it's not, not the entire picture, but what, what people kind of like did was they create some databases and they threw away everything that was kind of like reducing the performance or slowing down the performance of your database. So NoSQL databases typically have a, well, unless, well, they also have a different paradigm, but they also have like less strict authentication. They're quite frequently, they're schemaless, so you can put lots of heterogeneous data in, in, inside it. And in that, in that NoSQL section of database, you have quite a lot of database models, right? There's, so it's a key value store, but there's also graph.

• 7:57Speaker 03

  You could see search also as a, as a database model. There's a time series, which is effectively ramping up lately, which is a lower database model, for which we also, by the way, have a very interesting module. But yeah, I don't know if, if, if you want me to explain further, but talk more about NoSQL databases or how they, they came to life.

• 8:15Daniel Whitenack?

  That was really helpful for me. I know that if we kind of zero down in on the key value stores. So you mentioned like one distinguishing feature of Redis is that the values don't have to just be strings. So you gave some other examples of things that they could be. I forget what those were.

• 8:34Speaker 03

  There is, for example, there is a list. So it's a simple, it's a list data type, right? So you can, effectively, it's interesting. Sometimes when I give a talk, I ask people. So for example, let's do the exercise, right? So you have two sets of items, and inside Redis, they will reside as a set. So for example, imagine you have two products and they have a set of tags. Now, what could you do with these two sets? Which questions could you ask the database with two sets? Maybe I should hint.

• 9:06Daniel Whitenack?

  Overlapping entries or something like that?

• 9:12Speaker 03

  Correct. So you could do an intersection, you could do a union. So this is kind of like what Redis is, right? So your data structure, all the methods you could do in your programming language, they kind of like have commands that you could execute against Redis.

• 9:28Daniel Whitenack?

  Okay, so you could have like a bunch of values as lists and execute some command or in some Redis client or something to get the intersection of those values.

• 9:41Speaker 03

  Correct. So in a list, you could get the first element, you could get the last element. You could query your range or get the exact element. You could also pop an element.

• 9:51Speaker 03

  But in a set, you could ask, for example, well, is this in a very efficient way? Does this element belong or is it present in your set, etc.

• 10:00Speaker 02

  So I was gonna ask, could you kind of like, what are some of the most typical ways that Redis users are using Redis? What's a couple of really common use cases that people are most likely to do?

• 10:12Speaker 03

  It's very broad, right? It's a general-purpose database. You could use it for anything. But the most common thing that people are using Redis for currently is for session caching. Um. So so you're shopping baskets, et cetera. So everything that you you you keep track of in sessions, also for caching. Um. But there are also many things you could use as a queue, right? So uh um there is next to the, well, originally, there was the list. Um, uh, data type where you could effectively distribute messages with uh um blocking on these uh um on this list, on on retrieving the first element, the last element.

• 10:50Speaker 03

  Um. But now nowadays, we also have a a streams data type, um which is, well, we'll we'll call it Kafkaesque, right? So it has similar capabilities. Um, you you can have uh um, you can use it as a stream or as a topic, so sorry, sorry, as a topic or as a queue. So in a queue, you can have, for example, a consumer group, uh and each message will only be consumed by a single consumer. Uh. But if you have to use it as a stream, you could, for example, say, I want to start reading the stream from here, or I just want to catch up from the end of the stream, et cetera. And so so also for that, uh uh with that specific data type, you could effectively do a message broker or events, uh um, event sourcing. Um, uh, use cases. Gotcha. Okay. But there are many more. There are many more. Uh. People use it sometimes to do distributed locking.

• 11:35Daniel Whitenack?

  Yeah. So uh you've kind of al- also alluded to the fact that like at least some of these modules of Redis are are open source. Um, like Redis as a as a company, I know that like there's some things that are that are like open or open-ish. Uh, like, how does the company company operate? And like, Redis itself, like, can you find it on GitHub? And and what's the kind of the the model around Redis?

• 12:01Speaker 03

  That that's a good question, right? So so Redis itself is entirely open source, so it's a BSD uh license. And so you you can take it, you can you can uh fork it, you can resell it, you can do whatever you want with it. Um. Then there is a piece where where we have these modules. These modules have a a Redis source available license. Um. So the idea is that you can effectively do anything with these modules, uh apart from creating a database as a service um with these modules. And so you can you can embed it inside your product. Um. And effectively, it's a bit of a strategic way to to to compete to to to cloud providers. Uh, just simply taking all the open source products and then effectively offering them as a service.

• 12:42Speaker 03

  The last part is is our Redis Enterprise part, which is a is a closed source part. It it doesn't have any. Data or engineering specific capabilities, however, it has lots of enterprise features that your company might be interested in or is most likely interested in. So there's easy to scale your database, there is, effectively, it's an entire database management system. The way I sometimes position this is that, do you want to become an expert in building your application, or do you want to become an expert in deploying Redis, right?

• 13:11Speaker 02

  Yeah, that makes sense.

• 13:14Speaker 03

  If you don't want to do the second one, then we've got an enterprise solution for that.

• 13:17Speaker 02

  So could you give us some examples of some of the modules that you have available? I'm curious about what the scope of those are.

• 13:25Speaker 03

  Well, the story effectively is, so we have all these data types. And the original core contributor or creator of Redis, Salvatore Sanfilipe, he was receiving lots of requests and said like, "Oh, you don't have this data type. How about this data type?" So he created an API, so you could effectively create your own modules and your own, sorry, your own data structures and your own commands against those data structures. So as we, the commands were, for example, do the intersection in between two sets, and the set was effectively the data structure. And that way, effectively, you could say like, "Well, I'm receiving lots of requests. Now you can build your own data structures." And by creating those, you effectively inherit all the goodies from the ownership, right?

• 14:08Speaker 03

  So Redis has many things inside, so it has memory management, but it also allows you to cluster your database. And one of the misconceptions that many people have, by the way, is that Redis is purely a cache, it's effectively a database, however, they live in memory first, and then we have tunable durability. So now, if you implement your data structure inside a module, you can also benefit from that durability and also have an entire spectrum of client libraries that can effectively connect to your database and reach out to those data structures in those modules. Now, to answer your question, right, so there are quite a lot of them, but the ones that we effectively think that are, well, that we effectively...

• 14:51Speaker 03

  Contributing the most to, there are seven of those. So the most mature one is most likely going to be Redis Search. So it's an inverted index. It's kind of like Apache Lucene. And Apache Lucene, for those who don't know, Apache Lucene is effectively the core building blocks of Elasticsearch and Solr technology. So, but we've built our own inverted index.

• 15:15Speaker 03

  It's written in C. We serve the entire data or the index from within memory. And also, the index is instantly reflected. So we do have some advantages there. We don't have feature parity, of course, to these products that have been around for decades, I think. However, we get a very, very high performance in that search. And there is Redis Graph, which adds graph capabilities. It's, well, I know all these modules, right? So I can talk for hours about them. So Redis Graph itself is a very interesting technology. So I always try to differentiate graph-native technologies and graph technologies. So you can put a graph API on top of a relational database. However, yeah, it will be quite slow, right?

• 16:00Speaker 03

  You make one table which is nodes, another table which is relationships. But when you have to do a traversal or search, for example, a Dextra algorithm, which is a search algorithm between two nodes, you will have to do a join in between those tables continuously. So a graph-native database.

• 16:15Daniel Whitenack?

  I'm so glad you're calling this out because I actually got burned by this very issue on a project where, like, we were trying to implement a graph database and I mistakenly chose like a graph layer on top of MongoDB, which is another database. And it was so slow. And basically, the graph logic was really interesting and good for the project, but we got burned because of this issue that you're talking about. So thanks for explaining that.

• 16:45Speaker 03

  Yeah, yes. So the difference is that you have graph-native, you have an O1 time complexity to go from one node to another node, right? So instead of an Olog and a relational database. And typically, you do try to keep the data as close as possible residing within the same key effectively in Redis.

• 17:04Speaker 03

  And we've got Redis Bloom, which is a set of probabilistic data structures, like a Bloom filter, a COCO filter. And we also add a top K. And we've got Redis JSON, which, which is going to take your JSON document and it's going to split it up inside a tree. So that in atomic operation, you can, in a large JSON document, you could, for example, append some data to an array. Or you could increment a numeric value inside your JSON documents without having to fetch it and, and putting it back. And then there is Redis Time Series, which is the last one that went GA. Which effectively adds time series capabilities to, to, to, to Redis. It allows you to do, for example, in, in the role of IOT, it's very interesting that you, you've got lots of raw data, but you also would like to down sample it.

• 17:49Speaker 03

  You, you could do that inside Redis already, right? There are many ways to do that. However, you had to write lots of client side codes. And, and Redis Time Series comes with a toolbox to do aggregations over, over lots of samples or to down sample them. So you, you, you don't keep all your raw data the, the further you go into history. One more interesting module before we dig into potentially into Redis AI is, is Redis Gears. And it, it's interesting to mention because Redis Gears will be kind of like the serverless of, of Redis. And so, so you can do two things. You can, can do like MapReduce type of operations on top of your entire data sets. And because well, Redis allows you to key value store, so you can shard it, you can partition it.

• 18:34Speaker 03

  And so if you want to run a query across all these shards, there was no clean solution for that. So Redis Gears will, will, will give that to you. The second part is that it can also trip, well, execute certain scripts based upon certain triggers or certain events. So it could be, for example, you add something to a stream or you change a certain key. And but it could also be time based and then you execute certain, certain codes. And this kind of like makes the entire multi-model story very beautiful. And, and I know in our, our company, I'm, I'm the first one to say, if somebody says, hey, multi-module, multi-model, please be, be cautious about that. And, and so it's not because we have multiple modules in a database that it's, we're a multi-model database.

• 19:18Speaker 03

  But Redis Gears kind of like allows you to do that. So you have, for example, a stream of data, you get some, some. Some name change, for example, you could say, hey, the script is going to react upon that new piece of data is going to say, hey, Reddit search, please update the name of this person, but also, hey, Reddit's graph, make sure that graph person or the graph node is updated. So it can effectively talk to all these modules. And then it becomes very, very interesting.

• 19:48Daniel Whitenack?

  So you just mentioned kind of this idea of multi modules or multi module and like the ability to run scripts, the ability to put things together. You also mentioned Redis AI, which is really what we want to get into in the rest of this discussion. But I was wondering, just kind of in terms of in terms of background, what is the background on on Redis AI in terms of why the team was motivated to to create it? Was it something that users were were asking for? They're like, oh, we love Redis, but we want to integrate this somehow with like AI models? Or like, what's the backstory on that? And kind of how did it? How did it come about?

• 20:30Speaker 03

  It came effectively from three ways, right? So the first one is that Salvatore, as when he created this module API, the first module he created was Neural Redis. So that was the first kind of like indication that there was in it could be potentially in need for that. Then there was also some some engineers inside Redis Labs who created a module called Redis ML. So they thought that they could, for example, have very specific data structures, for example, for a random random forestry, for some regression, some some specific data structures for these models or these AI models, and effectively host them inside Redis. And then there was also somebody who was creating Redis DL, another module, which is then focused on on on deep learning. And we kind of like combined these efforts and

• 21:15Speaker 03

  we created one one model module, which is called Redis AI. Now, that's how it came to life, right? But the need or explaining the needs is is is is is also quite important. So um there are two things we believe there, right? So there is data locality and then there's effectively the um uh uh the DevOps parts of of of publishing your your model. The publishing of your model, it's it's quite, it it's always straightforward, it's always convenient. So you can, for example, have your your your your model, you wrap it inside a Flask application. Uh but then you have to scale it yourself, you have to add all these things uh uh um manually. Um so but but Redis, we effectively have all these things already.

• 22:00Speaker 03

  We can scale it, we can, we can cluster your database. So we can, we have all these clients already, they can connect to it. We already have high availability, we have effectively also durability. So if your your data, if if your Flask application goes down and you didn't, for for example, well, if there's no high availability, then your your your your model serving would no longer be available. And so all the goodies that that Redis brings as a database, you suddenly also get for free if you would add Redis AI to uh or some some these models to Redis. The second part is that that we believe is is the data locality. Um for that, maybe it's interesting to to give a model uh or to to to to tell a story, for example.

• 22:42Speaker 03

  You have a chatbot application and this chatbot application has a certain model, so it takes in two tensors or takes in two uh uh input parameters. The first one is your your latest message that the the chatbot received, but also the second one is the intermediate state. Uh so there's intermediate state of the past or the history of that conversation. And combined with that, it it it produces an output and the output is the response, but it's also a new intermediate state, uh that is a new history of your conversation. Um now you you wanna keep that as close as possible to that model. You don't wanna go and fetch it from another database, then bring it to your application or to to to your own Flask wrapper where you've written uh um where you try to deploy your your model, adds another uh uh data fetching.

• 23:26Speaker 03

  For sure, if that data you wanna fetch becomes large, uh there will be some some some latency. And and well, what we believe in Redis is that everything should be extremely fast, right? Because there's a high throughput, well, there's a demand for high throughput databases or a high high throughput requests uh with very low latency. Everything needs to be more and more snappy by actually creating these data structures or by creating Redis AI. We could now Run your inference where your data lives, right? Because people sometimes also have, for example, some extra input data, get a user profile and feed it to your model to do a better inference or classification, right? So that now that data doesn't need to be fetched from another host or from another database.

• 24:11Speaker 03

  It's already there. It can live within Redis next to the model.

• 24:13Speaker 02

  So, you know, as you talk about the chatbot example, you know, where is the tensor serving where you're executing deep learning models? How does that fit into this? Could you kind of give it to us, give the context of that so that we can kind of map it together?

• 24:26Speaker 03

  There are three data structures inside Redis AI that we add. So there is a tensor, there's a script, and there's a model. So you can effectively now, the key value store, the value now can be a tensor. So you can host your tensors inside Redis. Is that answering your question?

• 24:41Speaker 02

  I think so. I was just wanting to kind of like break it down in the sense of with the chatbot kind of walking through how you're using the module, how Redis AI is fitting into that example. If, so if you were coming into it for the first time as a user that was wanting to use this, just kind of taking that as a, as a contextual example about how I might do that. Like if I sit down after, after our conversation and try to do it, can you just kind of describe it end to end in terms of how that happens?

• 25:07Speaker 03

  Right. So, so your training part is still going to be, going to be the same, right? So, but what, once you, you've, you've trained your model and you're sure that it's the right fit for it, you can effectively import it inside Redis AI. So we're building quite some tooling around it to make it more easy so that you can do it, for example, directly from, from, from scikit-learn. There's already a connector for that. We also have connector, for example, from Spark. So you take your model, you can put it inside Redis. The next thing is if you want to build an application, you will have some, some client library that is going to connect to Redis. And from the client library, you could post effectively your tensor inside Redis. You could say, hey, here's my new message.

• 25:52Speaker 03

  There is also the output tensor, well, there's also the input tensor of the intermediate state. And then you can effectively run that model. It's another command that you can run from the client side or the client library application side to do your inference.

• 26:05Daniel Whitenack?

  Yeah, I'm gonna kinda, so I have a theory going in my, in my mind and I wanna kinda check it. So you mentioned the three data structures. You got the tensor, the graph or the, or the model and then the script. So would it be true that you could kind of have tensors stored in, in Redis? And these might be like test examples? Or maybe these are like knowledge base articles? If you're answering questions out of a knowledge base? Or maybe it's like some images or something? And you could execute like certain of the scripts of the script database or data structures on those tensors to maybe do like preprocessing or something? And like combine them in interesting ways before they would be passed off to maybe the graph data structure

• 26:50Daniel Whitenack?

  to actually perform an inference with, with a model? And then maybe it would go back to the scripts for like post-processing? And then stored, the result stored in like tensor data structure again? Is that, is that kind of on the right track or?

• 27:04Speaker 03

  That is exactly it. That is exactly it. That's how we want it to be. So you can have these, these three data structures and you can combine them and mix and match them as you go. So you can also use that as, well, if you would like to run several models and combine the results, you could also use a script and run them several graphs underneath. One more thing is that, well, it's, it's, that is coming up is effectively you will have a command, which is a direct, basically graph. So you can effectively execute those commands combined, right? So you could, for example, do, you set a tensor, but then the output is passed to a script. The script's output is then passed to a model. And then the out, model output is once again passed to a script. And you can, and the outputs of that script could be a tensor that is persisted inside Redis, right?

• 27:47Speaker 03

  So you can kind of like pipeline those, those commands in a single, a single call towards Redis. But your explanation was spot on.

• 27:56Daniel Whitenack?

  Awesome. Yeah, so I'm just a part of the reason why I have that, excuse me, I have that in mind is like I have, I have a direct use case I'm thinking of in the problems I'm solving now. So we have like a bunch of knowledge-based articles that we're doing reading comprehension on. But we're also doing like a full text search of those based on TF IDF. And so my thought is, oh, well, what if I could? So I don't know, check me on this to see if this would be a proper way to use Redis AI, but I could like store the knowledge-based article text as certain, maybe string elements, string data structures in Redis.

• 28:36Daniel Whitenack?

  And then I could use maybe a script in combination with the the graph that would store like my vectorizer, my TF IDF vectorizer, to transform those into like actual tensors in the tensor data structure. And then when I would do like a document search on those, maybe I could use some of the like overlapping or score generation script or something to actually do the do the matching. I don't know. What do you think?

• 29:04Speaker 03

  I think that makes perfect sense to me that this is effectively what we want to do. And it can even be more beautiful. Well, you mentioned, for example, you would fetch your knowledge graph as a tensor. If your knowledge graph is changing over time, right, you could effectively host that knowledge graph inside Redis graph. So then it's there, it's living as a database, you can effectively let it evolve over time. It might be, for example, that you want to add scoring, or you could say, well, on given searches, you want to bump certain certain nodes or certain relevancies. Or you might want to make custom queries you want to use as an input for your graph, right? If you want to say, well, if you want to make something very tailor-made. And then effectively, gears could effectively be also be cluing all of these things together, right?

• 29:49Speaker 03

  So your script could already, well, not at this moment in time, but it's something we plan to do. Your script could effectively then consult Redis graph and fetch that data for you.

• 29:58Daniel Whitenack?

  Okay, yeah, so there could be like, so there could be routing logic that routes between all of these different scripts and the graphs, and the scripts could call the graph, so the scripts could get get data from the tensors and all of that. Is that right?

• 30:13Speaker 03

  Yes, correct. It's a bit confusing here because we're saying graph. We can say, well, that's why we renamed it to, to, to model inside Redis AI.

• 30:22Daniel Whitenack?

  Okay.

• 30:22Speaker 03

  Because we already have Redis graph. That's why we.

• 30:25Daniel Whitenack?

  Yeah, I know. That, that makes sense. I'm looking at one of these figures from the, from the Redis AI docs. And there's a graph blob section. So that, that makes sense. That would, that would help with the, with the confusing terminology, which is, I guess, always the hardest part of software engineering.

• 30:43Speaker 03

  I'll take that as a note to myself to update the documentation. Yep.

• 30:48Daniel Whitenack?

  Right. This is, this is truly practical AI. So, you know, solving, solving problems on the fly.

• 31:08Speaker 00

  Hey, guess what? Brain Science is officially launched. Episode number one is on the feed right now. So head to changelog.com/brain science to listen, to subscribe and to join us on this journey of exploring the human mind. Once again, changelog.com/brain science or search for brain science in your favorite podcast app.

• 31:46Speaker 02

  It looks like Redis AI is currently supporting PyTorch via live torch and TensorFlow via live TensorFlow. And also Onyx runtime is backends. That's pretty awesome. How big of a role did Onyx play in your strategy, you know, around modules and stuff? And can you describe it a bit to our users on even defining what Onyx is actually?

• 32:07Speaker 03

  Onyx runtime is effectively the Microsoft backend or the Microsoft variant. And the reason why we added it was because we wanted to combine, well, we used to have Redis ML, right? So that was purely machine learning. And, and Onyx runtime effectively allows you to also serve machine learning models, well, compared to PyTorch, for example, on TensorFlow.

• 32:29Daniel Whitenack?

  So Onyx provides like an intermediate between a bunch of different frameworks, right?

• 32:35Speaker 03

  Yes, it does.

• 32:36Daniel Whitenack?

  Yeah, so the, am I right that you have like PyTorch and TensorFlow, sort of the native libraries? And then you kind of use Onyx for like everything else? Is that the kind of thought process or?

• 32:47Speaker 03

  That is a thought process, exactly, right? So we can, we can export, we can use Onyx runtime as an intermediate format. Effectively, we were using it in, in, in certain ways where we, for example, wanna fetch some moles from Spark. We can effectively then transform them in Onyx runtime, then, and then upload them into our backend. It's interesting to, to talk about these different backends because the, the model commands on, on how you execute or you effectively wanna, wanna do your regression classification, agnostic from, from the backend that it's running for. So when you would set your model, when you say, hey, here at R S A I, I want you to host or serve my model, you would have to, to add some, some, some backend specific stuff.

• 33:32Speaker 03

  But once you, you wanna run it, it, it doesn't know anymore which backend is effectively underneath. Obviously, R S A I knows, but it's, it's your, your client doesn't need to specify, hey, run me this with TensorFlow.

• 33:45Speaker 03

  So which makes that your, your, your client library, or well, your application developers that wanna run,

• 33:52Speaker 03

  wanna work with your model or your, your, as a data science trained model, they, they get a fixed API. And as you as a data scientist, you decide, well, over time, we think that our model in, in PyTorch is better than our model in TensorFlow, you can just update that by setting or setting a new model with the same key inside Redis, effectively R S A I, and, and all your client libraries will still keep on working.

• 34:18Daniel Whitenack?

  Yeah, that's, that's pretty cool. So. I'm just trying to think, so what you just said makes sense. I'm trying to connect it to like, you know, practically if like, let's say we take the example that I had before. So I've got like my vectorizer that I've created, let's say in scikit-learn, which I can convert to Onyx format. And I want to, and I create a bunch of knowledge graph or knowledge base article entries in Redis. What would be kind of the steps I would go through to, to actually get something running

• 34:57Daniel Whitenack?

  to where I could take all of those articles in and then vectorize them with my, with my scikit-learn TFIDF vectorizer and then save them to back into Redis. What would be required for me to do? Would that be like writing some, some Python? Would that be like some sort of custom client? What all would be involved there?

• 35:21Speaker 03

  So it's a good question. We do have a couple of client libraries. So there's a, there's a Python client for Redis AI. There's also a conversion toolkit that we kind of like created to help you convert in between these different models if that would be necessary. So via Onyx runtime, so, so which we can probably share later on or at the end of the podcast. So from within your Python code, you could effectively just publish your, your model inside Redis AI. You could push it directly. You could connect, hey, here's my database. I want to push this model. Obviously there might be other ways that you still want to, want to do this, right? You might have some intermediate, some, some versioning of your, or exporting your model as a binary.

• 36:03Speaker 03

  So you could actually version it and then you could ship it to DevOps. So they can effectively, well, there is, there is clearly, there is, there is a component missing there to make this easily, right? To, to, to, to version your model and to effectively say, hey, this is now the new release we want to publish inside. Uh, Redis AI, you could do it directly. You could say, hey, here's my, my, my client and, and I directly push it to Redis. And then afterwards, if you wanna, uh, wanna do that, uh, you, you wanna run that model, you can once again use the same client. Um, but however, that client can also be wrapped in any other application that wants to consume that model.

• 36:40Speaker 02

  So I'm just kinda curious. We've kinda talked a little bit about some of these use cases. Are there any, before we move on, are there any other really typical use cases that you're seeing Redis AI being used for that we haven't already covered?

• 36:50Speaker 03

  One of the things where we think is, is, is, uh, transaction classification. And the reason why we, we think that's a very good fit is because it's, it's, it's a high volume, uh, high number of, of requests you wanna do, and you only have limited time to do so. So you don't wanna waste, uh, time to find all your metadata or to fetch all your metadata from user profiles uh, that live in, potentially in, in different data source. Uh, so I think transaction classifications, some, some fraud detection uh, um, are, are the ones we think that, that will, uh, really benefit from, from that data locality and the scale we can offer with, with Redis.

• 37:27Daniel Whitenack?

  I have a kind of, um, I guess, more philosophical question to, to ask. So I would say that like, probably, at least from my experience, like there, there's probably a lot of, like, maybe even most, like, software engineers these days, like, they've heard, at least heard of Redis, or they may even be familiar or worked with it, but maybe like data scientists and AI people are like less familiar with Redis. So, um, do you envision, I guess my question is like, there's a lot of people already using Redis for certain things, like, uh, you've already mentioned like message brokers and other things where they, they have data already flowing through Redis.

• 38:07Daniel Whitenack?

  Do you think that it's likely that kind of the software engineers already using Redis will kind of be able to kind of bolt on AI sort of functionality into those existing applications? Like, maybe they wanna introduce like fraud detection in terms of their, the traffic going across their network or something like that? Or, um, are you kind of hoping to kind of position Redis AI more as like a general purpose sort of model serving framework where, um, you know, AI or data science people, when they go out looking for the best way to serve their, their model, kind of Redis AI is up there with like, you know, TensorFlow serving or, or whatever else it is. Do you think that there's like, uh,

• 38:46Daniel Whitenack?

  I don't know, maybe it's unclear right now in terms of which direction it was go, but it would go, but I was curious to hear your thoughts.

• 38:53Speaker 03

  I think we would love it to be both effectively, obviously, but so I do think that we still have to build some tooling around this, right? So to make the full-fledged model-serving engine, to make it very practical. By the way, one thing I also want to mention here is that we're effectively, not all the developments of Redis AI is done by Redis Labs. It's also done by Tensor Work, so we're partnering with them. And Tensor Work is a company that is building kind of like several tools for AI purposes. So one of the things they're building is, might be interesting for you guys to also look into, is a versioning system for data.

• 39:34Speaker 03

  So you could effectively, well, like Git, you could effectively branch and make PRs on top of your data.

• 39:43Daniel Whitenack?

  Yeah.

• 39:43Speaker 03

  The data you want to serve to, well, it's called Hangar, and I'll also, well, recommend you to look into it. But over time, effectively, we think we will get a very nice toolkit. And Redis AI will be one of those pieces into it, right? So if you want to be, I think, combined with Tensor Work, we would like to make the story where we can say, hey, we've got a complete solution towards you for serving your model. On the other hand, on the other end of the spectrum is, in that case, well, I don't know who the personas would be. For sure, it's a data scientist, but also, for example, DevOps people that want to have a good solution for preserving those models. On the other hand, there's potentially also data engineers and our software engineers,

• 40:27Speaker 03

  who do already have Redis as a tool in their toolbox. And with these modules, they can make all these beautiful new applications. So ways you could use Redis AI, combining them, for example, with other modules. So if you have a search application. And you wanna say, well, AI have the these top hundred results here, or have a hundred or two hundred results. Why don't I run them again through some model to to update or to to fine tune the scoring in between those results? Might have a big impact on the on the user experience. Um. So we we hope in that way, this is just an example, right? And we hope in that way that we will we will enable that many new or interesting applications will appear.

• 41:08Speaker 02

  Um. So we're kind of already talking about, you know, what you're thinking of going forward. So I'll just go ahead and, you know, ask, you know, what what does your development roadmap look like? Kind of what future directions are you interested in taking? And and I know that we've, you know, obviously there are features that that you're gonna have in mind. But, you know, it sounds like, you know, we've really covered a lot of tooling. Um. Are there any, as part of that, are there any kind of higher level applications that might sit on top of Redis as part of that development roadmap?

• 41:35Speaker 03

  Yes, we we should definitely look into that. And we're open for for for lots of tips and and recommendations on top of that for for that. We do have a kind of like a roadmap for Redis AI itself. We we do have a couple of features that we wanna add. So we would like this Redis Gears module to talk neatly to to to Redis AI. That might be interesting to, um, well, yeah, you have the script, but you could also fetch data more easily or maybe even connect to to to other data sources outside Redis. There is also some batching that we want to uh to to have, so so that we can effectively, you can run several models in in in in one go. So there will be also, well, another thing that we wanna add is is some tooling to get some performance statistics about how well your your model is running or behaving. So what is the the the average time?

• 42:20Speaker 03

  And and and the beauty about, well, the beauty that we we hope to to achieve is that, well, there's already an entire ecosystem around Redis. So some guy somehow will have written, for example, some Prometheus uh um connector, right? And so if you so if you can connect that up, then you immediately get a some Grafana dashboard out of the box that could show how well your model is behaving. That is one thing. I I'm sure we will have many more. There's some some versioning potentially of models that we also would like to do. So we we can have or we can keep track of several models or or versions. So you can have, you can switch in between them or do A B testing in between them. That could also be an interesting uh uh thing to look into in the future.

• 42:59Daniel Whitenack?

  Awesome. So as we kind of wrap up here, I was wondering if you could just share kind of maybe where, I think on one front, like where people that are interested in experimenting with Redis AI, where they should go to kind of get up and running, I guess. And then, second, I'm sure that it's, I mean, there's a lot of work to do. There's a lot of tooling that's needed, integrations, all of that stuff. So maybe there's people out there that have used Redis or not, but that would be willing to contribute. So where should they go to kind of get plugged in? So on one front, you know, people that want to be potentially users and then maybe people that want to contribute as well. What's the best way for

• 43:42Daniel Whitenack?

  those groups to get involved and find out more?

• 43:45Speaker 03

  So if you just want to get started with it, there's a Docker container. Effectively, all the info is on redis.io. To quick start, there's, for example, there's a Docker container, you can just pull it. It comes preloaded with all the backend. So it comes with TensorFlow, PyTorch, and Linux runtime backend. You can just get started with that. If you would like to contribute, there is a Google Groups where you can effectively, well, all Redis AI, right? So if you would search for the Google Group on Redis AI, where you can effectively reach out to all the developers that are working on Redis AI and contribute there, right? It's a GitHub project. Feel free to open issues. I mean, we would love that, right? All the feedback you guys can give us would be highly appreciated. I think we actually, we react quite quickly to those, those two, right? We would love

• 44:30Speaker 03

  to help, or to make Redis AI a success.

• 44:33Daniel Whitenack?

  Awesome. Yeah, I think that's, that's, that's great. And of course, we'll post all of those links that were just mentioned on our show notes. And of course, if you, if you're having trouble finding things or want to kind of follow up with questions, find us on our Slack team at changelog.com/community or LinkedIn page. We're happy to make the connections for you, forward on links. But I thoroughly enjoyed today's conversation. I was able to like nerd out a bit and think about my own, my own problems a little bit. So I appreciate you being, being patient with me, Peter, and sharing so much great information.

• 45:07Speaker 03

  You're welcome. I really enjoyed it too. So it's, it was an honor for me to be here. So thank you for asking me.

• 45:15Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor, go on iTunes, give us a rating, go in your podcast app and favorite it. If you are on Twitter or a social network, share a link with a friend, whatever you've got to do, share the show with a friend if you enjoyed it. And bandwidth for changelog is provided by Fastly. Learn more at fastly.com. And we catch our errors before users do hear changelog because of Rollbar. Check them out at rollbar.com/changelog. And we're hosted on Linode Cloud Servers. Head to linode.com/changelog, check them out, support this show. This episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com. When you go there, pop in your email address, get our weekly email, keeping you up to date with

• 46:00Speaker 00

  the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.