Private, open source chat UIs

We recently gathered some Practical AI listeners for a live webinar with Danny from LibreChat to discuss the future of private, open source chat UIs. During the discussion we hear about the motivations behind LibreChat, why enterprise users are hosting their own chat UIs, and how Danny (and the LibreChat community) is creating amazing features (like RAG and plugins). Sponsors: Fly.io https://fly.io/changelog – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog https://fly.io/changelog and check out the speedrun in their docs https://fly.io/docs/speedrun/. Featuring: Danny Avila – X https://x.com/lgtm_hbuChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Register for our next webinar (similar to this one) focused on multimodal AI https://tinyurl.com/genai-mastery3LibreChat https://github.com/danny-avila/LibreChatPrediction Guard https://www.predictionguard.com/Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:06Speaker 00

  Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related tech is changing the world, this is the show for you. Thank you to our partners at fly.io, the home of changelog.com. Fly transforms containers into micro VMs that run on their hardware in 30-plus regions on six continents, so you can launch your app near your users. Learn more at fly.io.

• 0:43Daniel Whitenack?

  Welcome to our next Practical AI webinar. This is our second webinar or live event that we've done. Chris, we've always done the podcast pre-recorded, and it's been fun. And last time to talk about text-to-SQL live, and now to have another chance to have a live webinar. Are you enjoying these?

• 1:08Chris Benson?

  I am enjoying them. I think we need to do these more often. They're a lot of fun.

• 1:11Daniel Whitenack?

  Yeah, yeah. And today, so I'll kind of frame the conversation for today. But first, let me also welcome Danny from LibreChat. He's joined us to talk about the topic today. Thanks for joining, Danny.

• 1:24Speaker 03

  Yeah, of course. It's an honor to be here.

• 1:26Daniel Whitenack?

  Yeah. Well, thank you for everything you're doing on the LibreChat project and in the community. Today, as those that are in the webinar know, and maybe if you're listening later, we're going to be talking about crafting the next generation of AI chat interfaces. And just to kind of frame the setup for this, we've talked a lot of times, and I still frequently, even this week, last week, I hear about people saying, "Oh, my company doesn't let me use the ChatGPT interface." Or even literally today, like three hours ago, I was in a meeting with a number of customers, and they were asking, the question was, "How do I get..."

• 2:10Daniel Whitenack?

  Chat interface that allows me to switch between models and like try different things with different models. Like if I want to try Lama three or something like that, are there ways for me to do that? Are you still encountering that as well?

• 2:24Chris Benson?

  Literally every day and I'm very sensitive to it. You may recall that I I was getting after teachers online and a teacher pointed out that, hey, this is not our choice. It's the school system. So I've been super sensitive to this ever since then. And we have challenges. I'm I'm hoping today can put a big dent in that one.

• 2:43Daniel Whitenack?

  Yeah, great. Well, I'm super happy that we have Danny with us because this is what Danny has devoted a huge amount of energy to with Libra Chat, both in terms of providing an open source chat interface, also providing a chat interface that allows you to plug in different AI systems, whether that be OpenAI or many others, closed source and open source types of models and systems, and providing even functionality related to, I think, even RAG and and plugins and other cool stuff. So I'm gonna, I think, now pass it over to Danny and let him kind of share a little bit about,

• 3:28Daniel Whitenack?

  you know, the background of Libra Chat, what it is and how he views kind of the need for private or open chat interfaces and what they're trying to accomplish with Libra Chat and how they see that fit into the industry and what's sort of needed. So over to you, Danny. Looking forward to this.

• 3:47Speaker 03

  Yeah, absolutely. You know, part of the original idea was kind of inspired by a ChatGPT leak. I don't know if you remember this, but there was someone whose messages were being seen by a different user.

• 4:02Daniel Whitenack?

  Yes.

• 4:02Speaker 03

  You know, he's from Poland or Russia and someone woke up one day and they all their messages were in Polish. That surprised me, but it really planted the seed for what I wanted to see. And I thought that was just a basic thing to overlook and just started crafting from that impetus. And yeah, I think it's inherently completely private with the flexibility of having like remote stuff in there too. Uh, which I think is important, but yeah, I honestly started off as a, a learning experience. It was chat GPT had just come out and rocked everyone's world. So I was like, wow, I really want to learn how this interface works. Like, what are they doing here?

• 4:44Speaker 03

  And I had just started learning about um, UIs in general and web development. So it really interested me and something I saw was, you know, this was such a huge tool in my learning. You know, as I learned this thing was being built out, but also, you know, there was that need right away because, you know, I posted it on GitHub and the next day I had six stars and I was blown away. I was like, what six stars?

• 5:11Daniel Whitenack?

  That's great. Who, who's looking at this?

• 5:14Speaker 03

  Yeah, it was already, you know, getting picked up by search algorithms or GitHub's algorithms. And I was just like totally blown away by that and totally motivated. You know, a lot of people started commenting right away what they want to see. And I think having access to these tools sooner rather than later is going to be a huge thing. No matter, you know, what your team size is, what your business is. And obviously, you know, you have to tread carefully with the privacy side. But I, I think I've built something battle tested at this point that um, you know, thankfully it's as much as a contribution from, if not more so a contribution from people using it would really help me along the way. So.

• 5:57Daniel Whitenack?

  You mentioned kind of like the initial problem that you saw that kind of motivated you to go down this rabbit hole, which was kind of seeing others' messages, which is definitely a, a piece of it in terms of how an application like this manages state and data and that sort of thing. What did you find like going down that rabbit hole? Is there a way that you can like categorize the? The main categories of things that people have come to find useful about having their own chat interface, rather than kind of a one provided by a model, data and privacy.

• 6:40Daniel Whitenack?

  But what are what are the kind of those main features or things that are on people's mind when they're thinking about having their own interface?

• 6:48Speaker 03

  I think for me too, it's just owning your own data and data is like the new commodity. It's so valuable even to these big AI companies. They're constantly releasing their own interfaces that are cutting edge, I might add. But they're also looking to collect data. And I think a trend I want to see in tech and especially from the open source world is just owning your own data that stays between you and these large language models and your company. And you really have that luxury through this app. And so that's a big driver for me, and I think that's a big component for a lot of people.

• 7:29Speaker 03

  And yeah, and it's also as you're learning from these things, it's like, I think it's so valuable to kind of like categorize and piece through like, you know, certain conversations you've had in the past. And that's why like one of the main features that's been like a mainstay since the very beginning was being able to search your messages. And to this day, it's not a feature on ChatGPT or or many different interfaces. So I think that's very interesting to see play out. But I also, I know a lot of people, just that one simple feature gets them on board.

• 8:03Chris Benson?

  I'm curious, as we talk about the applicability of this, you know, for for instance, large corporations, and I work for one of those large corporations which built its own interface some time back, and others will have well, but there is going forward, that takes a lot of maintenance, a lot of concern. If you were in front of like a chief digital and AI officer, a CDAO for a large corporation, and it may have already created its own, what would be your pitch for saying, you know, come over to Library Chat because of X? How would you convince the Fortune 500 companies that are out there that this is the way to go, rather than investing on their own compared to their whatever investment they've already made that sunk costs?

• 8:47Speaker 03

  Number one, it's completely open source. It's got a lot of contributions. There's nothing being hidden. In terms of its interoperabilities and and also number two, it's highly configurable with any kind of Intranet network you might want to do, or you know, it can be completely like sealed and even work with Large language models without needing to hit some kind of remote service. And it could be completely on local connections. It really just depends on on the admins level expertise and connecting all these things. But even just using the default Docker Compose, You can just spin up something that's only available to you.

• 9:29Speaker 03

  And it's configured in a way if you're using like insecure default variables and things like that, it'll warn you right away. So yeah, I think those are the top three things I'll say to try to convince someone.

• 9:42Daniel Whitenack?

  Well, I think people are eager to see a chat interface. We've been talking about them, so we'll kind of let you go from here and and show what you want to show. And I'm sure we'll have some questions and thoughts as you're showing these things. And yeah, if you could talk through kind of the the demo and and what's on your mind as you're thinking about the different features that that you're showing.

• 10:05Speaker 03

  All look good.

• 10:06Daniel Whitenack?

  All looks good.

• 10:07Speaker 03

  Great. So yeah, I'm running this locally and I'm using Ollama. I have it hosted on my computer, so I'll just write hi there and it should usually takes a second to load up.

• 10:20Daniel Whitenack?

  And Ollama, for those that aren't familiar, do you want to describe that just a second?

• 10:25Speaker 03

  I guess it's hard to find a specific term for what it does, but basically it helps you manage Local large language models, like it helps you pull down their latest build files. And then it helps with the prompt wrapping process and serves them on an API. So it just makes them really accessible wherever you can run them.

• 10:47Daniel Whitenack?

  Cool.

• 10:48Speaker 03

  So it finally got to me and now that it replied, you know, next couple of replies should be a little quicker. But basically, you know, this interface is, it should look pretty familiar to a lot of people. You know, uneventually taking a lot of inspiration from ChatGPT. There's just a couple of core things, like I mentioned, the search messages. And if I search here, it's already picking this up. It's a previous conversation I had. I was testing some file there. And aside from that, going back to this, this is kind of segmented, but for people who have like

• 11:28Speaker 03

  a need to set more custom parameters, or even just setting instructions here, making sure that it generates what they want to see. So I'll say, make sure to write code in Markdown.

• 11:44Speaker 03

  And I'll say, write me recursive Python function. Yeah, so it's doing, doing the job, whether or not it needed my instruction, it depends, but it had it. And so it kind of steered it right away to use Markdown, which gets rendered like this.

• 12:04Daniel Whitenack?

  It looks beautiful. And it even has like the copy, copy code and the nice sort of like edit button, copy, all of that stuff that one would expect.

• 12:16Speaker 03

  And really, it's pretty simple too. And I, I like that simplicity. I think I've seen a lot of interfaces kind of get lost with the, the technical side of it. And I'm sure that has an audience. And those are great interfaces for, for certain technical things, but something about this is, is just immediately accessible, I think. Of course, we mentioned that we could switch AI providers and I've got, I heard someone recently call this, you know, got to catch them all Pokemon of AI. But these are just a little showcase of like all the different ones we can use. So I personally like Grok, just because of its speed and it's, it's blazing

• 13:00Speaker 03

  fast. This is running Lama 370B.

• 13:04Daniel Whitenack?

  And this was like also a switch. So in the interface that you're showing, you're having a conversation, it's also was a switch between Ollama and Grok in the same thread. Am I understanding that right?

• 13:18Speaker 03

  Yeah, correct.

• 13:20Daniel Whitenack?

  That's awesome. Yeah. So you can kind of, and does that message thread sort of history carry through to the different models, I guess?

• 13:29Speaker 03

  Yeah. So that's where the database comes in, just keeping track of the conversation, not just the, the back and forth, but also any changes you might make. So if I make changes here on the fly. That'll get recorded with the conversation state.

• 13:47Chris Benson?

  Yeah, I love how you can stay in context of the problem that you're trying to solve. And yet still optimize against different models in terms of what they're better and worse on the fly without it kind of taking over. And becoming the primary concern. So very nice.

• 14:03Speaker 03

  Thanks. Yeah. And I, that brings up something really good user feedback I got maybe along the lines. There could be like a smart router that kind of knows, or even you could pre configure beforehand. Like which is the best AI for this sort of task and just kind of switches it for you. So maybe that's something down the line. I'm still kind of drafting in my head. But of course, like as these things evolve, there's like we mentioned rag. So a lot of people have the expectation for for files to work with these things. And of course, this Librechat supports that. And here I just dropped a CSV.

• 14:47Speaker 03

  I'll say, tell me about these sales. Yeah. And this was just mock data I made up. So it's about Gadget B, Gadget B, different sales data. So I was able to look at that and just kind of give give some context about it. And of course, I could switch the model just as before. So I switched to Co here and didn't give as good of a response as GPT four, but it was able to see that and kind of work with it. And even that, like the file processing, that's all based off a local rag solution. So it's using like a local data vector database and a local server that's just dedicated to the files. And yeah, that's one of the things there.

• 15:31Speaker 03

  One of the things I'm particularly excited about is agents and agent workflows. Of course, we have OpenAI recently made a solution of their own with assistance. And I think people are still discovering the capabilities of this, but I think it's exciting for me, like not just. Working with like what AI companies have as cutting edge, but also as inspiration for the open source side. Because I think they modeled this really well and it's giving me ideas of, you know, how can we do this for with Ollama? How can we structure something like this with the latest meta AI models? I have a test prompt here and this is how I generated my sales from before,

• 16:16Speaker 03

  but I'll add something here. Finally, output the file you created. So this can pretty much not just write code, but execute it within OpenAI Sandbox. So that's really great for things like that analysis and just generating mock data like this too. Yeah, so it might take some time.

• 16:43Daniel Whitenack?

  While that's running, there's a question. Did you test your local RAG system against others like OpenAI? So maybe some interest in kind of the performance of that RAG system with a variety of models and locally versus kind of those built into closed systems?

• 17:01Speaker 03

  I did. I definitely with OpenAI especially because they have a RAG system through assistance. And to be honest, it's right now, it's not doing nothing too special. It's what they call naive RAG. But I found even with naive RAG, you have a really good prompt. And so I tested that prompt several different iterations of it. You can really get something effective almost across the board like any LLM. And with OpenAI solution, it's really a black box. You can't even see the prompt that's being generated. I'm not sure how to like steer it better.

• 17:37Daniel Whitenack?

  Yeah, transparency is an issue in terms of, well, I guess it's nice when things go right.

• 17:44Speaker 03

  Mm hmm.

• 17:45Daniel Whitenack?

  And it's sort of auto magical, but then when things aren't going right, you really wish you could understand a bit more.

• 17:51Speaker 03

  Totally.

• 17:52Chris Benson?

  So where are you in in terms of your multimodal chat story? How far along are you in terms of what you're trying to get to?

• 18:01Speaker 03

  One of my main goals right now is to offer even more access controls and configuration over the interface experience like admins want to create.

• 18:15Speaker 03

  So for example, I understand that, you know, especially it's first time someone logs in here. They might not know, oh, what are all these models? I mean, I recognize Google, I guess, or they would need a little more clues. Or they might not even think to click here for the model. But I really want to see an update. It's I'm actively working on this where there's just one dropdown. And you kind of get a bit of more info on that, what you're selecting, what it's good at. And it's like, okay, this guy can search the internet and I need that for this task. And also just being able to control like which users can access what too.

• 18:59Speaker 03

  Because that's a pretty big need, especially in the enterprise setting. So in terms of multimodality, like in the sense of AI being able to work with different formats. I think down the pipeline, we'll see integrations with videos. But right now we're handling vision with images. And that's pretty useful and a huge help for me.

• 19:23Daniel Whitenack?

  So we started exploring Libra Chat at Prediction Guard because a bunch of our customers who are using Prediction Guard wanted a private chat interface. Because Prediction Guard itself is a platform that allows you to run large language models in a private, secure environment with safeguards around them for like factuality and toxicity and prompt injections and a bunch of other things. And so our customers are all those kind of privacy focused, security conscious customers who are maybe running Prediction Guard either on their own infrastructure and want a private chat interface for the models that they're hosting with Prediction Guard.

• 20:05Daniel Whitenack?

  Or they want an interface that's not a closed one for usage of our models. And so here, what you can see is we've taken Libra Chat, which again Danny mentioned is open source. And we've been able to take it into our kind of branding and we have Prediction Guard here. Where you can, you know, set your API key and use Prediction Guard running on top of our platform. And because it's open source, because it's transparent, we were able to take this and also, um, you know, integrate our own sort of flair into this. So I know, uh, engineer from our team, Ed and Danny worked together.

• 20:47Daniel Whitenack?

  So thanks for that, where we were able to integrate some of these checks for like toxicity and integrate our various models into the mix. So it's still kind of like Danny was showing in terms of running. Here I'm running with Neural Chat 7B. This is, uh, running in a privacy conserving setup in Intel's AI Cloud on Gaudi 2 infrastructure. So it's a very unique setup that we've kind of optimized. And we're able to connect to our own model, use this really slick interface, which is LibreChat. It's just sort of branded a bit with our colors and logos and that sort of thing. But also we can integrate the unique features of our take on, on an AI system, right?

• 21:31Daniel Whitenack?

  So let's say I'm really concerned because I'm using an open model that doesn't have some of the guardrails around it, like closed source models. Um, I can go into the config here and turn on a toxicity filter to make sure that the model isn't cursing me out or giving me any sort of like stuff that I don't want to see, right? And, and so here you can see we've have a little toxicity score. Thankfully it wasn't very toxic in, in this time around. So, uh, continuing similar to what Danny was showing. But again, our kind of own, own take on that with our models and kind of the safeguards around that.

• 22:13Daniel Whitenack?

  One cool thing that we found really useful is that a lot of our customers, they want an interface like this, but they also want it authenticated. Um, so they have their system set up. So we've integrated, we're a G Suite company. So we've integrated Google login here and it's only our org that can log.

• 22:35Daniel Whitenack?

  Again, so the prediction guard org and now I'm authenticated. Here's my chat like Danny mentioned that is private and searchable. So yeah, this has been a really amazing thing for us where we've been able to take and build on the great open source stuff that Danny has built at LibreChat and create something that works really well for our customers and for our setup. So before I leave and stop screen sharing, I saw that there was a question earlier on about translation with language models. A lot of what we've been showing is English, right? Some language models like OpenAI, they say that they'll do other languages,

• 23:18Daniel Whitenack?

  but sometimes that that doesn't always work out. So we have a translate endpoint in our API. And so we've done a bit of this testing with large language models, translation, and kind of standard translation systems like Google Translate and Bing Translate and others, or even other models like NLLB, No Language Left Behind from Meta. And in our translate endpoint, you can send a translation and then actually get the get the result along with a score. So we're using Comet scoring, which is a way to score translations. And I think the question was how well do large language models

• 24:00Daniel Whitenack?

  translate and are able to chat in in different languages versus machine translating with a commercial translation system? So what we've seen in scoring both commercial translation systems and large language models is that some large language models, depending on the language, like if you're going into Hindi with OpenAI, you might get a good translation or one that is comparable to Google Translate a small amount of the time, like five to 10%. But mostly the commercial translation systems are generally better. And definitely as you go down the longer tail of languages, it get sort of worse and worse. Even in even in chat in like Mandarin, a lot of models

• 24:44Daniel Whitenack?

  don't do so good, even though that's kind of the next highest represented language in in data sets out there. So yeah, it's definitely a mixed bag there. I don't know if Danny or Chris, if you have a comment on that, but before we go to other questions.

• 24:57Chris Benson?

  But I'm good.

• 24:59Daniel Whitenack?

  So some other questions on the LibraChat side. Are you building tools for LLM evaluation since you have all the comparison models out there? I think they're kind of imagining, oh, I can switch between models easily in this interface. How does that help me in an interactive way? It could help me evaluate the performance of different models. But there's probably a non-interactive, you know, version of that, I guess.

• 25:26Chris Benson?

  Maybe toward your roadmap, you were describing later, the idea of automatically switching to optimize. So this would be an incremental step in that direction.

• 25:35Speaker 03

  Yeah, absolutely. I back to the data ownership. I just think it's absolutely crucial to have built some kind of pipeline for evaluation as well, especially if you're really into fine tuning your own models. And really, it's crazy to think about, but even the data that, you know, we have just casually with these large language models. And if they're a very capable model, it's almost like a goldmine for the next model. And just having that, you know, at your own ownership, not just some cloud service. I definitely want to start it off simple, you know, being able to, you know, thumbs up, thumbs down. But then being able to integrate like complex evaluation tools like you guys already have

• 26:18Speaker 03

  with the toxicity score and or the translation rating, I think that's awesome.

• 26:24Daniel Whitenack?

  Cool. Could you talk a little bit? This isn't one of the questions, but I saw in your documentation discussion of like plugins. Could you talk about that a little bit, like what that means in the context of LibraChat?

• 26:35Speaker 03

  So they are inspired by just ChatGPT's use of plugins and really what the AI services now refer to as tools or functions. And really, it's just a way to be able to interact with some algorithm or API that's kind of programmed there already. And you're just kind of letting the model decide the inputs and outputs or the inputs and rather interpret the outputs. And I'm using it, obviously, in the plugin system where you can make requests to Dolly or Stable Diffusion for image generations. You can search archived papers, things like that.

• 27:18Speaker 03

  But really, what I came up with, the plugin system specifically is, is it almost a year old now, which is crazy. But I actually developed it before OpenAI had these functions in their API. So in the process, I learned kind of deeply how these LLMs were, you know, understanding certain tokens a little better for, for like formatting. And now we have like such a rich environment now for like getting, you know, only JSON responses or being able to use tools with Anthropic. So I've got a lot of things planned there where I want to see just that tool environment really grow. And also like for people who are building on top of LibreChat,

• 28:02Speaker 03

  I want to see better documentation and better developer experience in like adding those extra tools where, you know, this is a tool only my company can see and just being able to like plop it in real quick.

• 28:15Chris Benson?

  I'm looking at your GitHub and noticed 117 contributors listed there. Is your community built up around this has evolved going from, you know, sole developer in the beginning and now you have a group of people that are actively contributing at some level. How has that changed the project and changed how you're spending your time to fulfill the expectations of so many people and all the folks that they're serving in turn?

• 28:44Speaker 03

  Yeah, it's been amazing. I've learned so much in the process. I think I need to be conservative with my estimates on getting things done. So I can address contributions and things like that. And, and that is definitely a thing I want to keep, if anything, even more, more time. Because some people are making really great things, you know, I'll even shout out Marco, who is constantly contributing things and there's things that he just gets through so much quicker than I do that I don't quite find time to review, but it's sitting there and it's great. It's already working and I want to dig in the weeds a little bit. But also I think that's really what's helped the project explode to that there's such an openness

• 29:26Speaker 03

  to what people want to see in it, what, you know, I just had someone today. Say that they were, it was their first open source contribution and I just thought that was, you know, that's really cool to see just kind of people learning in the process and, and I was there too. I was always kind of daunted of like contributing to anything. So just seeing people, you know, step in the water. I definitely want to foster that more.

• 29:52Chris Benson?

  I'm curious, as a kind of a follow up to that, has there been a point where you've seen adoption occurring? And as part of that adoption, you know, not, not that you have favorite children, so to speak. And I understand, you know, that you're super happy for, for every organization out there. But has there been a moment where you've seen some organization that, you know, you might be super familiar with or something, adopt or know about, you know, and kind of went, holy mackerel, I can't believe that they're using my stuff. Has there been a moment like that for you?

• 30:25Speaker 03

  Oh yeah, for sure. I caught wind of Mistral using the app just to prototype their chat interface. That's the only one I know for sure. But I've also, you know, there've been people within Microsoft who are kind of just helping people prototype, you know, their own interfaces and things like that. And it's just like, that to me is, you know, a step back and I'm just kind of blown away.

• 30:50Chris Benson?

  The big boys of the space, definitely.

• 30:52Speaker 03

  Yeah.

• 30:53Daniel Whitenack?

  In our system, in the way that we've kind of customized Libra Chat here, we use this model-based factuality score, which is actually factual consistency between reference text and text out of an LLM. So you can do a factuality check between two different pieces of text to get a score that would show kind of factual consistency between the two, which that's kind of the most relevant thing for most LLM use cases because many people are using RAG or they have internal company data that represents a source of truth. So in Libra Chat here, we've, we're working on the integration with the RAG piece, which would be a cool integration there. But for now, we just have this sort of factuality context.

• 31:38Daniel Whitenack?

  So facts that shouldn't be violated. Right? And I could put something here and turn on the factuality check and then ask a question. So the fact I put in was that the sky was green. I could ask, you know, what, what color is the sky? And then I think neural chat will actually respond factually, but I'll do the check against the gold standard information that I put in, which is actually that the sky is green. And you can see that I get out a factuality score which ranges from zero to one. And in this case, it's very low because I put in that information about the sky being, being green.

• 32:17Daniel Whitenack?

  So yeah, that's the sort of interesting way, you know, I'm so thankful for this project being open source and being customizable because this is the kind of cool stuff that people are enabling within their own chat interfaces that we're working with. And it's awesome to have a robust system that works well in that way. Looks like the, there's another question. Danny, how feasible? And something that you would venture yourself into is combining LibreChat with such frameworks as Flowwise or Cray?

• 32:53Daniel Whitenack?

  I don't know how to say that. I don't know what that is.

• 32:56Speaker 03

  I think it's Crew AI.

• 32:57Daniel Whitenack?

  Crew AI. There we go. Yeah. I don't know if you know of those things, but.

• 33:02Speaker 03

  Yeah, I'm familiar with both. I think Flowwise is really great. Giving that, that string based logic, user interface logic that, you know, no programming, you kind of put all the pieces together. I see that being integrated much sooner in Crew AI, which is more of like an agent orchestration framework. But Flowwise, I think it could serve as kind of like another backend, you know, like, just like you see another so many endpoints, as I like to call them, which are Mistral, Google, OpenAI, so forth. I could see it being easily integrated like that where I don't really want to reinvent

• 33:42Speaker 03

  the wheel with something like that because they've done such a great job and I just want to be able to handle the integrations. But, you know, because obviously it's not going to be everyone's need. And for Creo AI, you know, I definitely have a lot of ideas there. I'm trying to establish kind of like a framework for agents first and then potentially, you know, get into agent orchestration where agents are talking to each other and things like this. But we're not quite there yet. We've got the OpenAI side shaping up, but we want to see some open source integrations there.

• 34:21Daniel Whitenack?

  Awesome. Well, I think this is a good question to maybe kind of draw near to a close here. It's something we, something asked in the webinar chat, but also something we usually kind of ask people that we're talking to on the podcast is, you know, you're following and kind of plugged into all of these different things that are happening in the AI ecosystem and, you know, things even I'm learning about today that even though we're, I think, you know, we would be plugged into many things that we hear about this, but there's just so much going on. After kind of looking at that landscape and, you know, how innovation is happening, how people are using your interface, but also more widely the things

• 35:05Daniel Whitenack?

  that you're seeing people do in the open source space or otherwise, where do you see all this kind of going? And how do you see the future of both Libra Chat and maybe even just things that you're, is there something you're particularly interested in, in the AI space to see how it develops in the coming year?

• 35:25Speaker 03

  Yeah, I think, kind of been hinting at this already, but just, I think the future, it's the future I want to see. And I feel like a lot of people in tech want to see it too. And it's the open source future where these large language models are getting so good every day. There's a lot more time and money invested in being able to like host these things just from a consumer grade computer. And I think catering to that is probably going to be the direction of my project and many similar projects because it even blows my mind that I can use something like Llama 3 where a year ago I might have thought, oh, this is two years away. And I really think that's the direction, both on the high level and low level.

• 36:07Speaker 03

  And I think it's part of the reason it's the project's really taken off just because these things are so accessible and we don't have to pay SaaS subscription money to, you know, just to use a message for AI. So.

• 36:23Daniel Whitenack?

  Yeah, that's awesome. I think that's a future that at least Chris and I are looking forward to, and I'm sure many on the webinar.

• 36:32Chris Benson?

  You mean, I get my wallet back at some point?

• 36:36Daniel Whitenack?

  Well, I'm sure there'll be other things to pay for, but, you know, new AI PC or something to run all your local models. But.

• 36:44Chris Benson?

  That's right.

• 36:45Daniel Whitenack?

  Cool. Well, thank you so much, Danny, for for taking time. Thank you to everyone that joined the webinar. This was, this was a ton of fun. We're going to be doing another one of these webinars very soon. The next one, I think, will be around multimodal AI and some practical, like, hands-on instruction in how to create things like multimodal RAG systems or kind of search over images and videos. And so that's going to be a ton of fun. So keep on the watch for that. That's going to be a fun one. Until then, yeah, looking forward to seeing you next time, Chris, on the on the podcast.

• 37:24Chris Benson?

  Absolutely. Thank you, Danny. Thanks to everyone who joined us today.

• 37:28Speaker 03

  Yeah, you guys are awesome. Thanks for having me.

• 37:37Speaker 00

  All right, that is Practical AI for this week. Subscribe now if you haven't already. Head to practicalai.fm for all the ways and join our free Slack team where you can hang out with Daniel, Chris, and the entire changelog community. Sign up today at practicalai.fm/community. Thanks again to our partners at fly.io, to our beat freaking residents, Breakmaster Cylinder, and to you for listening. We appreciate you spending time with us. That's all for now. We'll talk to you again next time.