The practicalities of releasing models

Recently Chris and Daniel briefly discussed the Open RAIL-M licensing and model releases on Hugging Face. In this episode, Daniel follows up on this topic based on some recent practical experience. Also included is a discussion about graph neural networks, message passing, and tweaking synthesized voices! Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Daniel’s team license from recent work https://huggingface.co/spaces/sil-ai/model-licenseGraph Neural Network courses from Zak Jost https://www.graphneuralnets.com/Coqui voice studio https://coqui.ai/Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Daniel Whitenack?

  So there was this idea that the data that we were using, in this case, it was from the Bloom Library, which is a product from S I L where people can create their own books online. So each book, the author releases that under a certain creative, well, not all are Creative Commons, but the majority are Creative Commons licenses. And so we had to look into whether the models that we were creating off of Creative Commons data would be subject to the same sort of restrictions as the Creative Commons data that we were training it on. And so there's various writings, you know, within you can actually look up,

• 0:45Daniel Whitenack?

  Creative Commons has some commentary on this of when certain things are derivative works or adaptations and that sort of thing.

• 1:03Speaker 01

  Welcome to Practical AI, a weekly podcast making artificial intelligence practical, productive, and accessible to everyone. Subscribe now if you haven't already. Head to practicalai.fm for all the ways. Special thanks to our partners at Fastly for delivering our shows super fast to wherever you listen. Check them out at fastly.com and to our friends at fly.io. We deploy our app servers close to our users and you can too. Learn more at fly.io.

• 1:39Daniel Whitenack?

  Well, welcome to another fully connected episode of Practical AI. This is where Chris and I keep you fully connected with everything that's happening in the AI community. We'll take some time to discuss the latest AI news and dig into some learning resources to help you level up your machine learning game. I'm Daniel Whitek. I'm a data scientist with SIL International and I'm joined as always by my co-host, Chris Benson, who's a tech strategist at Lockheed Martin. How are you doing, Chris?

• 2:08Chris Benson?

  Doing great today, Daniel. How are you?

• 2:10Daniel Whitenack?

  Doing well. It's been a, it's been a pretty fun week. Last night, got to speak virtually, anyway, at the Utah ML Ops meetup. So that was pretty fun. Had a few connections there and a lot of good questions and such came out of that. So it's interesting to see meetups happening again, but also still embracing this like bringing in virtual speakers thing. So it's like a still hybrid in that sense, because if you're a local meetup, you've got to bring in speakers and it's a lot easier to bring in virtual speakers. And I think, I think it worked out pretty good, the way that they had it set up. So.

• 2:48Chris Benson?

  That sounds interesting. And now, since you mention it, I'm speaking at a virtual conference next week. I've been taking, like through COVID, I had taken more or less a long break after doing way too much conference talking.

• 3:01Daniel Whitenack?

  Right.

• 3:01Chris Benson?

  In the year, in the years leading up to it. It's a national security conference and I'm going in to talk about AI, data and software in the context of national security, intelligence and defense. So that'll be.

• 3:13Daniel Whitenack?

  Wow, exciting.

• 3:14Chris Benson?

  Yeah, yeah, I'm looking forward to it. It'll be the, the day after this episode is released.

• 3:19Daniel Whitenack?

  Yeah.

• 3:19Chris Benson?

  So I will put a link in the show notes in case anybody wants to hop in. I believe it's free to attend. So if anybody wants to do that and we'll see what happens there.

• 3:29Daniel Whitenack?

  Yeah, we're also kind of gearing up quite a bit because in December is EMNLP, which is the sort of biggest natural language processing research conference, one of the main ones, but I think kind of considered the main one. And that's in December in Abu Dhabi. And I'm going to travel there with a couple colleagues. So that'll be one of the kind of bigger excursions I've taken after COVID time. Excited to be there at EMNLP. And we had a paper accepted, so excited to present that there and also hear from, from the rest of the community. Actually, it's interesting because some of what we've released with what

• 4:14Daniel Whitenack?

  we did release and what we are releasing with this paper at EMNLP. We had to think a little bit about licensing around those things. And I don't know if you remember, I forget which episode it was recently, we were talking about these open rail licenses for

• 4:32Chris Benson?

  Absolutely.

• 4:33Daniel Whitenack?

  So the idea that, okay, for data, you have, you know, maybe like Creative Commons or licenses like that. For software, you might have Apache or MIT or GPL or whatever it is. But models fit into this weird space where they aren't quite either of those things. So how do you license them? And people at various groups have tried various things, but there's an effort called Rail Responsible AI Licenses. And I think if you just go to licenses.ai, you can learn more about what they're doing. But we made an attempt at some of our benchmark models that go along with our paper.

• 5:15Daniel Whitenack?

  We made an attempt at creating one of these rail licenses for the release of those. And that was quite an interesting experience. I don't know if you remember kind of the tenants of what goes into a rail license at all. Do you remember that?

• 5:32Chris Benson?

  I remember, I say this halfway tongue in cheek. I remember thinking I'm in the wrong industry for this because I may need to blow things up with inference.

• 5:40Daniel Whitenack?

  Oh, yes.

• 5:41Chris Benson?

  If I recall, they don't want me to blow things up. So that's stuck in my head.

• 5:45Daniel Whitenack?

  Yeah, yeah. So there's really this, I think the biggest thing that people can keep in their mind with these responsible AI licenses is that there's usage restrictions. So Chris, just because there's a rail license doesn't mean that you couldn't blow things up or, you know, run a drone or something with your model. But it would mean that if they put that in the restricted use. Clauses of the license, right? So how this works is there's kind of a main bit of the license and it says various things about, you know, copyright and, and patent and warranty and all these things that you might see in a normal license. But all of those things are subject to the terms of the license and the terms of the

• 6:30Daniel Whitenack?

  license have, are subject to this clause at the end of restricted usage. So for example, like the recent release of stable diffusion, uh, used a, a rail license, an open rail license. And they put in restrictions around like, don't use this for misinformation or to harm others or, you know, various things like that. So they recognize there's dangers with these models and they want to put some barriers around that. So that's kind of the idea with these rail licenses is that it's a, a way to distribute and make your model available with certain guardrails around usage. I don't know. What, what are your general thoughts on, on that?

• 7:15Chris Benson?

  No, I mean, I think it's, I think we, we need that and that's part of the maturing of this industry. And, you know, we, we had that for a long time with those other types of intellectual property, uh, in terms of things like software and such. And so I remember as we did that episode not too long ago, it felt like a good time for that to happen at this point because it was one of those questions people have having. And by the way, I should just in the, for the sake of being responsible, let people know that I'm not blowing things up, nor is my employer asking me to blow things up. They don't do that. They don't ask me to blow things up. So I, I-

• 7:48Daniel Whitenack?

  Sometimes we use sarcasm on this show.

• 7:50Chris Benson?

  We use sarcasm on the show from time to time, but I thought maybe I should just specify that I'm-

• 7:56Daniel Whitenack?

  That's a good disclaimer.

• 7:57Chris Benson?

  Yeah. I'm not, I'm just claiming blowing anything up at all. So there we go.

• 8:00Daniel Whitenack?

  Yeah. I think though, like in our license, we kind of balanced a couple of things around the restricted use that probably would restrict your usage of our models in certain cases, Chris, not because of like military usage necessarily, but commercial usage. So one of the things that we- We wanted to do internally because of how we had sourced our data and the fact that this data actually came from local language communities. So it's actually, we trained our models on books that were written by local language community members and they released those books under certain rights, some of those being non-commercial licenses.

• 8:42Daniel Whitenack?

  And so we wanted to make sure that we both honored that, but we also released the models. Cause what can happen sometimes with language data is like big companies could use language data from language communities to make money without any real benefit going back to those language communities, right? And so that was partly also in our mind with this license. And so we put in our restricted use, two things. One, a restriction around commercial usage. This would be like in our thinking around, you know, what benefit goes back to the language community. But then secondly, putting in there a restricted use around uses that are particularly discriminatory against indigenous peoples.

• 9:27Daniel Whitenack?

  So you could use indigenous language data to discriminate against indigenous peoples, right?

• 9:34Chris Benson?

  Yeah.

• 9:35Daniel Whitenack?

  And there's actually a nice clause in the UN Statement on Indigenous People, where they talk about discrimination against indigenous people. And so we kind of pulled a reference to that into our restricted use. So I don't know, it's our try at this. It was an interesting exercise to actually try to put this into practice and figure out like, okay, we talked about this on the podcast, but can I actually create one of these licenses for my own models? It was an interesting exercise.

• 10:05Chris Benson?

  So, so I really liked the fact that you were thoughtful enough, for instance, on the question of discriminatory practices against indigenous folks to, to think about that and make sure that was in. But another thing that was, that I was wondering as you were discussing that was, you know, kind of going back to the start of this particular topic a moment or two ago on the show. You know, there's software, there's data, there's the model. There's all of these intellectual property overlaps between different types of IP, and they're relying independent. Did you have any question in your mind as you went through the process about whether a license from one type of thing, such as software,

• 10:47Chris Benson?

  could clash with the model license? And did you have to think about that and kind of resolve that a little bit?

• 10:53Daniel Whitenack?

  Yes. So there was this idea that the data that we were using, in this case, it was from the Bloom library, which is a product from SIL where people can create their own books online. So each book, the author releases that under a certain creative, well, not all are Creative Commons, but the majority are Creative Commons licenses.

• 11:19Chris Benson?

  Sure.

• 11:19Daniel Whitenack?

  And so we had to look into whether the models that we were creating off of Creative Commons data would be subject to the same sort of restrictions as the Creative Commons data that we were training it on. And so there's various writings within, you can actually look up, Creative Commons has some commentary on this of when certain things are derivative works or adaptations and that sort of thing. In our case, the models that we trained off of this data, whether it's surprising or not, according to how we read those, were not derivative works and so wouldn't be restricted to the same

• 12:00Daniel Whitenack?

  sort of license. However, what we tried to do was, we tried to match the kind of restrictions of the original data, just in good faith to

• 12:12Chris Benson?

  Sure.

• 12:12Daniel Whitenack?

  how people might have expected that data to be used. But I think technically, we had more latitude there.

• 12:20Chris Benson?

  No, that, that sounds good. And I'm not surprised that you and your, your folks were, were doing that. I would hope that everybody out there in the larger community would be thoughtful in that way about it. It's interesting, as we, we've talked so many times about, about kind of having these different, you know, these different constructs, you know, blending, you know, having software blending with the data, blending with the models now and getting them out. But we just haven't spent a lot of time talking about kind of the, the legalities of how to do that and how to honor those across the format. So, good to hear.

• 13:04Daniel Whitenack?

  So Chris, I don't know if you remember not that long ago in one of our recent episodes, which we can link in the show notes, we had Josh from Koki on.

• 13:15Chris Benson?

  Absolutely.

• 13:15Daniel Whitenack?

  And we even made some clones of our voices and that sort of thing. That was a lot of fun. So Koki's doing amazing things in sort of open source speech technology and really enabling a lot. Actually, we're using a lot of their libraries in our own work. But they had an announcement that I, you know, I thought I'd share in terms of the news side of things, which is you can now join their waitlist and get access to what they're calling their like Voice Studio Audio Manager Advanced Editor features within their system. Sorry if I'm not getting the names right, but it's pretty cool. This is, I don't know if you remember when he was on the episode, but he talked

• 13:58Daniel Whitenack?

  a little bit about how they were thinking about managing the sort of tone, emotions, expressions of synthesized voices more flexibly. So you don't just get sort of one synthesized voice and it's kind of either monotone or having the same expression throughout. You can actually match different portions of your content with different kind of expressive qualities.

• 14:24Chris Benson?

  I remember him talking about it.

• 14:26Daniel Whitenack?

  Yeah, this Voice Studio does. And there's some pretty cool things where you can actually look at different words and the different phonemes in those words and adjust some of these expressive features, emotion and pitch and mixing every, mixing different voices together as well, like to create a mix of synthesized voices. You can do all this within this kind of advanced editor, which seems, seems really possible or really powerful is what the word I was looking for.

• 14:55Chris Benson?

  Yeah, I'm really looking forward to using it, but I'm a little dismayed that I'm currently in number 6,466 in line to receive it. So it may be a little while before I receive the joys.

• 15:09Daniel Whitenack?

  Well, shout out to Josh. If you're, if you're out there listening, you can bump Chris up the waitlist.

• 15:17Daniel Whitenack?

  But yeah, I think it's really interesting where, you know, it's one thing to like produce a synthesized voice. It's another thing to kind of have multiple voices, maybe in a video that you're mixing down and mix voices together, change the expressive qualities, like the sort of almost like working with synthesized voices like people do with like computer production of music, right?

• 15:44Chris Benson?

  Right.

• 15:44Daniel Whitenack?

  Where you can change things and mix things together and all of that very fluidly.

• 15:49Chris Benson?

  I want to ask you a question that's very specific to the work that you're doing on a day to day basis. And for working with indigenous populations and their languages and stuff, what are some of the ways that you think that this will change that going forward or add, you know, add to it that you guys have been talking about? Like, what's the future look like for someone in your line of work on that? I'm, I'm just curious about that kind of a real world aspect.

• 16:13Daniel Whitenack?

  Well, yeah, I think that there's, there's definitely the side of this, which is probably the more commercial side of it, which is, you know, media production and that sort of thing where like, let's say that you produce a video in one language and, you know, you're wanting to do the dubbing across languages or something like that, or maybe even you're using like an avatar and using synthesized voices in your video and the whole thing is synthesized. I mean, that's happening quite a bit right now as well. And so the ability to bring in multiple voices and do all that without going into a recording studio, of course, that has huge applications for like advertising, marketing, media production,

• 16:58Daniel Whitenack?

  entertainment, all of those different areas, which is where I would guess, and I can't speak to Koki's business model, but I would guess that their, their tooling is quite applicable across those areas. For local language communities, I think, of course, they're also involved oftentimes in the production of media or content for their communities. So that, that's also relevant there. But I think there's also unique things that are relevant in those scenarios. So. Imagine that you're part of an indigenous community, a local language community, and you are kind of marginalized by the the national government or discriminated against in one way or another.

• 17:40Daniel Whitenack?

  It might be a kind of big ask for you to, for your community to say, hey, could we put up 100 hours of content with your voice and maybe your likeness? Right? That's a potentially painting a target on yourself.

• 17:54Chris Benson?

  Right? Yeah.

• 17:55Daniel Whitenack?

  When you're associating yourself and you're the face of that that community. So I think it's really interesting that there's tools like this where you could create high quality voice and expressive voice. That's maybe synthesized and not someone's, maybe even style transferred, where it's not someone's voice that can be tracked to a certain person. But then if you think about then combining that with the video elements. So think about having a video that maybe is recorded with someone talking. You can use things like stable diffusion and other things now to actually shift that video and obfuscate the identity of the person in that.

• 18:38Daniel Whitenack?

  Now, the more nefarious use of that, of course, would be misinformation and

• 18:42Chris Benson?

  Of course.

• 18:42Daniel Whitenack?

  deep fakes and that sort of thing. But there is a very positive use of this for these sorts of communities where it is important that they want to produce media content for their community. But if you're marginalized or discriminated against, it's interesting now that there's these tools that are accessible to and have really nice user interfaces accessible to community members where they could actually produce some of that content themselves. Yeah. So it's it's interesting dynamic.

• 19:13Chris Benson?

  To your point, I'm just kind of thinking about in my world a little bit and thinking about the fact that, you know, as we're recording this in the current day and current months, the war in Ukraine, which the Russian invasion of Ukraine has been going on and Belarus is also a part of of the that Russian effort. And I was reading this morning, an article about some That are, you know, trying to a survive, you know, that situation and B escape and, and, and, you know, and try to help and do the things that they, that their conscience is, is dictating. And that I kind of going back to, to some of the, the ideas that you just enumerated about marginalized

• 19:55Chris Benson?

  populations of indigenous populations and, and being able to kind of find some protection while generating content. I could, I could imagine that in this as well. And so.

• 20:04Daniel Whitenack?

  I mean, you saw it sort of way in the past with online hackers and when they would like hack SeaWorld or whatever, they would release a video, we are anonymous or whatever. And, and it would all be synthesized voices, right? Cause you don't want someone, you don't want to put your voice on that, right?

• 20:21Chris Benson?

  That's right. And don't get me started on animal protection because it will stop being an AI podcast and we will just go off into a totally different realm. So, so be good to your animals, folks. That's our little sideline.

• 20:34Daniel Whitenack?

  Yeah.

• 20:35Chris Benson?

  Right here.

• 20:35Daniel Whitenack?

  Do you think, one question that was actually brought up to me last week, which I think is kind of an interesting question for people like us that do produce content. I mean, this is just our voices. We're recording our voices on this podcast, but let's just imagine that, you know, Koki or whatever, their voice studio, it's great enough that we can just, I mean, we already have a lot of sample of our voice, right? If we can create really nice voices, you and I could just type out a script back and forth. And, you know, when we're traveling, quote, record a podcast and just mix our voices together with content and, and release it. How does that sort of thing strike you? Cause I got into this conversation

• 21:20Daniel Whitenack?

  with someone last week and there was some sort of mixed feelings about you, you know, what you lose when you do that, or what you gain when you do that.

• 21:29Chris Benson?

  So that's a great point that you make there. And, and who knows, maybe we both have tough schedules. And for listeners who don't know, for Daniel and myself, this is a passion project. And so, you know, who knows, maybe there is a moment of tight schedule for us where we do exactly that. The thing we would lose is that there is the, there is the element of the unexpected in our conversations often. And, uh, and a lot of banter back and forth. It's not scripted, uh, completely unplanned. Um, I know people think that we plan every word out, but we don't. And so maybe that would be lost. So you might get the information you wanted to share out, but you may lose a little bit of the human element behind it.

• 22:10Daniel Whitenack?

  Yeah. The context for the conversation I had last week was it was someone who does produce video content specifically. And so their face is sort of part of their brand, right?

• 22:24Chris Benson?

  Yes.

• 22:24Daniel Whitenack?

  And so the question was, well, if we dub your video into another language, it would make sense. You know how? Everyone hates the thing about dubbed video where the lips don't match the voice, right? And you can kind of sync it up pretty good in a lot of cases, but ultimately the, you know, what you could do is just modify the person's face and lips.

• 22:44Chris Benson?

  Of course.

• 22:45Daniel Whitenack?

  Match the dubbed, you know, they recorded the video in English, but now we're dubbing it to Chinese and we match up their lips. Using some sort of video manipulation. They reacted very negatively to that because they're like, my face is, is like my brand, right? Like, I don't want anyone messing. They actually said they prefer the dubbed content because their original like expression of how they expressed themselves in the video was what was important to them. So yeah, it was interesting.

• 23:18Chris Benson?

  I'm going to challenge that. I'm going to suggest to you that in the not so distant future, not only will that exist, but when you're getting on, you know, we're all through COVID and certainly continuing post-COVID, we're all on video calls all the time. And so I'm going to suggest that that's going to be one of those killer features. That one of the video call providers is going to, to do, and that is not only doing translation in real time, which, uh, which I think is entirely possible in the, you know, not so distant future. But using some of these technologies we've been talking about in recent episodes to do exactly that. Because especially if you're using their service quite a lot, which some of us are,

• 24:02Chris Benson?

  then they also have a thorough data set to train on. And I think that with the video and everything, I think, I just think that's going to be very doable not too far down the road. And I think it'll also be able to be done live. So I think it will be part of what we do. I think you and I will find ourselves doing that before long is what I'm, what I'm suggesting.

• 25:00Daniel Whitenack?

  So Chris, one of the things that we had just kind of started talking about before we started recording was you were asking a few questions about graph neural networks, which I've thought have been interesting for quite some time. And I think you ran across it in some Nvidia posts or something like that, right?

• 25:19Chris Benson?

  So Nvidia has a blog that's widely read. They blogged as we record this. It was, it was actually just two days ago on the 24th of October, 2022. But they had a blog about kind of what are graph neural networks? And it occurred to me, I was kind of glancing through it and there's a fair amount of stuff that I'm familiar with in it. And there was a few items there that I, that I hadn't thought about. But it occurred to me that we have kind of touched on graph neural networks quite a number of times on the show without ever really diving into it. So, so maybe there is a fully connected episode where we do a full show to dive into the detail. But it made me start wondering a little bit about how many of our listeners out there

• 26:04Chris Benson?

  are using graph neural networks and some of the I saw something the other day about starting, and this is very typical for some of our conversations, where you're putting together multiple kind of deep learning approaches to try to get something new. And we've, we've had a lot of shows talking about that. But before I go on, have you had any opportunities to use graph neural networks yourself?

• 26:26Daniel Whitenack?

  I recently did train a graph neural network for a question, a question answering task. So one of the areas where people have applied this is to this task of automated question answering, where there's a text prompt and you're looking for the, the answer within some set of documents or something like that, right? So I did dive a little bit into that. And to be honest, I'd love, I'd love to learn more. That was definitely an interesting experiment as I was kind of diving into that and learning, you know, what does it mean to have a graph neural network? Well, there's certain approaches and for maybe graph neural network people out there that are experts,

• 27:09Daniel Whitenack?

  maybe I'm simplifying this too much, but there seems to be a, a cluster of techniques that are focused on representing graph structured data, a sort of flat form or a matrix or tensor form, right? And so there's ways to embed a graph or learn an embedding for a graph in a sort of flat form. There's also methods that exploit the structure of the graph neural network or the graph itself, which I think is what I think of when I think of graph neural networks. So one way to, I guess, think about it is if you think about a convolutional layer, right?

• 27:49Daniel Whitenack?

  You're running some kernel or filter over your image or your set of inputs. But what you're doing is you're always considering like one data point in the context of a fixed number of other data points, even if you're running your filter over in some various ways. It's sort of one data point in relation to a number of other fixed data points. And actually, you know, transformers or, or recurrent neural networks and this sort of thing also. Behave similarly, right? You're comparing one data point in reference to maybe a sequence of other things, but which have a fixed sort of structure. And what's interesting, I think, about graph neural networks is that the graph neural networks that I'm thinking,

• 28:33Daniel Whitenack?

  which are built around these concepts of message passing, consider one data point in reference to sort of arbitrary structure of other data points. And what happens in these graph neural networks is that you have a stage where you take an embedding for one node in your graph and you look at all the neighboring nodes, or maybe a certain number of neighboring nodes, but all the neighboring nodes that fit within a certain structure, and you combine or concatenate or perform a function over the combination of the embeddings for those nodes and the embeddings for the node that you're considering. And so what happens as you apply this across all of the nodes of your network,

• 29:19Daniel Whitenack?

  you actually pass a lot of information between all of the nodes of your network. And if you iterate that, then the idea is like all of this messaging and information is transferred from all of this different complicated graph structure to the data point under consideration. And so oftentimes this involves this sort of message passing and iterative approach, which is quite interesting and has been applied in a variety of ways. One of the ways, of course, that's maybe very well known is AlphaFold, which is one of the protein folding approaches.

• 29:56Chris Benson?

  We had a show about that not too long ago. So one of the things is you're looking, in your line of work, at large language models, and we're looking at graph neural networks and how they can merge. And you're talking about the flat structure. And I know Nvidia kind of talks about the unstructured nature of the message passing within the graph itself compared to other neural networks that are a lot more structured. Can you clarify for a moment, like, what does it change in terms of how you're approaching large language models when you have this flat, unstructured node approach, where you're doing the message passing and you have an arbitrary number of them that are there? Does it dramatically change the workflow for when you're working on those large language models? Or is it similar?

• 30:42Daniel Whitenack?

  Yes, so I mean, a lot of times the large language models take a very naive approach to how text is structured. So most language models now work around something called subwords, which means I have a piece of text, I'm gonna split that up into subcomponents, but not necessarily words. Like you could tokenize something into words, but the problem with that is like, how do you know how to tokenize what is a word and what isn't a word? You've got all these sort of weird structures in language. The other thing is, if you tokenize into words, what happens when you see an unknown word in your input?

• 31:25Daniel Whitenack?

  And so what often happens is you figure out what are the most frequently occurring subwords across my corpus of known language? And so maybe, you know, like your name, Chris, there's a subword, you know, C H R I, and then you tack on an S subword, and that forms your your name, right? And so if you figure out what are those frequently occurring subwords, that's how you split things apart, and that's how you sort of look at the attention across these different subwords in a in a large language model. But that that's somewhat, I mean, it's statistical in terms of how you get those subwords and,

• 32:08Daniel Whitenack?

  you know, it's useful, but with language in general, language is much more structured in many cases like a graph. And so to do this sort of large language model approach works well and it's scalable because you don't have to know as much about the structure of your input. But if you do know more about the structure of your input, you can do maybe really powerful things with less data, right? So for example, if you know all of the parts of speech of your language, right? Like, here's my noun subject, and it's connected with a verb via node in the graph, right, in this way, and you draw out all the tree structures.

• 32:52Daniel Whitenack?

  And the syntax of your language, there's tons of information encoded into that structure that you lose when you just treat words like a sequence of subwords, right? And I think there's arguments that these large language models do learn some of that structure. There's some work out of Stanford on that. But the way that language is structured, I think the hope would be that if you're creative about encoding this linguistic information into your model, and then maybe using something creative, like a graph neural network, maybe you can do more things with less data, or you can do really powerful things, or you can be more robust to sort of changes and that sort of thing.

• 33:34Chris Benson?

  I see. Good exclamation.

• 33:36Daniel Whitenack?

  And yeah, if if other people have have input on graph neural networks or have used them in certain ways, definitely let us know. I, one of the other interesting ones that I know, just doing kind of searching when I was looking into graph neural networks is from Pinterest, I guess, the, the system that does recommendation of items for users within Pinterest is their real time system for recommendation is built on some sort of graph neural network called Pixie, which, yeah, if any of you are on that team out there and you want to come on the show and talk to us about it, would love to hear more. But you can look up, they do have a paper about it and all of that.

• 34:16Chris Benson?

  Absolutely. I know that in the Nvidia article that we were talking about, they mentioned LinkedIn does the same. I would imagine that other social networks do as well, quite honestly, that it seems like a very logical fit in terms of trying to get that functionality.

• 34:31Daniel Whitenack?

  Yeah. Well, as we kind of get near the end here, Chris, I mean, we always try to share some sort of learning thing. And as I was learning about graph neural networks, I found several interesting resources, but one which if you're into sort of paid courses, this, this one seems to be quite full of great information about graph neural networks. So this is from Zach Jost. He actually has a YouTube channel called Welcome AI Overlords, but he's, he's really Into Graph Neural Networks. I think he works in, has worked in sort of variety of big tech places. And he has a full course of, you know, introduction to graph neural networks.

• 35:15Daniel Whitenack?

  If you just go to graphneuralnets.com, pretty simple link, then you can see as introduction to graph neural networks, foundational theory of graph neural networks, basics of graph neural networks, and the basics of graph neural networks is free. So at least you could get like the sense of the graph neural networks from the free version.

• 35:37Chris Benson?

  Well, I may very well be a student on that course and maybe some of the other stuff out there as well. I have a very specific doing graph database work that I'm working on in my day job without going into specifics. And I can totally see how one of the associated problems with that could be solved by graph neural networks. And so I think, I think it's time for me to go level up. And I encourage our listeners that maybe not already in that to go consider it as well.

• 36:05Daniel Whitenack?

  Sounds great, Chris. Well, it's been been fun as always. Good to connect our two nodes via the edge of practical AI as always. So.

• 36:17Chris Benson?

  Oh, boy. Oh, boy. That was, that was like the AI version of a dad joke right there.

• 36:22Daniel Whitenack?

  All right. Well, on that note, Chris, we'll see you before I make another joke.

• 36:27Chris Benson?

  Okay, no worries. Have a good one, Daniel. Thanks.

• 36:29Daniel Whitenack?

  Bye.

• 36:39Speaker 01

  All right, that is our show for this week. If you dig it, don't forget to subscribe. Head to practicalai.fm for all the ways. And if practical AI has benefited your life, pay it forward by sharing the show with a friend or a colleague. Word of mouth is the number one way people find shows like ours. Thanks again to Fastly for fronting our static assets, to fly.io for backing our dynamic requests, to Breakmaster Cylinder for the beats, and to you for listening. We appreciate you. That's all for now. We'll talk to you again on the next one.