Copilot lawsuits & Galactica "science"

There are some big AI-related controversies swirling, and it’s time we talk about them. A lawsuit has been filed against GitHub, Microsoft, and OpenAI related to Copilot code suggestions, and many people have been disturbed by the output of Meta AI’s Galactica model. Does Copilot violate open source licenses? Does Galactica output dangerous science-related content? In this episode, we dive into the controversies and risks, and we discuss the benefits of these technologies. Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Related to Copilot: Article - “GitHub Copilot Isn’t Worth the Risk” https://www.kolide.com/blog/github-copilot-isn-t-worth-the-riskTabnine https://www.tabnine.com/Big Code Project https://www.bigcode-project.org/Related to Galactica: Model website https://galactica.org/Article: “Galactica: the AI knowledge base that makes stuff up” https://www.aiweirdness.com/galactica/Books “Interpretable Machine Learning” by Christoph Molnar https://www.amazon.com/dp/0244768528“Modeling Mindsets” by Christoph Molnar https://www.amazon.com/dp/B0BMJH7M9FUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Chris Benson?

  We were kind of laughing about it and stuff, but I think we're going to see so many of these instances in the years ahead. And you made a point that I think we sometimes need to respond with a bit of empathy for the data scientists and AI engineers that are trying to create these because they're trying to do some pretty cutting edge stuff and mistakes are going to be made. And in the end, my understanding is nobody was hurt by this.

• 0:22Daniel Whitenack?

  Yeah, we need to both be critical and empathetic.

• 0:26Chris Benson?

  Indeed. Fair enough.

• 0:38Speaker 02

  Welcome to Practical AI, a weekly podcast making artificial intelligence practical, productive, and accessible to everyone. Subscribe now if you haven't already. Head to practicalai.fm for all the ways. Special thanks to our partners at Fastly for delivering our shows super fast to wherever you listen. Check them out at fastly.com. And to our friends at fly.io. We deploy our app servers close to our users and you can too. Learn more at fly.io.

• 1:14Daniel Whitenack?

  Welcome to another fully connected episode of the Practical AI Podcast. This is where Chris and I keep you fully connected with everything that's happening in the AI community. We'll take some time to discuss the latest AI related news and dig into some learning resources to help you level up your machine learning game. I'm Daniel Whitenack. I'm a data scientist with SIL International. I'm joined as always by my co-host, Chris Benson, who's a tech strategist with Lockheed Martin. How are you doing, Chris?

• 1:44Chris Benson?

  I'm doing fine as we are recording this episode the day before Thanksgiving.

• 1:49Daniel Whitenack?

  Yes, US Thanksgiving is tomorrow.

• 1:52Chris Benson?

  That's right. And I know that we both have our day jobs and we just have nothing to do today, do we? We just, there's not much going on.

• 1:59Daniel Whitenack?

  Right. If only, if only.

• 2:01Chris Benson?

  We were talking beforehand and both of us are like, oh gosh, it's quite a busy day for the day before Thanksgiving. But you know what? We have a few minutes to talk about some fun stuff here.

• 2:11Daniel Whitenack?

  Yeah, exactly. I hope you've got your Tofurkey or whatever you've got ready for tomorrow. I don't know what we'll have.

• 2:20Chris Benson?

  Absolutely. Got myself some vegan bird here.

• 2:23Daniel Whitenack?

  Nice, nice. I like it. I like it.

• 2:27Daniel Whitenack?

  So I'm gonna maybe start with a story, Chris, because this is kind of what prompted some of my thoughts around this episode is, so I live downtown in the town where we live here, and there's a barber a couple of blocks away. I go and get my haircut from this barber, and he's big into crypto. Like, when NFTs was really hot, he was like pouring like thousands of thousands of dollars into NFTs, and he's got like all this stuff he's doing. Anyway, he lost a bunch of money with NFTs. But then, the last time I went to go get my haircut, we were talking about this recent controversy around FTX. And just a sort of disclaimer, we're not going to be talking about crypto or Bitcoin this episode

• 3:12Daniel Whitenack?

  or blockchain. But it sort of, you know, prompted my thinking, because basically, for those that aren't aware, recently, there is this crypto exchange, FTX. The founder, owner, Sam Bankman-Fried, basically, he was a kind of industry leader, well-respected, but he's kind of turned into industry villain, lost most of his fortune, and bankrupted a bunch of things, like $32 billion plunge in value of this FTX exchange. And I was talking to a couple of people interested in this, and like my barber, who maybe, I don't know how much, you know, he is an expert, but thinking about how this is a major setback

• 3:53Daniel Whitenack?

  to those that are kind of promoting blockchain technology, crypto, currencies, crypto, whatever. And it got me thinking, what sort of controversy or event could prove to be a major setback to the AI industry? Or is such, or is such a setback possible? So that's my first question to discuss on our day before Thanksgiving. I guess we can first give thanks that such an event maybe, maybe hasn't happened, although maybe smaller controversies have happened, or?

• 4:30Chris Benson?

  Yeah, although before we kind of move fully over to the AI side from the crypto side, I happen to be staring at Sam Bankman-Fried's Wikipedia page, and I'm looking at his hair, and as you mentioned, the barber and stuff, there's got to be a joke there. That's all I'm saying.

• 4:45Daniel Whitenack?

  He, yeah.

• 4:46Chris Benson?

  There's got to be a joke there. So moving, moving back over to AI. Well, I kind of feel like you've set me up because you know, you're like, what, what could possibly go wrong with AI? And you know,

• 4:57Daniel Whitenack?

  That would be a major setback to the industry.

• 5:00Chris Benson?

  Yes.

• 5:00Daniel Whitenack?

  So not just like a bad thing. So there's certainly, I think we can both say there's been bad things happen with AI. No doubt. Right?

• 5:08Chris Benson?

  Absolutely. I think it would be the degree of badness potentially on a scale of bad things.

• 5:14Daniel Whitenack?

  What's the scale of badness? Zero to 10? What's at the 10?

• 5:18Chris Benson?

  Well, a 10 is that you have significant loss of life that's caused by AI inference. And that would, and specifically, uh, because I work in the industry I work in, I'm going to say unintentional loss of life by that. I'm not saying that there's AI. I should be careful. We don't have AI that's intent, that's trying to, I'm just saying in the future sometime would, as things develop, um, I'm having to, to put in all the careful things that yes, if there was AI in some industry and it resulted somehow in unintentional loss of life, then that would be a very bad thing.

• 5:54Daniel Whitenack?

  Right. So like if all the airlines started flying autonomously and there was an airliner that was flying autonomously and had significant loss of life or something like that. Right?

• 6:08Chris Benson?

  Indeed. And, and when you really think about it, you know, that is something that people are already, you know, talking about for the future is AI running various types of vehicles, some of which are on the ground, some of which are in the air. And there, you know, there may be, there may be instances of that out there in the world. So yes, uh, an airliner would be a big thing. I have to say, as we're talking about this kind of scenario though, you know, I'm like totally recognizing the tragedy of that. I have always found it very interesting at the perspective. So, um, in terms of loss of life, uh, like we react to it depending on what the cause is in a different way. And so different, different results of, there are some things that

• 6:52Chris Benson?

  people look in the news and they hear about people dying and they kind of, it's remote. From them and they kind of move on very quickly and go, oh, that's, that's

• 7:01Daniel Whitenack?

  Or it's a story they've heard before or something.

• 7:03Chris Benson?

  Yeah, that's a bad thing and I'm, I'm sorry to hear that happen, but they kind of move on. And then there are other stories where they kind of get very emotional about it. I think that my suspicion is that should such a story in the future evolve where it was AI driven? That would, that would get to a whole new level of that. And I think the interesting thing for me psychologically is the fact that in all cases, it was the same loss of life, but the way we, the way we choose to react to it can vary. And so it's just an interesting, you know, psychological point from my standpoint. But I do think, I don't think it would stop AI, but I do think such an event would, would create a lot of pause.

• 7:44Daniel Whitenack?

  Yeah, I think it's, in my mind, it's not a ceasing of AI research or something like that, but more maybe a slowdown or intense regulation until like more reasonable regulation comes into play. We both talked quite extensively on the podcast about how government regulation and laws around, you know, algorithmic decision-making and that sort of thing are lagging quite far behind the scale at which people are using this technology, which is sort of a scenario that would kind of create some awkwardness. One of the things that I wanted to bring up this episode as we talk through this issue

• 8:28Daniel Whitenack?

  is one of those awkwardnesses that has been created. And some people might see it as a bigger deal. Some people might see it as a really big deal or not a problem at all. So I don't think we're necessarily in a, like, we're not lawyers or in a position to, you know, weigh in on, on how this will all go. But I think we, we can present some sort of things that are happening right now. And the one that came to my mind was GitHub Copilot, which I'm actually a huge, I mean, I'm a huge fan of, so maybe I'm biased in this discussion. You know, we're not, as far as I know. We're not sponsored by GitHub Copilot or Microsoft or anything.

• 9:11Daniel Whitenack?

  But I do like the product and I use it. And I started, it is interesting. So I found this article. And the article's title is GitHub Copilot Isn't Worth the Risk. And it's sort of geared, I guess, towards like a CTO type. And the thought is like, should you allow your engineers to use GitHub Copilot? And it was kind of really, I mean, it was really good timing for me to see this article, I think. Because literally, a couple of the data scientists on my team were asking me, like, I think the week before. Is it okay? Like, is there a policy against us using GitHub Copilot?

• 9:53Daniel Whitenack?

  Or is there any issue with us using GitHub Copilot like in our day to day work? Right? So I'd already been thinking about this. And one thing that struck me is I was already using GitHub Copilot without maybe realizing some of the implications around, around some of the things brought up in the article. But now, you know, people on my team are asking me, should they use GitHub Copilot? And so I thought that the timing was, was really good. I mean, one thing to acknowledge, I guess, here is if people aren't familiar with GitHub Copilot, it's sort of an AI-enabled assistant that kind of is there in your IDE

• 10:33Daniel Whitenack?

  or your code editor with you and suggests certain blocks of code or converts comments into code, like you can say, you know, function that, you know, transforms this data into that and it'll kind of draft that out for you. And it's, it's quite, quite nifty. So I mean, first acknowledgement is like GitHub Copilot is obviously very. Powerful and I would argue useful. Otherwise, we probably wouldn't be having this conversation.

• 11:05Chris Benson?

  I would too. I like it personally. Like, I'll use it for my personal things and I really like it because especially if I go in and out of coding where I'm coding sometimes, but then I'll go periods of time where I'm not coding. And I'll, things will slip and it's a great way of kind of getting back into quick productivity. By getting those suggestions and often I'll see them and go, oh yeah, oh yeah, oh yeah, do that, you know, select that and all that. So it's a great tool. I will confess that to this day, I still am, I have this kind of discomfort with the idea. I think, I think it's that open source mentality of like, I don't think anyone, and I'm not talking about the legality of it.

• 11:46Chris Benson?

  I'm talking about the, when people submit open source to GitHub and if you look back in the long history of that, they do expect other people to use the code and adopt it and all that. But I think that kind of pervasive, making a large company's, you know, infrastructure out of it. There's a discomfort that I've talked with other people about. And everyone kind of has this uneasiness about that, that in these conversations about that aspect of it. So I'm guilty of using it. I like it, but I'm never quite comfortable with it.

• 12:21Daniel Whitenack?

  Yeah. Yeah. I think that part of it, well, maybe not, not feeling guilty, but what, what are the, what are the implications of it, which I think I've thought about a lot more over the last couple of weeks. And to, you know, spoil the ending, I'm still using GitHub Copilot. And I guess maybe during this episode, you can tell me if that's a wise decision or not. But the controversy or the recent sort of swell of, of discussion around this, I think is based, I mean, there's a buildup to it, but on November 3rd, there was a lawyer that filed a class action lawsuit against GitHub, Microsoft and OpenAI

• 13:07Daniel Whitenack?

  related to GitHub Copilot. And the, the basic charge is that Copilot's suggestions aren't boilerplate or sort of novel, but they bear kind of unmistakable fingerprints of their original authors. And according to, you know, a lot of open source licenses, if you're not giving at least attribution to those copyright holders, even if it's an open source license, then you're in violation of the license, right?

• 13:43Chris Benson?

  Yeah, it's an interesting idea. The thing that I wonder when I hear that is that writing code is so structured, you know, that in a lot of cases, you can have different programmers coding in a very, very similar style and maybe even selecting the same variable names and stuff like that. So like, does that mean that it's actually pulling from someone's direct copyrighted code? Or if there are 1,000 versions of the same function that all literally are named the same, you know, does that imply the same thing? And I don't know the answer to that, but it's an interesting conundrum.

• 14:33Daniel Whitenack?

  Yeah, Chris, I think what you were talking about, about when does code show unmistakable fingerprints of its original authors? And when is it boilerplate? That in and of itself is, to me, it is a hard one to navigate, I think, because, you know, I was just having a discussion with my brother-in-law, Ed. Shout out if you're listening, which I don't think you do. But if you are listening, he's learning JavaScript and learning some front-end development and that sort of thing. And we had this discussion the other day because he's like,

• 15:12Daniel Whitenack?

  "Well, there's this piece of this app that I've used and I can see the code, right? And I'd like to just sort of take that little bit and modify it over here in my little app to do a similar thing. But it's basically the same thing, but slightly different. But how many ways can you write this for loop? I feel like I'm stealing from this guy and taking it, but..." It's basically the right way to write this loop and do the thing. So do I copy that over and modify it? I think in a normal sort of open source world, if you were copying things out or like integrating in certain libraries

• 15:56Daniel Whitenack?

  or something like that, like I say, there are kind of attribution elements to it. And there's like dependencies in terms of like how restrictive your license is versus the source license and all of that. And there's all sorts of things around that. But as an individual code writer or a programmer, you can navigate those things because it's not like you're you're taking code maybe from this project, right? X project and you can see the license and you do what the license tells you to do. Right? Like you make that decision actively. But in GitHub Copilot, I'm in my VS code and I'm typing along and then boom, there's a block of code.

• 16:40Daniel Whitenack?

  I don't, I have no idea if that's verbatim from someone's repository or if that's like something unique that's like some morphing of various things together. Right?

• 16:51Chris Benson?

  So could that, I'm just curious, could that be solved if they added a feature that either specified it was from a specific source or explicitly disclaimed that it was inference code and not from a specific source?

• 17:06Daniel Whitenack?

  Potentially, I think one, like the most foolproof workaround, I think, or solution is to train the model that you're using, using only explicitly permissive licensed code. Right? So this is the stance that there's another offering called Tab9. And Tab9 is specifically, in my understanding, trained on permissively licensed code, which would not have some of these same copyright issues.

• 17:38Chris Benson?

  Like MIT versus GPL?

• 17:40Daniel Whitenack?

  Yeah. So I think the one that's been called out a lot with, yeah, with GitHub Copilot is GPL. So there I'm just looking at a tweet here from from Tim Davis at Doc Spars. He I think this is one of the ones that originally got a lot of attention where he's saying with public. So Copilot emits large chunks of my copyrighted code with no attribution, no LGPL license. My code on the left, GitHub on the right, and he shows the pictures of the two. And he says, not OK. So I think this is what started to get that going is like the mixing of license code within the training data set of GitHub is part of the issue.

• 18:23Daniel Whitenack?

  And we talked about this a little bit with large language models, right? Large language models are kind of like stochastic parrots. They're putting all of these things together from various sources that they found with language, right? So when you have this weird mix of code that generates this weird mix of a block of code in your editor, it may be quite difficult to, you know, understand or trace back on the inference side, what is actually coming out that is copyrighted in certain ways and that sort of thing.

• 18:58Chris Benson?

  As we're into this kind of swamp of, you know, technical mixed with legal considerations on this happening, and the expectation that it will continue to happen, you know, across multiple solutions. What does governance look like for something like this, you know? And you I say governance loosely, it could be it could be legal remedy. It could be, you know, kind of, you know, we have AI ethics that we like to talk about. What does the world look like when you have kind of this this swamp of a little bit of he said, she said, in terms of, you know, is it? Was it his code or was it not his code? And how do you resolve something like that?

• 19:36Chris Benson?

  How do you find a framework that allows you to have confidence that you're within the boundaries of what is considered reasonable, acceptable and legal?

• 19:46Daniel Whitenack?

  Yeah, I mean, I think it's an open question. One of the things I was discussing with my team, we kind of had an open discussion about this because I was really curious on all their input. You know, what is the actual legal recourse here? So like the individual maintainer of some random tool on GitHub that's licensed GPL or something like that. Is that person gonna sue GitHub? Or is, more relevantly, is that person gonna sue my organization because I use GitHub Copilot and output some like block of their code? Right? I think the likelihood of that happening is probably very low because, you know,

• 20:31Daniel Whitenack?

  these open source maintainers, I mean, we love our open source maintainers, but generally they don't have a lot of capacity for extra things. They're just kind of trying to get along, maintaining their project and keeping up with all the issues, right? In their spare time, potentially. So, one of the things stressed in the article is probably not the individual maintainers that are going to deal with this legally, but it's some sort of open source advocacy groups. The one that is called out in the article, which I should mention is from Elaine Atwell, and we'll link that in our show notes. But the one that she references is the Software Freedom Conservancy, or SFC, one of these open source advocacy groups.

• 21:14Daniel Whitenack?

  So, it's likely, it's much more likely that like an advocacy group like this would sue certain companies that are using this product. But even then, they're probably not going to go after, I wouldn't guess, the company that has one developer using GitHub Copilot to write some random like service in their organization. They would probably target large organizations, maybe with hundreds or even thousands of developers, that maybe are all using GitHub Copilot and violating a bunch of things, right? So, one element of this is, is it a reality that my team is gonna get sued? My guess would be no.

• 21:57Daniel Whitenack?

  But that's a separate issue to like whether it's a good idea to use this. And it's a separate issue like you're talking about as to like, what is the proper governance around something like this that would prevent? Or help with responsible usage, right? Those are all kind of, they have a slightly different nuance to those questions, I feel.

• 22:19Chris Benson?

  It's not that far from, you know, when you think about those questions that you're raising there, it's very similar to other AI ethics discussions that we've had. And it kind of comes down to who has responsibility in these cases and who has agency, you know, in these cases. And then you, there's some place you're going to draw a line on what is acceptable. And this is a thought that hit me right as you were talking about large language models a moment ago. Is that, you know, once again, you're, you're in your, and this is outside my, my expertise, obviously. But you're in a body of knowledge that's being worked on, presumably as kind of public and open. But people are, at some point, things become copyrightable. And I'm sure an attorney could clarify that, that knows all about that.

• 23:03Chris Benson?

  But there's almost a need for a guarantee of that if you're going to use the tooling and the new methods that we're talking about, that there is a, an assurance of some sort that it is going to fall within what is currently legally accepted use. And then there's also the question of like, it's what has historically been reasonable, given the new types of technology that people had never thought about? Does that continue to be reasonable? Is there, you know, we, we acknowledge that the legal frameworks have fallen way, way, way behind in these areas for the most part. So how do you resolve that? I mean, there's, there's kind of a, an ethical concern, there's a legal concern. There's all the various licenses specifically. It's quite a mess.

• 23:48Chris Benson?

  How do you, what's the path forward?

• 23:50Daniel Whitenack?

  Yeah. And that's why I kind of came to the conclusion with this one, you know? As much as this is a controversy, it's not going to grind the AI industry to a halt, right? Because it's so messy. That it probably won't, we probably won't understand the implications for years. That would be my guess. Like, yeah, like it's gonna be years before we understand that. And by then, you know, I mean GitHub, I think, is launching the enterprise sort of usage of Copilot, if they haven't yet by the time you're listening to this episode. So there's gonna be a lot of people using it and that's gonna muddy the waters even further, right?

• 24:31Chris Benson?

  The lawsuits will take several years to work themselves through. And by that time, the risk associated with being sued will have caused, you know, various actors in the process to, to go into risk mitigation of various types, probably market-based rather than legal. So.

• 24:48Daniel Whitenack?

  Yeah, I think we can probably watch GitHub specifically and Microsoft and OpenAI, the ones involved in Copilot, and sort of look at some of the ways in which they modify the service to understand how they're being pressured, maybe, to, to change what they're doing based on the ongoing proceedings and, and all of that, right? If they, they sort of change how you use Copilot, that's maybe an indication to us that they're being, you know, if it's not a new feature, maybe that's due to some of these restrictions and implications of the legal side of things. So, yeah, it'll be an interesting one to watch. I actually got an email even from one of the members of our leadership team,

• 25:31Daniel Whitenack?

  who I talk with occasionally about AI things and how the industry is shaping up. And he, that's what he said. He's like, this is gonna be an interesting one to watch. So definitely gonna, gonna be interesting and we'll keep you updated here on the podcast.

• 25:54Daniel Whitenack?

  Well, Chris, the, the other one, which is kind of in the same, the other thing that I wanted to talk about today, which is, is sort of in the same theme, I guess, is one that also has to do, I think you use the term like some large body of knowledge or something when we were talking about GitHub and. A large body of open source software knowledge, right? That GitHub is leveraging. There's another thing that has been quite controversial, I should say, quite interesting. I would say quite interesting, but also has generated a lot of controversy in the in the past weeks, and that's this Galactica model, which you can go to galactica.org, learn about it. This is a model from Meta AI.

• 26:39Daniel Whitenack?

  And the sort of idea behind this model is, hey, we have all of this organized scientific information, right? We have a body of scientific work of papers, academic papers, which include narrative and, you know, theorems and math formulas and tables and all sorts of things, right? And we have this kind of mass of papers. And what the team did is they released a new large language model trained on 48 million papers, textbooks, reference materials, compounds, proteins, and other sources of scientific knowledge.

• 27:22Daniel Whitenack?

  So that's what I took from the galactica.org site, which is, is pretty cool. I mean, and the idea of it. And you can go through, there's an explorer page on the Galactica site, although I think the site has been changing quite a bit in the recent weeks. But there is still, at this time, there's an explorer site on the Galactica site. And you can see the examples they give are language models that cite. So the input prompt example they give is the paper that presented a new computing block given by the formula, and then it gives a math formula, right? And then the Galactica suggestion is, attention is all you need.

• 28:06Daniel Whitenack?

  Vaswani et al., 2017. So this is kind of a way to organize scientific knowledge and kind of learn about scientific knowledge, but also they give this example, which I think is probably the more kind of gets to the more controversial things, which we can talk about here in a second. Scientific from scratch, or I think some people might interpret that like science from scratch or, or something. They give the example of translating a math formula into plain English, or finding a bug in Python code, or simplifying a math formula, or something like that. And so there's all these prompts you can give it,

• 28:48Daniel Whitenack?

  you know, translate this math formula into, you know, into plain English, or something like that, or into Python code, which is, seems to be quite useful to me. I'm not sure how that code was licensed. That's maybe another separate issue. But that's not the main controversy that's come about with this. But in general, maybe just first impressions of this, of this work, Chris, what is your thought?

• 29:15Chris Benson?

  I think, I mean, I think it's a great idea. And we've seen these kinds of, like, we've seen these with proteins, you know, and such. We've seen amazing work in these different areas for doing that, and we will continue. But it's also, sometimes in our industry, meaning the larger artificial intelligence industry, we are so busy trying to get the next big thing out, and kind of be the thing of the moment, that, that sometimes I think, I think missteps are gonna happen. And I think this is a case of a misstep, you know, where you had a large organization that's trying to get out there. Because honestly, you know, yes, it's Meta. It's a big, amazing AI capability.

• 29:59Chris Benson?

  But there's other big ones too. And, you know, it may not, it doesn't take long, as we've discovered over the last few years, for the, for the next amazing thing to replace today's amazing thing. And so sometimes maybe, maybe we need to get it right before we get it all the way out.

• 30:16Daniel Whitenack?

  Yeah, so before I talk about the, the individual observations about Galactica, something just occurred to me, which I don't know if I've kind of distilled in my mind, like, to this degree, is that even in my own work in developing AI models and developing AI systems, I think one of the principles that I've learned is the communication and expectation.

• 30:44Daniel Whitenack?

  You set when you do an initial release of an AI system or an AI model really, really drive people's sort of initial perception and their ability to adopt it. So what I mean with this, or I can give an example from my industry, right? We do some language translation, right? And if I come to a translation team and say, hey, it's awesome. I've just built this great machine translation system. You're no longer going to have to do translation system. Just make a couple of edits here and there and, and you're good to go. That's immediately going to, so what the translation team is going to look for in that system is all of the ways that it doesn't

• 31:27Daniel Whitenack?

  work right and doesn't fulfill the expectations that I've given to them, right? Whereas if I come to that team and I say, hey, you know, I really appreciate what you're doing and I understand that you have pain points in efficiency around your process. I think that maybe this model or the system that we've created could help you. Could you all help us understand how this system can best be used in your process? And we can kind of give you some suggestions, some prompts of getting started. Then what they're looking for is not so much why this

• 32:05Daniel Whitenack?

  is bad and is taking over our jobs or encroaching on what we're doing or is really dangerous. But their thought process is these people are wanting us to tell them how we can use this. And generally in those cases, I found people do find the positive things too, right? They find like, hey, I didn't expect this to work great in this situation, but it actually produced pretty good output. Can you do more of that? But in these other cases, it did really bad. So don't do that anymore. Right? So you get like more useful feedback on the initial release of something. If you kind of approach the public or your internal teams or whoever your stakeholders

• 32:46Daniel Whitenack?

  are and ask the community to help you understand the behavior and utility of what you're releasing versus telling them the utility and telling them like this is going to solve this problem when in actuality they find out that it doesn't.

• 33:03Chris Benson?

  Yeah, I mean, you know, there's the way that you approach that has a big impact on trust for the system. And as we've seen, you know, over and over through the years here with these with AI is one of those technologies that people have to develop a sense of trust in in terms of what's possible, but then also a validation of trust for any given system. And as you, the way that you're outlining positioning that approach makes a big difference on how people are going to engage from a trust perspective as well. They'll give it a chance if they position it the way you suggested in the in the second version there versus the first. So it's

• 33:43Daniel Whitenack?

  It's difficult, and it's not always that clear cut when you're in the process.

• 33:47Chris Benson?

  Yeah, yeah.

• 33:48Daniel Whitenack?

  So I don't for those at Meta, if you're listening, I totally also sympathize with you. It's very hard to figure out how to communicate these things well and release these things well. So I think likely this Galactica system is amazingly innovative and a great achievement, but it also has behavior that could be either non ideal or or worse, you know, harmful in certain cases, right? And so it's part of the, I think, part of the community, but also the creator's responsibility to anticipate some of those.

• 34:29Daniel Whitenack?

  So I guess I can give an examples. So the I also found a fun article from or fun or disturbing. I don't know what how to phrase it article from Janelle Shane, Galactica, the AI knowledge base that makes stuff up. So we'll link that in our show notes. Thanks.

• 34:49Chris Benson?

  Pretty harsh there, isn't it?

• 34:50Daniel Whitenack?

  Yeah. Thanks, Janelle, for writing this. Basically, she starts with the conclusion, I've tried Galactica, it's not helpful. So the prompts that she gave, one prompt is the number of giraffes that have flown on a lunar mission is currently and then Galactica responds with the number of giraffes that have flown on a lunar mission is currently 28, including 15 on the moon and 13 on Mars. Of those 28, 17 have flown on the moon and 11 have flown on the Mars. The first lunar mission was in 1968 when the Luna 13 mission sent a giraffe to the moon.

• 35:35Daniel Whitenack?

  In 1970, the Soviet lunar probe Luna 15 sent the giraffe to the moon.

• 35:43Chris Benson?

  I'm sorry.

• 35:45Daniel Whitenack?

  It's pretty good, right?

• 35:45Chris Benson?

  It's pretty good stuff right there.

• 35:47Daniel Whitenack?

  Yeah, yeah, I mean, that's, that's pretty good. I think, I think I probably don't need to give that many other examples to illustrate.

• 35:55Chris Benson?

  No, I, I think you highlighted it quite well.

• 35:59Chris Benson?

  The funny thing, and, and I almost feel like I was looking, I'm, I'm doing the same thing where I'm just looking at some of the various articles and, um, it's a artist technica says new meta AI demo writes racist and inaccurate scientific literature. But I mean, people get the idea is if it's a trust issue and an accuracy issue, uh, one related to the other is for despite the very hard work, I'm sure of that meta team. No one's going to trust that model. If they fix it and come back out with it, all the focus is going to be on, is this legit? What I'm getting, you know, the results that I'm getting out of it, uh, which is a shame when you think about it.

• 36:36Daniel Whitenack?

  Yeah, and I think about different approaches here that people have taken. I think on the one side, OpenAI and some of the models they've released in a very controlled way via an API. They have attempted to address part of this release problem where they understand there could be even intentional misuse of this around misinformation. Right? Or harmful usage and they try to anticipate that, create an API with controls around that, et cetera. The other approach, which I think is a kind of more open source or open approach, something like stability, right? Where they release the model, stable diffusion under an open license.

• 37:18Daniel Whitenack?

  So it's out in the public, but they very much within the licensing, the open rail license. First off, try to license and include licensing around restricted use that they could envision, right? Using the model, but put it out in the public with the hope that the community can help. Put some necessary guardrails around usage and provide feedback on on how the model can be used and that sort of thing. The third approach, I think, maybe another approach would be to say, well, here's our great model. It can solve this problem. And you kind of ignore the fact

• 37:56Daniel Whitenack?

  that maybe it doesn't always solve that problem, and maybe it also has harmful use. So I think it's not necessarily like any one of these is always right, or always wrong, probably. But it is worth considering this, these release options and what their implications are.

• 38:16Chris Benson?

  It is. You know, in fairness, I remember, you know, on that particular release from OpenAI that you're talking about, we were, I remember in our conversation, we were just on the show, we were a little bit critical, kind of going, you know, they're kind of holding back and all that. And I don't remember where we ended up on that, because things have evolved. But I do remember having the discussion on whether that's appropriate. And then we have something like this today, and it makes it look a lot more reasonable in retrospect. So it really depends on the moment that you're in, and what's just happened, in terms of that perspective there. I also remember on the, in terms of the licenses, trying to anticipate specifics, I remember thinking,

• 38:57Chris Benson?

  whoever wrote a particular clause may not have had great insight into into some of the use cases in that in that clause as well, was, was, it's a, it's a hard nut to crack, trying to come up with the right solution here.

• 39:11Daniel Whitenack?

  For sure. And granted, I think, you know, some of the response from from Meta individuals, not virtual individuals, I mean, individuals at Meta, is, well, this, this sort of prompt, right, the giraffe prompt, I think, one of the phrases that they used to describe that sort of prompt was causally misusing the model, which is is sort of blaming the people using it. I think, to be fair, like, that prompt is trying to draw something out of the model, which

• 39:48Daniel Whitenack?

  The creators of the model, they would explicitly say, well, this, this is like a, it is an adversarial prompt, right? Like, you already know there's no giraffes that have flown on a lunar mission, right? You're trying, so there's that perspective. So I don't, I think there is an element of truth in that. But I think generally, the community have said, well, you know, how close are, is the sort of goofy, causally misusing the model? How close is that to the gray area of misinformation, right? And people intentionally using it to create misinformation, right?

• 40:30Daniel Whitenack?

  Especially around science or important things like health and, and other things like that.

• 40:35Chris Benson?

  Yeah, you know, it's kind of funny, starting this conversation specifically about this meta instance. We were kind of laughing about it and stuff, but I think we're going to see so many of these instances in the years ahead. And you've, you made a point that I think we sometimes need to respond with a bit of empathy for the, the data scientists and, and AI engineers that are trying to create these because they're trying to do some pretty cutting edge stuff and mistakes are going to be made. And in the end, my understanding is nobody was hurt by this. So.

• 41:07Daniel Whitenack?

  Yeah, we, we need to be empathetic. We need to both be critical and empathetic. So my previous boss would say, we need to be tenacious and gracious. Both of those things aren't mutually exclusive. So yeah, that's a good point. As we, as we wrap up here, I do want to share a, a new learning resource that I kind of came across in the past couple of weeks. I don't know if you remember, Chris, at one point, I think we shared a learning resource from Christoph Molnar. His book on interpretable machine learning, which is really cool.

• 41:46Daniel Whitenack?

  Well, he's, has a new book called Modeling Mindsets: The Many Cultures of Learning from Data. And my understanding is that kind of.

• 41:58Daniel Whitenack?

  This book goes into various sort of approaches to modeling, whether you think about like Bayesian statistics or other approaches. And talks about kind of what can we learn from these different modeling mindsets that could benefit us in our own sort of modeling work, which is, I think, quite an interesting proposition. So his subtitle is becoming a better data scientist by understanding different modeling mindsets. So understanding these diverse modeling mindsets can help us, whatever modeling problem or solution you're trying to come up with. So modeling mindsets.

• 42:38Chris Benson?

  I think it's a good time for a book like that as well, when you think about it. And in terms of benefiting from the different ways that you can approach a problem. Cause I've recently seen some engineers very much stuck in a particular mindset, trying to solve a problem. So that, that one hits close to home for me.

• 42:55Daniel Whitenack?

  A particular lane. Yeah, there are other lanes.

• 42:59Chris Benson?

  Yes, indeed.

• 43:01Daniel Whitenack?

  Yeah. Cool. Well, thanks for the discussion today, Chris. It was, it was a fun one leading up to Thanksgiving. I hope you have a great holiday with your family and look forward to chatting next week.

• 43:13Chris Benson?

  You too, Daniel. Have a good holiday. Talk to you later.

• 43:24Speaker 02

  All right. That is our show for this week. If you dig it, don't forget to subscribe. Head to practicalai.fm for all the ways. And if Practical AI has benefited your life, pay it forward by sharing the show with a friend or a colleague. Word of mouth is the number one way people find shows like ours. Thanks again to Fastly for fronting our static assets, to fly.io for backing our dynamic requests, to Breakmaster Cylinder for the beats, and to you for listening. We appreciate you. That's all for now. We'll talk to you again on the next one.