"Foundation" models

The term “foundation” model has been around since about the middle of last year when a research group at Stanford published the comprehensive report On the Opportunities and Risks of Foundation Models https://crfm.stanford.edu/report.html. The naming of these models created some strong reactions, both good and bad. In this episode, Chris and Daniel dive into the ideas behind the report. Sponsors: Fastly https://fastly.com/ – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorshipChangelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Report: “On the Opportunities and Risks of Foundation Models” https://crfm.stanford.edu/report.htmlFollow up Blog post: “Reflections on Foundation Models” https://hai.stanford.edu/news/reflections-foundation-modelsUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Daniel Whitenack?

  But then there's this idea of misuse. You can create a really flexible, extremely powerful model, which lets you do a whole variety of downstream things much easier. So in other words, you can do named entity recognition, sentiment analysis, create fake news, or target marginalized groups. On the one side, we're creating these large language models that we would like to be more accessible to people because we don't want the concentration of power around these foundation models being with just a few entities. So I don't know. I don't know what the right balance is there.

• 0:46Speaker 01

  Hello, friends. Jared here to tell you about Change Log++, our membership program for those of you who want to directly support our work. Your++ membership gets you closer to the metal with extended episodes, makes the ads disappear, and takes our audio to the next level with higher bitrate MP3s. You can join today at changelog.com/plusplus.

• 1:19Speaker 01

  Welcome to Practical AI, a weekly podcast making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join us at practicalai.fm/community and follow the show on Twitter. We're at PracticalAIFM. Thank you to our partners at Fastly for shipping our pods super fast all around the world. Check them out at fastly.com.

• 1:52Daniel Whitenack?

  Well, welcome to another fully connected episode of the Practical AI Podcast. In these episodes, Chris and I keep you fully connected with everything that's happening in the AI community. We'll take some time to discuss some of the latest AI news and then we'll dig into some learning resources to help you level up your machine learning game. I'm Daniel Whitenack. I'm a data scientist with S I L International. I'm joined as always by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How you doing, Chris?

• 2:24Chris Benson?

  Doing great, Daniel. How you doing today?

• 2:26Daniel Whitenack?

  Doing pretty good. How's the strategy world after that, our last fully connected, when we were talking about innovation teams and stuff? A lot of those ideas have been swirling around in my mind and I just figure that's like what the life of a strategist, like you're, you're thinking about those things all the time. So I've been thinking about you over the past week as you've, you've been strategizing.

• 2:48Chris Benson?

  We pretend like we know what's gonna happen next. It's a great gig if you can get it.

• 2:52Daniel Whitenack?

  Yeah, well, that's the thing. Isn't the saying like, if you try to predict the future, that's like the one thing you can be sure you're gonna get wrong?

• 2:59Chris Benson?

  Yeah, yeah, yeah. I'm pretty okay being wrong, too.

• 3:02Daniel Whitenack?

  Yeah, you sort of embrace that.

• 3:04Chris Benson?

  We did the strategy one and then I know we've, we've talked about data fabric recently and that's also been another interesting thing. I'm still working on that pretty intensely and some stuff that I'm doing and having a good time because you know what? This, this whole AI thing keeps rolling forward.

• 3:20Daniel Whitenack?

  Yeah, yeah, definitely. And there's something I wanted to bring up in this episode and, and sort of talk through with you. It's not like an extremely recent thing. It was back in, I think, August that some of this came out, but we haven't talked about it yet. And I realized we haven't talked about it yet. And at the time, there was a lot of language floating around that I didn't really know, like what people were meaning when they said this stuff. You know, I've kind of circled back to some of that and kind of figured out what, what we're talking about at least a little bit. And I'd love to talk through that with you. And that's around the topic of foundation models. So

• 4:02Daniel Whitenack?

  This is something that was work done at Stanford University, actually in a Center for Research on Foundation Models, CRFM, at Stanford University. And they came out with a fairly lengthy and very interesting report on this topic, which they're calling "Foundation Models." And I remember at the time, people were sort of like responding to, I think mostly the the name, "Foundation Models," like, "Oh, well, what are you..." Like, maybe there's some hubris around like saying, "Hey, we're working on foundation models, which are the foundation of AI."

• 4:45Daniel Whitenack?

  I think clearly, they've also released some articles around this or blog posts, and I think that clearly wasn't, you know, at least their intention. But it's a super interesting article. So, the article that they wrote is on the opportunities and risks of foundation models. First off, I guess, Chris, have you come across this term?

• 5:06Chris Benson?

  I have come across it, and I too have found it confusing in the same way. And so, this is an interesting article that you found here. You want to give us a definition?

• 5:16Daniel Whitenack?

  Well, I figure that being in the good spirit of a programmer, I would just rather copy and paste than create my own. So, we'll go for the definition from Stanford. And that's actually, I think, a good thing to start with, just sort of parsing through. So, they say, "We define foundation models as models trained on broad data, in parentheses, generally using self-supervision at scale, that can be adapted to a wide range of downstream tasks." So, let's sort of take a step back and parse through that. I think most of our listeners are familiar with some type of models, that jargon.

• 6:02Daniel Whitenack?

  By this, probably we're meaning mostly like neural network type of models, modern model architecture, but basically some sort of parameterized

• 6:12Daniel Whitenack?

  Function in code, right, that is trained based on some data using training algorithm of some type, an iterative training process. And so that's a model and it is, they're saying foundation models are trained on broad data, generally using self-supervision at scale. So some of those terms, particularly self-supervision, might not be the most familiar to people. Have you, have you run across this in your own work, Chris?

• 6:42Chris Benson?

  Yes, in terms of different levels of supervised learning and, and the, the fact that we are now more and more training models that are either unsupervised or at some level between that self-supervised. The thing that I am curious about here is how should I think about this in the, in terms of transfer learning? Is this something that I would use as a base for transfer learning to build upon?

• 7:07Daniel Whitenack?

  Yeah, I think that when they're doing their report, two things are the two kind of pillars on which foundation models that they're calling them are built. If I'm remembering right, they talk about transfer learning and scale. So for those that are maybe familiar to transfer learning, that's where you have some sort of pre-training that goes on and that's on a task that's maybe related to the task that you want to eventually do. But then you sort of transfer and learn or fine tune that model for the actual downstream task, which you're interested in, which is different from the one, the, the pre-training task.

• 7:50Daniel Whitenack?

  And so I think this gets, it does get to that, Chris, because they're talking about a variety or a wide range of downstream tasks. So this is the scenario where you had some pre-training that happened on one of these foundation models, a, a large model. And then your transfer learning, or maybe in a zero-shot or few-shot way, that model's able to perform a variety of downstream tasks. So fit into that zone of like transfer learning, zero-shot, few-shot type of scenario. So that's one side of it, that pillar of transfer learning. The other side is scale. Now, the interesting, I think, connection between

• 8:31Daniel Whitenack?

  scale and self-supervision is that in the sort of, I guess, more traditional supervised machine learning world that we kind of started with, you always had a data set that was annotated or labeled for the task that you're interested in, right? And then you trained your model on that data set. The problem with that, right, is that that's hard to scale, right? It's hard to scale sort of across a wide variety of tasks because you have to have a data set for each of those tasks. And it's very expensive and time-consuming to create those labeled data sets. But also, if you think of the size of even one of those data sets,

• 9:15Daniel Whitenack?

  the more data you need or the more complex task it is, you might need to label a whole lot of data. So, for example, the question answering task in natural language processing, I've learned, because I'm working with some of that data now, it's really time-consuming to build those data sets or like conversational AI data sets. Sometimes a lot, a lot of very hard and difficult work goes into making those. So if you want to scale your data sets for training, in that sense, self-supervision is actually really kind of another important feature here of these models. Because let's say you had just a bunch of text, just like blah, a bunch of text, right?

• 9:59Daniel Whitenack?

  Well, you can actually, in a self-supervised way, create your own training data to train this foundational model or this large language model by, let's just take some sentences, remove words, and say the task is, tell me what word went here. And since you are the one that took the word out, right? You know what word should go there. So you're self-supervising this pre-training task, which is much more scalable in terms of preparing the data for training.

• 10:31Chris Benson?

  So I misused the term earlier. I was conflating unsupervised with self-supervised. Pardon my confusion there.

• 10:37Daniel Whitenack?

  Yeah, no, it's confusing terminology, right? There's this semi-supervised, self-supervised, unsupervised, supervised. There's a lot of supervision going on of different types. So yeah, it's worth kind of diving into those. But yeah, I think the idea is we're really talking about scale. And you can't build large data sets, at least not in a cost-efficient, time-efficient way, if you're always thinking just in terms of supervised learning.

• 11:05Chris Benson?

  Yeah, fair enough. So how might you, how might you think about implementing this? If you're out there and you're interested in moving into this as a new skill set that you want to develop for your organization, what would a good first step be?

• 11:17Daniel Whitenack?

  Yeah, well, I think that that's a really interesting question in this, in this work. Because actually, one of the things that they talk about, because they're talking about opportunities and risks with foundation models, is that accessibility and homogeneity are things that come up with foundation models. Meaning that actually, me, like my team, working, I might not be able to create one of these foundation models, right? Because I don't have the resources. So just to give a kind of couple of examples, we're talking about here, things, maybe that, certain things that people might be familiar with are things like BERT or GPT-3 in the language space.

• 12:02Daniel Whitenack?

  But then there's other computer vision models and other sort of modalities, speech models, like thinking of Wave2Vec or other things. So if you imagine like tasks in computer vision, tasks in natural language processing, tasks in speech technology, and you look at what people are doing now, very often, a bulk of the things that people are doing are sort of homogeneous in the sense that they're all built on one of these foundation models, whether it's BERT or Wave2Vec or one in the GPT variety. Or, you know, all of these are sort of Built on one of these foundation models like we fine tuned BERT to do this or we,

• 12:46Daniel Whitenack?

  you know, transfer learned this foundation model to do that. And so this is one of the things that they bring up in the paper as a risk is that there's sort of two things that come out from this, right? One is not all researchers and practitioners can create their own foundational model, because I don't have like racks of GPUs maybe, or the computational resources, the manpower to actually create a model on the scale. So that's one thing is like, hey, is it bad? Is that necessarily a bad thing? I think it is concerning that like this sort of concentration of people that can actually create this state of the artwork.

• 13:27Daniel Whitenack?

  But then the other side of that is like, if there's an unexpected behavior or bias in that foundation model, then that actually filters through down to a huge number of downstream tasks and applications, because no one's retraining that foundation model, right? It's just there and everybody's using it, right? And so that's, that's another risk. So I think the implications are really what should be on the mind of practitioners in terms of this trend, maybe.

• 13:55Chris Benson?

  You know, what that reminds me of, just as an analogy, as you were saying that, is if we look at the software world in parallel, there are some software bits out there that you find in open source that are in just about everything, you know, they flow downstream. And if you get a bug, you know, thinking about the bias in the model or something, but if you get a bug in the software, something unintended, it can have fairly substantial consequences as it flows downstream. And I, is that the right way of thinking about this? Is that since you're relying on this so much, that if you do go awry, it's going to, to multiply itself many times?

• 14:30Daniel Whitenack?

  Yeah, definitely. I think that that is exactly the, it's not an exact parallel, because we're thinking of like mostly issues maybe stemming from data and bias in data, maybe, or something like that. But it is parallel in the sense that like, hey, if everybody imports this one package and it, and it breaks upstream, then there's a huge Consequence downstream, right? And the parallel here would be everybody's using, let's say BERT to create all of these different NLP applications. Well, what happens if there's this bias in BERT? And we've seen examples of failures of these large models, right? And so, yeah, I think that is a, that is a good parallel.

• 15:13Daniel Whitenack?

  So Chris, I think maybe one of the things that I remember when, when this came out is that people were kind of thrown off by the terminology. I think even in the blog post that I was reading, which I'll link in our show notes, they say the name foundation model has also drawn significant attention. And given that they want to clarify it. So there's a whole clarification in this article. I'll link in the show notes.

• 15:40Chris Benson?

  They threw me off at the beginning of the show.

• 15:42Daniel Whitenack?

  Yeah. I mean, naming is difficult, right?

• 15:44Chris Benson?

  It is.

• 15:44Daniel Whitenack?

  So like, what is that? Like naming, naming variables is the most difficult part of software engineering.

• 15:51Chris Benson?

  In the software world, there's nothing harder than naming variables. Well.

• 15:54Daniel Whitenack?

  Yeah. So I do feel for them and I think that they were making a, a good attempt at this. And I've, I've actually felt this tension maybe in our previous conversations because, you know, I come from the NLP world and we have this like term large language model, right? Which is essentially what they're calling a foundation model for our space, right? But the thought process around that type of model and the way it's used downstream, the way it's trained, that is pervasive across other modalities, right? In computer vision, in audio, even in areas where people are doing like biological structures

• 16:36Daniel Whitenack?

  and, and other things, this idea of having a large base model that's trained in a self-supervised way at scale and, and used in a variety of downstream tasks, that idea is, is kind of pervasive. So I, I do like the fact that like in the past, I felt this tension, even when I've taught classes, like what do I call this like thing we're doing? This like new trend in how we're operating. And I think you could talk about it under the heading of transfer learning, under the heading of like few shot, zero shot, or self supervision. But I do appreciate the attempt to create a term

• 17:17Daniel Whitenack?

  that sort of encompasses all of these things that are at play because there's definitely this this trend. So I don't know if you have any thoughts on that. I'm not opposed to the name, actually. I I kind of like having a name to refer to this as, but I don't know. I don't know if it's catching on. I don't hear like tons of people using it out there, so maybe it's not catching on. But I I kind of wish that it or something like it would, right?

• 17:42Chris Benson?

  I had heard the term, but just like self supervised learning, I don't think I had really gotten the correct meaning right on. It's interesting is in the article here, they they kind of talk about this trend, as you pointed out, and where it's going and whether what the implications are. And they they talk to that a little bit lower in the article. And they specifically kind of address the pace of technological process, the entrenchment of the models themselves because of, you know, some of the limitations that you pointed out earlier. And the demand of kind of the the human, social side of it and technology. And it raises a good point. I mean, they empower us to be able to use these things because it gives us a capability we might not otherwise have.

• 18:25Chris Benson?

  But there's definitely some pitfalls there. You know, especially in terms of of, you know, bias in the data that you pointed out. Where do you think, I mean, as someone who is using large models on a regular basis yourself and and the work, do you think that this is probably where things will continue going for some time? Or do you see any alternative? I think it's kind of a necessary evil to have foundational models, you know, by that name or otherwise, to build work upon, you know, shoulders of giants kind of idea. Any thoughts there or whether that's are the risks too great?

• 18:58Daniel Whitenack?

  Yeah, I think that, as you mentioned in my own work, like, we don't train for the most part. We're training a lot of downstream task models and few, maybe what would be Considered foundational models or in a this sort of self supervised pre-training way, although we have done it a bit. So I've benefited a lot in my own work from from this trend. And I think it does come with risks, though. I think when I was going through this article and thinking things through, a lot of it centered around sort of concentration of power and other kind of trends in the AI world, especially because I work at a nonprofit that works with

• 19:43Daniel Whitenack?

  minority language communities. For the most part, local languages, minority languages are left out of foundational models in the NLP world, right? So I I already view this as an issue. And they bring this up in terms of they talk about, you know, supporting diverse research. And I think regardless of the modality that we're working with, it's important that we consider whether it's geographic diversity, language diversity, but also diversity of those creating the data sets and training the models and having representation. All of those things I think are both are really critical. If this trend continues and we don't think about that side of things,

• 20:27Daniel Whitenack?

  then those sort of marginalized groups in terms of how they're able to use these foundational models and the implications of the applications that are produced, they're only going to become more marginalized. Because if foundational models are a key piece, or I keep calling them foundational, foundation models are a key piece of this sort of new tech stack that we're building for the digital sphere. And those groups are just left out, or there's bias in the models against those groups in various ways, then that's a real problem in terms of their own sort of flourishing in the digital sphere.

• 21:05Chris Benson?

  I think that's a fantastic insight that you just made there. And that is that, you know, from an economic standpoint, people who make these large models, these foundation models, are incented to solve the problems which are probably being done by kind of what is current mainstream, you know, kind of current top of the of the power stack, if you will, you know, in terms of companies and countries even. And so there is an incentive to unintentionally perpetuate inequality with these. And so it's a real, I think that would be a real dilemma. How might you tackle that going forward?

• 21:43Chris Benson?

  Where if if we're looking at foundation models as being somewhat core to most workflows in the AI space, because they're available and they accelerate where you're trying to go, but they are built on imperfect data that has bias that that leaves out marginalized groups. Do you just iterate and add those in and redo that? There's a certain investment, obviously, to be made in that. Do you have any thoughts on least evil path forward to get the best foundation model with the least amount of unintended consequence over time?

• 22:18Daniel Whitenack?

  Yeah, it's a difficult question. My sense is that that is why Stanford created the center, although I wasn't, I haven't, we would love to have someone from the center on the podcast, but I haven't talked in detail about the motivation to them. So I'm not putting words in their mouth. But I think there's multiple sides to this. I think one side of it is having, you know, think tanks or research groups that are really thinking about the problem and how foundation models, large language models, you know, computer vision models and other pre-trained models are influencing society more generally. And the risks, the opportunities with that,

• 23:01Daniel Whitenack?

  because there's clear opportunities, but there's also, there's also risks. So having that research side of things, I think, is a key piece. But also, I think, as practitioners, this needs to be part of our Our thought process when we're using these models. And I mentioned this fairly frequently on the podcast, but the idea of kind of probing the behavior of the models that you're releasing into production, I think is a key piece of this. So not just trusting that like, hey, Google or OpenAI created this great model. I'm going to fine tune it on this task. Okay, it does great on my test data set. So, you know, push it up to production.

• 23:45Daniel Whitenack?

  I think thinking of behavioral testing where you actually probe, how does my model respond to these changes in my data? What if I perturbed my data in this way? What if I switch out this for that? What if I make this change, which should be invariant in the output? Is it actually invariant in the output? And creating that sort of test suite of behavioral tests for your model, I think is a big thing that practitioners can keep in mind in terms of how these foundational models might impact their downstream customers. The report actually does, they talk about the social impact, but they have this mindset, there's a figure three in the paper.

• 24:30Daniel Whitenack?

  Their figures are really nice. I wish I could make figures that nice. They talk about these sort of almost like a value chain or something of usage of these foundation models, starting from data creation through data curation, through training and application and deployment. And they think about the impacts of those models along that chain. So like, there are real people on either side of that chain, right? What is the diversity that's represented in those groups? And what might be the impact of having biases in this sort of data producers or the consumer's side? The makeup of the consumers? So I think that's a really good way of thinking about it.

• 25:15Daniel Whitenack?

  And having that more holistic picture in mind is helpful.

• 25:19Chris Benson?

  Yeah, you know, as you were going through that, it struck me that that in addition to kind of accounting for the unintended consequences behaviorally as you perturb data and stuff, that the flip side of that coin becomes vulnerabilities from a security standpoint in terms of affecting the outcomes of foundation models or any model for that matter by doing that. And so it just has a greater multiplier, you know, associated with the foundations that you're building so much on top of that. So as we look at this in the days ahead, it will be interesting to see as well what the security implications are of protecting

• 26:01Chris Benson?

  the end users that are impacted by this, meaning, you know, the model will change the behavior of lots of different, you know, executions on that where people are are using it for various use cases that affect people. I would conclude by noting that in all of this that we're talking about, it's starting with human beings and it's ending with human beings. And we are a little bit at the mercy of the models as we're trying to make improvements in other ways in our lives to do things productive in the world. It's a curious set of problems that we face going forward.

• 26:31Daniel Whitenack?

  Okay, Chris, so in continuing our way through this material that Stanford has put out, they actually do have, I think your point made around like people on either side of this, they talk about emphasizing the role of people. So I think they're very much with you in that. They also use this terminology in terms of how to mitigate some of this risk and vulnerability stuff. They talk about how foundation models can and increasingly should be grounded. Now, that was also a term that I wanted to make sure that I sort of dug into a little bit

• 27:12Daniel Whitenack?

  because maybe it could mean a variety of things. Is that terminology you've come across?

• 27:18Chris Benson?

  No, I ran across that in the article a moment ago, just before you said it, and I was wondering, and then you went there. I was wondering, what do they mean by grounded? So what do they mean by grounded, Daniel?

• 27:29Daniel Whitenack?

  So it is interesting that there's sort of a couple of ways that you could come at this. And I found people like talking about this idea of grounded models coming from the vision perspective, but also coming from the NLP perspective.

• 27:47Daniel Whitenack?

  Both, I think you can kind of get to the same idea from both directions. If you're coming from the computer vision side, you might want to look up some things about CLIP, which is a model that's being talked about, or a methodology that's being talked about quite a bit. But from the language side, the interesting thing about language is that language is sort of a way to encode knowledge about the real world in some sort of concrete way, right? Like even the words that I'm talking about right now, I have thoughts and ideas in my mind. I have perceptions of the real world and I'm encoding them in a

• 28:30Daniel Whitenack?

  in a concrete set of symbol representations, right? And so there's a set of symbols that aren't really like like the set of symbols DOG, you know, conjures up ideas in my mind about what that, those, that set of symbols mean. And I know it does for you as well, having 10 or more of them at any given time. But those symbols are representative of things that are grounded in the real world, right? It's just a sort of symbolic representation of things that are grounded in the real world. And so this idea of grounding is thinking about more widely like, well, instead of,

• 29:11Daniel Whitenack?

  you know, always just having a labeled data set that says like, oh, here's DOG, here's dog in my text. And that's an entity, I'm going to label it as an entity of some type and, you know, do named entity recognition or something. Well, thinking about a language model being grounded is a much broader way to look at this where you realize, oh, well, that has other representations that are actually grounded in kind of multiple modalities of how people experience dog. And language is just one of those, right? Or textual language is just one of those. There's also a way to say dog, right, in audio? And there's all

• 29:56Daniel Whitenack?

  Sorts of, of course, the internet is filled with images and videos of dogs. And there's drawings of dogs. And so there's not a single sort of element that's representative of dog in any modality. Right? You could, you could do the same from vision, right? If you have a single picture of dog in your data set, your model is going to assume that's the only way you can represent dog. Right? And so when you see a different type of dog, it might cause your model to have a vulnerability. Because it's not used to classifying a pug as a dog when it all it's seen is a golden retriever. Right? And so the idea of grounding and, you know, if anyone wants to correct me, our listeners can correct me.

• 30:39Daniel Whitenack?

  But it really has to do with pulling together these modalities of data to ground our model in more than just a sort of single symbolic representation of, of an idea. But building multimodal models that can ground our ideas or our perception of these ideas in these sort of multiple modalities to increase robustness and flexibility, especially if we're using these on a variety of downstream tasks. I feel like I said a lot there. I don't know if any of that came true.

• 31:13Chris Benson?

  If I'm understanding you correctly, correct me if I'm not, but you're basically saying having that diversity to achieve what the model is representing. So the representation is built on diversity, gives it that robustness, it gives it that ability to, to recognize all the things that that representation might take the form of. Is that a good way of summarizing?

• 31:34Daniel Whitenack?

  Yeah, yeah. So I looked up a few things here in terms of this idea of grounded. I liked the description from, actually from DARPA, which, um, you know, I know you're, you're a fan of. But, uh, they talk about the explosive growth of language models and all this stuff. And they say, you know, ML suffers from a need to train on large amounts of annotated data. And then they talk about this sort of project that they're talking about, which is related to grounding. Ames to a enable computers to acquire language in a manner similar to how children do. Cause children acquire language based on their perception of oral and visual information,

• 32:17Daniel Whitenack?

  not just textual information. It's not just like you teach a child to read based on just text, apart from audio and visual. So it's a sort of broader way of thinking about how you teach a certain learning, how we teach our models, the information that they need to know, which is actually encoded in multiple modalities. And it might not always be like audio and video. Maybe it's related to like physics about how the world works or something like that.

• 32:50Chris Benson?

  Understood. I, I just want to note as an aside that I prefer your definition of grounding far more than the one I usually get, which is me on the weekends. It's my wife telling me, no, you're not going to go fly that Cessna. You're going to do yard work instead. You, you sir are grounded. That's what I usually hear. So yours is a much better definition in my book.

• 33:11Daniel Whitenack?

  Yeah, yeah. And you can think of how this would create or could create vulnerabilities, right? Like we, based on our perception of the world, we know that like a bird belongs in a tree, right? But then if I put like a car in a tree, we have the sort of common sense to reason that like, based on all our experience and all of our modalities of existence, and we know that that's odd. And that's, that's not quite right. But that's, that's not what, how machine learning and AI systems typically work. They would just know, hey, there's a tree here and there's a car here. Yeah. And you know, that can create an issue.

• 33:50Chris Benson?

  If it's not a whopping willow, it should not have a car in it.

• 33:52Daniel Whitenack?

  Right. That is definitely true. Okay. Well, I'm really interested in this idea of grounding. I think, I don't know if you remember back when we talked with Jeff Adams, who was the, he was on the team or helped lead the team that created a lot of speech technology at Amazon related to Alexa and

• 34:14Chris Benson?

  Yeah, I do.

• 34:15Daniel Whitenack?

  and other things. So. Talked about what he was really excited by in the future was thinking more holistically about language that that's particularly what he's interested. But when he said that, what he kind of expressed was that we experience language in all sorts of modalities. So language models and speech technology moving forward is going to be very multimodal, was kind of how he was thinking about that. And that was something really exciting to him. So yeah, I think this is something that I'm really excited about. And yeah, this, this idea of grounding is, is really interesting.

• 34:52Chris Benson?

  It is. It'll be interesting to see how it develops. I know that in the past, as we've hit interesting ideas, we tend to have multiple shows than include them in the discussion. So I suspect that grounding is, is one of the new things and probably foundation is another one. Though I might ask our, our larger community, please be better at naming because these are, we've spent a good bit of the show defining some confusing terms as you come into it, until you get used to it.

• 35:18Daniel Whitenack?

  Yeah. So I would say you and I are only just even touching on this idea of foundation models and the other implications of them. Just to give listeners a, a sense of this, there's other things that are discussed in the work from Stanford, their report, including what I thought was a really interesting one around misuse. So we talked about kind of the vulnerability of these foundation models, one being that there's sort of a single point of failure. And then we've talked about this idea of grounding where maybe these foundation models don't always have the common sense that they should, or aren't grounded in multiple

• 35:58Daniel Whitenack?

  modalities of how we experience the world. So that's one. But then there's this idea of misuse, which means that, and I thought maybe you would find this one intriguing, Chris, particularly that the idea that You can create a really flexible, extremely powerful model, which lets you do downstream things, a whole variety of downstream things much easier. Means that you can do a whole variety of downstream things much easier, right? So in other words, you can do named entity recognition, sentiment analysis, other things. But you can also do other things like create fake news or target marginalized groups. Or you know, whatever that you can do those things much easier too.

• 36:42Daniel Whitenack?

  So, you know, I think that's a, that's a difficult one. Like on the one side, we're creating these large language models that we would like to be more accessible to people. Because we don't want them sort of the concentration of power around these foundation models being with just a few kind of entities. But the wider you distribute these, maybe the more possibility there is for misuse in the community. Which is, I think, why we saw OpenAI kind of restrict usage of GPT-3 for so long. So I don't know, I don't know what the right balance is there, or if there is a right balance.

• 37:18Chris Benson?

  It's interesting, I think that you, you know, when we, a while back, you know, way back early in the show, there were really only data scientists. And then as, as this industry has matured and we've gotten many different roles with different names. We've got deep learning engineers, we've got all sorts of different names now. I think that you just pointed out that there's yet another one there that is coming to being. And that is, you know, like the foundation model, security analyst, you know, who is a person charged with evaluating downstream intended consequences by a malicious actor. So there's probably a whole cybersecurity industry that you just coined just now, Daniel.

• 37:57Daniel Whitenack?

  Well, I'll, I'll take credit for it.

• 37:59Chris Benson?

  There you go, it's yours. It's a resumeable thing, my friend.

• 38:02Daniel Whitenack?

  Yeah, it's like coming up with like voice first or

• 38:06Chris Benson?

  That's right.

• 38:06Daniel Whitenack?

  Yeah, some of those other things.

• 38:08Chris Benson?

  Yeah, they'll call you Daniel father of... dot, dot, dot.

• 38:11Daniel Whitenack?

  Father of the foundation model security analyst.

• 38:15Chris Benson?

  That's right.

• 38:16Daniel Whitenack?

  Yeah, I'll take that. So as we finish up here, just to tie in a learning resource to this whole discussion of foundation models, I don't really know a better one than this, but I have mentioned it before and it keeps, you know, growing and I think becoming better is the Hugging Face course. So if you just look up Hugging Face course, and of course, we'll mention it on our show notes. Hugging Face, maybe people think it's just natural language processing, but it has grown far beyond that. And really, like, if you want to get a sense of like, what is it like to use foundation models and import them and fine tune for a downstream task, fine tune a pre-trained model, or use these big data sets or portions of these

• 39:01Daniel Whitenack?

  data sets that are being used in foundation models. Hugging Face and the model hub, the data set hub and their, you know, training API is really the best way to start into that topic, at least in my opinion. And they have their own course that includes videos. And, you know, just looking at the thing here, they have already on the left-hand side, they talk about, you know, give an introduction, they talk about transformers, which is their library, but you can access all sorts of model architectures. And then they talk about fine tuning, a pre-trained model, sharing models and tokenizers. These are all sort of themes that we've talked about in this, in this episode. So

• 39:42Chris Benson?

  Indeed.

• 39:43Daniel Whitenack?

  Definitely take, take a look.

• 39:44Chris Benson?

  I feel the need to tell people that you, you're not paid to do that because you've offered them up as a fantastic learning resource, which they are a number of times. So if our listeners haven't gone there, they really, really should.

• 39:56Daniel Whitenack?

  They really should. Yeah. And they're, uh, they continue to just boost our own work at S I L. So yeah, I don't feel bad at all about, about giving them another plug.

• 40:06Chris Benson?

  When you're good, you're good, you know, and they're good.

• 40:08Daniel Whitenack?

  Yeah, I know. Well, Chris, I appreciate you talking through the subject with me. It's been fun.

• 40:13Chris Benson?

  Thanks for teaching me.

• 40:15Daniel Whitenack?

  Yeah, on to the next. Hope you have a good evening and, uh, and rest of the week.

• 40:19Chris Benson?

  Sounds good. See you next time.

• 40:20Daniel Whitenack?

  Bye.

• 40:24Speaker 01

  All right, that is Practical AI for this week. If this is your first time listening, subscribe now at practicalai.fm, or just search for Practical AI in your favorite podcast app. We're in there. And if you're a longtime listener, please do share the show with your friends. It is the best way you can help Practical AI succeed. Thanks again to Fastly for shipping our shows super fast all around the world, to Brake Master Cylinder for the Beats, and to you for listening. We appreciate you. That's all for this week. We'll talk to you again next time.