[Practical AI] AI Trends: a Latent Space x Practical AI crossover pod!

Part 2 of our podcast feed swap weekend! Check out Cognitive Revolution https://www.latent.space/p/cogrev-tinystories#details as well. "Data" Dan Whitenack has been co-host of the Practical AI podcast for the past 5 years, covering full journey of the modern AI wave post Transformers. He joined us in studio to talk about their origin story and highlight key learnings from past episodes, riff on the AI trends we are all seeing as AI practitioner-podcasters, and his passion for low-resource-everything! Subscribe on the Changelog https://practicalai.fm/227, RSS https://practicalai.fm/rss, Apple Podcasts https://practicalai.fm/apple, Twitter https://twitter.com/PracticalAIFM, Mastodon https://changelog.social/@practicalai, and wherever fine podcasts are sold! Show notes * Daniel Whitenack – Twitter https://twitter.com/dwhitena, GitHub https://github.com/dwhitena, Website https://www.datadan.io/ * Featured Latent Space episodes: * Benchmarks https://www.latent.space/p/benchmarks-101#details * Reza Shabani https://www.latent.space/p/reza-shabani#details * MosaicML and MPT https://www.latent.space/p/mosaic-mpt-7b#details * Segment Anything https://www.latent.space/p/segment-anything-roboflow#details * Mike Conover https://www.latent.space/p/mike-conover#details * Featured Practical AI episodes: * From notebooks to Netflix scale with Metaflow https://changelog.com/practicalai/150 * Capabilities of LLMs 🤯 https://changelog.com/practicalai/219 * ML at small organizations https://changelog.com/practicalai/207 * Prediction Guard https://www.predictionguard.com/ * Data Dan https://datadan.io/ Timestamps * 00:00 https://changelog.com/practicalai/227#t=0 Welcome to Practical AI * 01:16 https://changelog.com/practicalai/227#t=76 Latent Space Podcast * 04:00 https://changelog.com/practicalai/227#t=240 Practical AI Podcast * 06:20 https://changelog.com/practicalai/227#t=380 Prediction Guard * 08:05 https://changelog.com/practicalai/227#t=485 Daniel's favorite episodes * 10:21 https://changelog.com/practicalai/227#t=621 Alessio's favorite episode * 10:54 https://changelog.com/practicalai/227#t=654 Swyx's favorite episode * 12:44 https://changelog.com/practicalai/227#t=764 Listener favorites * 15:14 https://changelog.com/practicalai/227#t=914 LLMOps * 17:06 https://changelog.com/practicalai/227#t=1026 Reza Shabani * 19:06 https://changelog.com/practicalai/227#t=1146 Benchmarks 101 * 20:06 https://changelog.com/practicalai/227#t=1206 Roboflow * 21:38 https://changelog.com/practicalai/227#t=1298 Mode collapse * 26:21 https://changelog.com/practicalai/227#t=1581 Rajiv Shah * 28:01 https://changelog.com/practicalai/227#t=1681 Staying on top of things * 33:11 https://changelog.com/practicalai/227#t=1991 Kirsten Lum * 34:31 https://changelog.com/practicalai/227#t=2071 datadan.io * 38:48 https://changelog.com/practicalai/227#t=2328 Prompt engineering * 40:38 https://changelog.com/practicalai/227#t=2438 Unique challenges engineers face * 42:51 https://changelog.com/practicalai/227#t=2571 AI-UX * 45:31 https://changelog.com/practicalai/227#t=2731 NLP data sets * 50:49 https://changelog.com/practicalai/227#t=3049 Unlabeled data sets * 55:07 https://changelog.com/practicalai/227#t=3307 Lightning round! * 55:20 https://changelog.com/practicalai/227#t=3320 What's already happened in AI? * 56:27 https://changelog.com/practicalai/227#t=3387 Unsolved questions in AI * 58:01 https://changelog.com/practicalai/227#t=3481 Get hands on * 58:53 https://changelog.com/practicalai/227#t=3533 Outro Transcript Full transcript is over at the Changelog site https://changelog.com/practicalai/227#transcript! This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe https://www.latent.space/subscribe?utm_medium=podcast&utm_campaign=CTA_2

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Hello again, it's Swyx back again with the second of our crossover episodes. This time with the Practical AI folks who have been covering the AI space since before it was cool, as we often like to say. Something that Alessio and I are pretty mindful of every time we take to the mic is that we're pretty new podcasters to this space and there is a longer history and heritage behind all this, as well as a lot of areas and niches that we'll probably never even touch. And so I always like recommending these podcasts with a much longer backlog that actually give you a way to travel back in time for kind of an oral history of AI, which is very rare and very valuable.

• 0:43Speaker 00

  These are tokens that are generated in real time, just like we're doing it now for this current wave of AI trends. And so what we did, podcaster to podcaster, I always love talking to other podcasters because they know what's up, is we went through our respective stats and picked out the crowd favorite episodes as well as some personal picks. I do have to note that this was recorded just before our George Tots episode, which obviously is now our top downloaded episode across podcasts, as well as our newly launched YouTube. Go subscribe, please, if you haven't seen it. But anyway, if you're looking for good AI podcasts, these guys have been covering it for a long time, very, very consistently, and covered a lot of topics. Definitely check them out and please enjoy our special crossover episode with Practical AI in the studio.

• 1:32Speaker 01

  Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related technologies are changing the world, this is the show for you. Thank you to our partners at Fastly for shipping all of our pods super fast to wherever you listen. Check them out at fastly.com. And to our friends at Fly, deploy your app servers and database close to your users, no ops required. Learn more at fly.io.

• 2:07Speaker 02

  Well, hello. We have a very special episode for you today. I got the chance to sit down with the guys from Latent Space, Wix and Alessio out in San Francisco. They were kind enough to let me into their podcast recording studio, and we got a chance to talk about our favorite episodes of both of our shows and some of the overall takeaways we've had from those discussions. We cover some of the trends that we've been seeing in AI, and they even get a chance to grill me on my opinions about prompt engineering. So enjoy the show.

• 2:41Alessio Fanelli?

  Hey, everyone. Welcome to the Late in Space podcast. This is Alessio, partner and CTO in residence at Decibel Partners. I'm joined by my co-host, Swyx, writer and editor of Late in Space.

• 2:52Speaker 00

  And today, we're very excited to welcome Dan Whitenack to the studio. Welcome, Dan.

• 2:56Speaker 02

  What's up, guys? It's great to be here.

• 2:58Speaker 00

  This is a podcast crossover. If you recognize this voice, Dan is the host of Practical FM. He's been in my ear on and off for the past five years, covering the latest and greatest in AI, before it was cool.

• 3:08Speaker 02

  Yeah, yeah, yeah. Before the AI hype back in these weird data science times, whatever that is now.

• 3:15Speaker 00

  Yes, everything is merging and converging. So I'll give a little bit of your background, and we can go into a little bit on your personal side. You got your PhD in mathematical and computational physics, and then you spent 10 years as a data scientist, most recently at S I L International, which I actually, I thought it was like an agri-tech thing, and then I went to the website. It's actually a nonprofit.

• 3:35Speaker 02

  International NGO, yeah. So they do language-related work all around the world. So I spent the last five years building up a team that's been working on low-resource scenarios for AI, if people are familiar with that. So doing machine translation or speech recognition, that sort of thing, in languages that aren't yet supported.

• 3:53Speaker 00

  Yeah, yeah, we'll talk about this later, but I think episode three on Practical AI was already featuring the global community that AI has and addresses.

• 4:02Speaker 02

  Yeah, yeah, yeah. It's been an important theme throughout the whole time, throughout over 200 episodes.

• 4:07Speaker 00

  Yeah, yeah. And you recently left S I L to work on Prediction Guard, which we can talk a little bit more about that. You are also interim Senior Operations Development Director at T. Candle Co.

• 4:18Speaker 02

  Yeah, yeah.

• 4:19Speaker 00

  And yeah, what else should people know about you?

• 4:21Speaker 02

  Yeah, I mean, I, as probably can be noted from the intro, I love working on various projects and having my hands in a lot of things. But yeah, I've code on the side for fun, and that's how I usually get into these side projects and that sort of thing. But outside of that, yeah, I live in Indiana. I was telling you guys that I'm trying to coin the term cerebral prairie. So we'll see if that catches on. Probably not.

• 4:49Speaker 00

  Well, you're a second guest in a row from Indiana. Linus from Notion was Indiana. We're talking about how there's a surprising number of international students in there.

• 4:58Speaker 02

  Yes, very true.

• 4:59Speaker 00

  Purdue's a strong university.

• 5:01Speaker 02

  Yeah, yeah, very strong university. It's a great place to spend time. And there's a lot of fun things that happen around that area, too. So I'm also very into music, but not any sort of popular music. I play like mandolin and banjo and guitar and play folk music.

• 5:16Speaker 00

  Low resource.

• 5:17Speaker 02

  Yeah, low resource music, low resource languages. Yeah, all those things, anything low resources in my, is in my territory for sure.

• 5:25Speaker 00

  And maybe we can cover the story of Practical AI. How'd you start it? Tell us what the early days were like and just fill everyone in.

• 5:31Speaker 02

  Yeah, it was kind of a winding journey. Some people might be familiar with the Change Log podcast, which I think they've been going now for like 11 or 12 years. It's pretty prolific in around, I think originally around more open source. Now it's kind of software development in general, but they have a network of podcasts now. And at Go Conference, actually, so I'm a fan of the Go programming language. That's another fun fact. But at GopherCon, I think it was in 2016, maybe?

• 6:03Speaker 02

  I met Adam Stokoviac, who is one of the hosts of the Change Log. At the time, I was giving a talk about data science, something, I forget. But he kind of pitched me, he's like, we've been thinking a lot about doing like a data or data science podcast. And at the time, he had a name, it was like, I think, I think it was Hard Data, or something like that, which, which never caught on for obvious reasons. But I kind of stored that away. It didn't really do anything with it. But then, over the next couple of years, I met Chris Benson, who's my co-host on Practical AI, and helped him with a couple of talks at conferences. We met through the Go community as well. And eventually, he's a, he was working at a different company at the time. Now he's a strategist

• 6:46Speaker 02

  with Lockheed Martin, working in AI stuff. But he reached out to me and said, hey, would you ever consider doing kind of like a co-host podcast thing? And at that point, I remembered my conversation with Adam. So I reached back out to Adam with the Change Log. And then we kind of started working on the idea. We wanted it to be practical. So at the time, well, there's a lot of people doing things now with AI, like hands-on. Back then, there were kind of some podcasts that were really hyped AI, like not practical at all, which is why we kind of came to Practical AI, something that would actually benefit people. And that, that's like a great thing to hear from people that when they listen to the show, they do actually learn something that's useful for their day to day. That's kind of the goal. Yeah.

• 7:30Alessio Fanelli?

  Nice. And I think that's one of the things in common with our podcast, you know, there's a lot of content out there that can get a lot of clicks with a fear of AI, you know, and all these different things. And I think we're all focused on more, yeah, practical and day-to-day usage. Yeah. Tell us more about Prediction Guard, you know, that kind of fits into making AI practical and usable.

• 7:50Speaker 02

  Yeah, yeah, sure. Appreciate that. So yeah, Prediction Guard is what I've been working on since about Christmas time-ish. Originally, I was thinking a lot about large language model evaluation and model selection, but it's kind of morphed into something else. What I've realized is that there's this market pressure, there's internal company pressure for people to implement

• 8:13Speaker 02

  Kind of generative AI version of models into their workflows because enterprises realize the benefits that they could have. But in practice, when they go and they go from like chat GPT, they type in something, it's amazing. And then like, how do we do this in our enterprise where we have maybe rules around data privacy or compliance issues? And also like, we want to automate things, maybe, or we want to do data extraction. But I just get like text vomit out of these models. Like, what do I do with that? I just get unstructured text. How do I build a robust system out of inconsistent text vomit? So Prediction Guard is really focused on those two things. One is kind of compliance and running kind of state of the art AI models in a compliant way.

• 8:58Speaker 02

  And then layering on top of that, layers of control for structuring output and validating output. So some people might be familiar with projects like guardrails or guidance or these things. So we've integrated kind of some of the best of those things into the platform. Plus some ways to easily do like self-consistency checks and factuality checks and other things on top of large language model output.

• 9:22Alessio Fanelli?

  Nice. We did have a Shriya Rajpapal from Gorillas as a guest.

• 9:25Speaker 02

  Yeah, yeah.

• 9:26Alessio Fanelli?

  So yeah, that's another episode that people really like. Yeah, maybe, you know, just to give people a sense of what practical AI is as a podcast. You want to talk about maybe like the two, three favorite episodes that we have and we can go maybe alternate.

• 9:40Speaker 02

  Back and forth.

• 9:40Alessio Fanelli?

  You know, like our favorites.

• 9:41Speaker 02

  We've done some prep for this episode. Yes, yes.

• 9:44Speaker 00

  This is kind of like, I think our conception of this is kind of like a review for listeners who are new to us, either of our podcasts, to go back and revisit the favorites.

• 9:53Speaker 02

  Yeah, yeah, I think I can talk about some personal favorites of mine and then maybe like favorites from the audience. I think some of my personal favorites have actually been, we call them like fully connected episodes, where Chris and I actually talk through a subject in detail together without a guest. To be honest, those are great episodes just for like me to learn something, like have an excuse to learn something. And we've done that recently, like with chat GPT and instruction tune models. We did it with stable diffusion and diffusion models. We did it with AlphaFold. So all of those are episodes with us too. And just talking through like, how practically can you like form a mental model for how these models were trained?

• 10:38Speaker 02

  And how they work and what they output? Those are some of my favorites, just because I learn a lot, because I do a little bit of prep. We talk through all the details of those and it helps me form my own sort of intuition around those things. Another personal favorite for us was that we did a series about AI in Africa. That was really cool. You mentioned like the global AI community. We did actually a series of those. They're all labeled AI for Africa, highlighting things like Masakane. So people don't realize that like some of the models that we develop here, like in the West Coast or wherever, they don't work great for all use cases around the world. And there's a lot of thriving grassroots communities like Masakane and Turkic Interlingua

• 11:22Speaker 02

  and other communities that are really building models for themselves. Machine translation, speech recognition, models that work for their languages around the world or agriculture, you know, computer vision models that work for their use cases around the world. So those are a couple of highlights on my end.

• 11:41Speaker 00

  Do we go with our personal highlights? Yeah, go ahead. I think you already picked one out.

• 11:46Alessio Fanelli?

  Yeah, I think mine is definitely the episode with Mike Conover from Databricks, who's the person leading the Dahlia first there. I think obviously the content is great and Mike is extremely smart and prepared. But I think the the passion that he had about these things, you know, the red pajama data set came out the morning that we recorded. And we're all kind of like nerding out or like, yeah, why is that so interesting? Like, he was so excited about it and it's great to see people that have so much excitement about things that they work on. You know, it's kind of like an inspiration in a way to do the same for us.

• 12:19Speaker 00

  I think personally, so I tend to drive the news-driven episode ones, like the event-driven ones, where something will happen in AI and like I'll make a snap decision that we'll have an episode recording on Twitter Spaces and we'll have just a bunch of people tune in. I think the one that stood out was the ChatGPT App Store, the ChatGPT plugins release, where like 4,000 people tuned in to that one.

• 12:40Speaker 02

  That's crazy.

• 12:41Speaker 00

  And we did like an hour of prep, right? And I think it's important for me as a quote-unquote journalist to be the first to report on something major and to provide a perspective of something major, but also capture an audio history of how people react at the time. Because this is something that we're talking about in the prep, ChatGPT plugins have become a disappointment compared to our expectations then. But we captured it. We captured the excitement back then and we can consider compare and contrast like where we thought things were going and where things have actually ended up. It's a really nice piece of, I guess, audio journalism.

• 13:16Speaker 02

  Yeah, yeah. I mean, it was just last year. I mentioned Stable Diffusion and all that. We were talking about this. It was like I always had in my mind, oh, everything's going to image generation. Like, should I quit doing NLP and start thinking about image? And now all I do is NLP and language models. But at the time, that was, you know, that's what was on our mind.

• 13:37Speaker 00

  Same thing. I was working on web UI for Stable Diffusion, just like a thousand other front-end developers were. And yesterday was the first time I opened Stable Diffusion in six months.

• 13:48Speaker 02

  And a lot has changed and it's still an area that's developing, but it's not, yeah, it's not driving thought process at the moment.

• 13:56Speaker 00

  Yeah. Well, especially because I think just it depends on what you're, what you think you want to do. And I'm definitely in less visual and more for a text-driven person. So I naturally lean towards LLMs anyway, like NLP.

• 14:09Speaker 02

  Yeah, I can hit some listener favorites. Yeah, crowd favorites. So we have like one clear favorite, which is actually, I would say it's a surprise to me. Not because the guest wasn't good or anything, but just the, so the topic was Metaflow. So I don't know if you've heard of Metaflow. It's a Python package for kind of full stack data science modeling work developed at Netflix. And we had Villa Tullos on, who was the creator of that package. And that has had so, it's like maybe a 30% more listens than any other episode.

• 14:48Speaker 02

  And I think the title, so we titled it from, I think, from notebooks to production or something like that. Yeah. So it's like this idea of from notebooks to production. There's all sorts of things that prevent you from getting the value out of these sorts of methodologies. And my guess would be that talking about that is probably like the key feature of that episode. And Metaflow is like really cool. People should check it out. It is one way to kind of do this, both versioning and orchestration and deployment and all of these things that are really important.

• 15:26Speaker 02

  But I think a takeaway for me was like practically bringing into the, some people might call it like full stack data science or like model lifecycle things, like the model lifecycle things interest people so much. So beyond making like a single inference or beyond doing like a single fine tuning, what is the lifecycle around a machine learning or an AI project? I think that really fascinates people because it's like the struggle of everyday life in actual practical usage of these models. So it's one thing to go to hugging face, try out like a hugging face space and like create some cool output, or even just pull down a model and get output.

• 16:10Speaker 02

  But how do I handle like model versioning and orchestration of that in my own infrastructure? How do I tie in my own data set to that and do it in a way that like is fairly robust? How do I, you know, take these data scientists who like use all this weird tooling and like mash them into an organization that deals with like DevOps and like non-AI software and all of that? I think those are questions people are just wrestling with all the time.

• 16:38Speaker 00

  Yeah, it feels a little bit in conflict with the trends of foundation models where you the primary appeal is you train once.

• 16:47Speaker 02

  Yeah.

• 16:47Speaker 00

  And then you never touch it again, or you'd release it as a version and people kind of just prompts based off of that. And I feel like this, I feel this evolution moving from essentially the ML Ops era into

• 16:58Speaker 02

  Yeah.

• 16:59Speaker 00

  for lack of a better word, LLM Ops.

• 17:01Speaker 02

  Yeah.

• 17:01Speaker 00

  How do you feel about that?

• 17:02Speaker 02

  No, I think you're, I think you're completely right. I think there will always be a place for like these models in organizations that are task-specific models, like scikit-learn models or whatever that solve a particular problem, because organizations like finance organizations or whatever will always have like a need for explainability or whatever it might be. But I do think we're moving into a period where like I've had to rebuild a lot of my own intuition as a data scientist from thinking about gather my data, create like my code for training, output my model, serialize it, push it to some hub or something,

• 17:42Speaker 02

  deploy it, you know, handle orchestration to now thinking about, okay, which of these pre-trained models do I select and how do I engineer my prompting and my chain? Maybe going to fine tuning, like that is still like a really relevant topic. But some of these things that like I've been working on with Prediction Guard, I think are the things that have a parallel in ML Ops, but they're slightly like, there's just a slightly different flavor. I think it's how ML Ops is graduating to something else versus like people are still concerned about Ops. It's just like you say, it's a kind of a different kind of Ops.

• 18:22Alessio Fanelli?

  Yeah. And I think that's reflected in our most popular episodes too. So I think all three of our most popular episodes are model-based. They're not more like infrastructure-based. So I think number one is the one with Reza Shabani, where we talked about how to train the the Replit code model and the AmJUD vibes that they used to figure out whether or not the model was good. And I think, you know, that makes sense for our community. It's mostly software engineers and AI engineers. So code models are obviously a hot topic.

• 18:49Speaker 02

  Yeah.

• 18:50Alessio Fanelli?

  Yeah, that that was really good. And I think like it was one of the first times where we kind of went beyond just listening traditional benchmarks, you know, which is why we did a whole thing about AmJUD eval. It's like a lot of companies are using these models and they're using off-the-shelf benchmarks to do it. And what You know, another episodes that we'll talk about is like the one with Jonathan Franco from Mosaic ML. And he also mentioned a lot of the benchmarks are multiple choice, but most production workloads are like open-ended text generation questions. So how do you kind of reconcile the two?

• 19:22Speaker 02

  Yeah. Did you all get into at all, you know, the whole space of LLMs evaluating LLMs? And sort of, this was something on a recent episode, we talked to Jerry from Lama Index about in terms of, on the one hand, generating questions like you're talking about to evaluate LLMs or using an LLM to look at the context and a response and provide an evaluation. I think that's definitely something that I think is interesting and has come up in a few of our episodes recently, where people are struggling to evaluate these things. And so, yeah, we've seen a similar trend in one direction, thinking about benchmarks, and in another direction, thinking about this sort of on-the-fly or model-based evaluation,

• 20:06Speaker 02

  which has existed for some time. Like in machine translation, it's very common. So like UnBabel uses a model called Comet. That's like one of the most popular, highest-performing machine translation evaluators as a model. It's not a metric and that sort of thing like blue. So yeah, that's a trend that we've seen is evaluation and specifically evaluation for LLMs, which can kind of get dicey.

• 20:31Alessio Fanelli?

  Yeah, we did a benchmarks 101 episode that is also well-liked. And we talked about this concept of like a benchmark-driven development, you know, like the benchmarks used to evolve every like three, four years. And now the models are catching up every like six months. So there's kind of this race between the benchmarks creators and like the models developers to find, okay, the state-of-the-art benchmarks is here and GPT-4 on a lot of them gets like, you know, nine-kid percentile results. So, you know, GPT-4 is not a G I. Therefore, to get to a G I, we need better evals for these models to start pushing the boundaries. And yeah, I think a lot of people are experimenting with using models to generate these things, but I don't think there's a clear answer.

• 21:15Speaker 02

  Yeah.

• 21:15Speaker 00

  Something that I think we were quite surprised to find was specifically in Hello Swag, where the benchmarks, instead of being manually generated, were adversarially generated. And then I was very interested in our, I mean, this is kind of like segueing, we're not, we're not really going in sequence here. Segueing into our second most popular episode, which was on RoboFlow, which covered segment anything from Meta. I think you guys had a, had a discussion about that too.

• 21:41Speaker 02

  Yeah, it's been mentioned on the show. I don't think we've had a show devoted to it, but.

• 21:45Speaker 00

  Well, the most surprising finding when you read the paper is that something like less than 1% of the data of the mass that they released were actually human generated. A lot of it was AI assisted. So you have essentially models evaluating models and the models are trained, themselves trained on model-generated data. Very few, we're very many layers in at this point.

• 22:08Speaker 02

  Yeah, yeah. And I know that there's been a few papers recently about the sort of things that were done with Llama and other models around model-generated output and data sets. It'll be interesting to see. I think it's still early days for that. So I think at the very minimum, what all of these cases show is that models, either evaluating models or using simulated data. I think back a few years ago, we would probably call this simulated data, right? I don't think that term is quite as popular now.

• 22:40Speaker 00

  Augmented?

• 22:40Speaker 02

  Yeah, or augmentation, data augmentation, simulated data. So I think like this has been a topic for some time, but the scale at which we're seeing this done is, is kind of shocking now. And, and encouraging that like we can do quite flexible things by combining models together, both at inference time, but also for training purposes.

• 23:02Speaker 00

  Well, have you ever come across this term of mode collapse? What I fear is, especially as someone who cares about low resource stuff, is that stacking models on top of models on top of models, you just optimize for the median use case or the modal use case.

• 23:16Speaker 02

  Yeah, yeah, I think that one maybe. So yeah, that is a concern. I would say it's a valid concern. I do think that these sort of larger models, and this gets, I guess, more into like multilingualism and the makeup of various data sets of these LLMs. The more that we can have linguistic diversity represented in these LLMs, which I know, I think Cohere for AI just announced like a community-driven effort to increase multilinguality in LLM data sets. But I think the more we do that, I think it does benefit the downstream, lower resource languages and

• 23:56Speaker 02

  lower resource scenarios more because we can still do fine tuning. I mean, we all love to use pre-trained models now. But like in my previous work, when you were looking at maybe an Arabic vernacular language rather than standard Arabic, there's so much standard Arabic in data sets. Making that leap to an Arabic vernacular is much, much easier if that Arabic is included in LLM data sets because you can fine tune from those. So that is encouraging that that can happen more and more. There's still some major challenges there. And especially because most of the content that's being generated out of models is not in, you know, Central Siberian

• 24:39Speaker 02

  UPIC or one of these languages, right? So we can't purely rely on those. But I think my hope would be that the larger foundation models see more linguistic diversity over time. And then there's these sort of grassroots organizations, grassroots efforts like Masakane and others that rise up kind of on the other end and say, okay, well, we'll, we'll work with our language community to develop a data set that can fine tune off of these models. And hopefully, there's, there's benefit both ways in that sense. Yeah.

• 25:11Speaker 00

  Since you mentioned Masakane a couple of times, we'll drop the link in the show notes so people can find it. But what exactly do they do? How big of an impact have they had?

• 25:20Speaker 02

  Yeah, I would say, so if people aren't familiar, if, yeah, if you go to the link, you'll see it. They talk about themselves as a grassroots organizations of African NLP researchers creating technology for Africa. So we have our own kind of biases as people in an English-driven sort of literate world of what technology would be useful for everyone else. Like, it probably makes sense for maybe listeners to say, well, wouldn't it be great if we could translate Wikipedia into all languages? Well, maybe, but actually, the reality on the ground is that many language communities don't want Wikipedia translated

• 26:02Speaker 02

  into their language. That's not how they use their language. Or they're not literate, and they're in oral culture, so they need speech, right? Text won't do them any good. So that's why Masakane has started as a sort of grassroots organization of NLP practitioners who understand the context of the domain that they work in, and are able to create models and systems that work in those contexts. There's others, you can hear them on like the AI for Africa episodes that we have that talk about like agriculture use cases. Agriculture use cases in the US might look like, you know, John Deere tractor with a cam, like, I don't know if people know this, but like John Deere tractors are these big

• 26:46Speaker 02

  tractors. They literally, they have a Kubernetes cluster on, like, some of them have a Kubernetes cluster on the tractor. It's like a at-the-edge Kubernetes cluster that runs these models. And like, when you're laying down pesticide, there's cameras that will actually identify and spray like individual weeds, rather than like spraying the whole field. So that's like at the level that, you know, maybe is useful here. In Africa, maybe the more useful thing is around disease or drought identification, or disaster relief, or other things like that. And so- There's people working in those environments or in those domains that know those domains that are producing technology for those cases. And I think that's really important.

• 27:30Speaker 02

  So yeah, I'd encourage people to check out Masakane. And there's other groups like that. And if you're in like the US or Europe or wherever, and you want to get involved, there's open arms to say, hey, come help us do these things. So yeah, get involved too.

• 27:46Speaker 00

  What else is in your top three?

• 27:47Speaker 02

  Oh, yeah. So one recent one from Raj Shah from Hugging Face. Some people might have seen his really cool videos on LinkedIn or other places. He makes TikTok videos about AI models, which is awesome. And his episode is called "The Capabilities of LLMs." And I thought it was really a good way to help me understand like the landscape of large language models and the various features or axes that they're kind of situated in. So one axis is, for example, closed or open, right? Can I download the model? But then on top of that, there's another, there's another axis, which is,

• 28:31Speaker 02

  is it available for commercial use or is it not? And then there's other axes, like we already talked about multilinguality, but then there's like task specificity, right? Like there's code gen models and there's language generation models. And there's of course, image generation models and all of those as well. So yeah, I think that episode really helps set a good foundation, no pun intended, for language models to understand where they're situated. So you can kind of, when you go to Hugging Face and there's, what is there like 200,000 models now? Maybe there's, I don't know how many models there are. How do I like navigate that space and understand what I could pull down? Or do I fit into one of those use cases where it makes sense for me to just connect to OpenAI

• 29:16Speaker 02

  or Coher or Anthropic? Helps kind of situate yourself. So I think that's why that episode was so popular is he kind of lays all of that out in an understandable way.

• 29:26Speaker 00

  How do you personally stay on top of models? You know, there's leaderboards, there's Twitter, there's LinkedIn.

• 29:31Speaker 02

  Yeah, I think it's a little bit spread out for me between the sources that you've mentioned as podcasters. I think that's one of the...

• 29:40Speaker 00

  Yeah, it's our job.

• 29:41Speaker 02

  Yeah, it's one of... Well, it's also a benefit for us. I think like if I didn't have every week on Wednesday, like I'm gonna talk about this topic, whether like I'm planning to think about, think about a certain thing or not, it kind of helps you prompt and look, look at what's going on. So I think that is an advantage of like content creators is it is kind of a responsibility, but it's also an advantage that we can have to like have the excuse to have great conversations with people every week. But yeah, I think Twitter's a little bit weird now as, as everybody knows, but it's still a good place to find out that information. And then sometimes too, like to be honest, I go to Hugging Face

• 30:24Speaker 02

  and like I'll search for models, but I also search and I look at the statistics around the downloads of models because generally when people find something useful, then they'll download it and download it over and over. So sometimes when I hear about like a family of models, I'll go there and then I'll look at some of the statistics on Hugging Face and like try a few things.

• 30:45Speaker 00

  Yeah. And some of these forks, I see the download numbers, but I've never heard of them outside of Hugging Face.

• 30:51Speaker 02

  Yeah, it's true. It's true. Yeah. And some of them, like there'll be a fork or like a fine tune or something. And it's a, you do have to do a little bit of digging around like licensing and that sort of thing too. But it is a useful, like there's tons of people doing amazing stuff out there that aren't getting recognized at the like, you know, Falcon or MPT level. But there's a lot of people doing cool stuff that are releasing models on Hugging Face, maybe that they've just found interesting.

• 31:20Speaker 00

  Any unusual ones that you recently found?

• 31:22Speaker 02

  Well, there's one that I'll I'll highlight, which I thought was cool because I don't know if you you all saw the Meta released this.

• 31:31Speaker 00

  The six six modality model.

• 31:33Speaker 02

  Yeah, yeah. And it was interesting because we did this work with Masakane. When I was at S I L, we did this work with Masakane and Koki, which is a speech tech company to create these language models in like six African languages. And I was like, OK, that's that's cool. Like, we did that. We form like the data sets. It was satisfying. But now I'm like learning that then Meta went and found that data on on Hugging Face. And that's kind of incorporated in these these new models that Meta has released. So it's cool to see like the full cycle thing happen where there was grassroots organizations seeing a need for models,

• 32:17Speaker 02

  gathering data, doing baselines, and now there's like extended functionality in kind of like a more influential way, I guess, at like that higher level. Yeah.

• 32:28Alessio Fanelli?

  Yeah, I think, I mean, talking about open and and close models, when we started the podcast, it kind of looked like a cathedral kind of market where we had Coher, Anthropic, OpenAI, Stability, and those were like the hottest companies. I think now, you know, as you mentioned, you go on Hugging Face, like I just opened there right now. There's this others news research 13 billion parameters model that just got released, fine tune on over 300,000 instructions. It's like the models are just popping up everywhere, which is which is great. And yeah, we had an episode with, as I mentioned, with Jonathan Frankel and Abhinav from Mosaic ML to introduce MPT 7B and some of the work that they've done there.

• 33:09Alessio Fanelli?

  And I think like one of their motivation is like keeping the space as open as possible, like making it easy for anybody to go, obviously, ideally on the on Mosaic ML's platform and Yeah. train their own models and and whatnot. So that's one that people really liked. I thought it was really technical, so I was really a little worried at first. I was like, is it gonna fly over most people's head? But.

• 33:32Speaker 00

  We're going more technical.

• 33:34Alessio Fanelli?

  Exactly. Now, now that was a good learning.

• 33:36Speaker 00

  Leaning in.

• 33:37Alessio Fanelli?

  Exactly.

• 33:39Speaker 00

  And Jonathan is super passionate about open source. He had this rant halfway through the episode about why it's so important to keep models open. And I actually edited in the crowd applause into the podcast, which I kind of love. I love little audio bonuses for people listening along. And I think the Change Log guys do that really well, especially in their newer episodes.

• 34:01Speaker 02

  Yeah, we need to, there is a way for us to integrate some of those things. Yeah, like the soundboard thing. And we've never got into it too much. I need to work with Jared from the Change Log and see.

• 34:11Speaker 00

  It just spices it up.

• 34:12Speaker 02

  Exactly, exactly.

• 34:13Speaker 00

  You can have, you can only have so many hour-long conversations about ML.

• 34:18Speaker 02

  We're, yeah, I keep thinking that, but then we keep going, so.

• 34:23Speaker 00

  Right, right, right, right. Sorry. I didn't mean, I didn't mean like it was like.

• 34:26Speaker 02

  No, no, I got you.

• 34:27Speaker 00

  It just switches it up and makes the audio interesting to add variety. Cool. I don't know, I don't know if there are any other highlights that we want to do for.

• 34:36Speaker 02

  I'll just highlight maybe one more. Kirsten Lum was on, she had an episode about machine learning at small organizations. I think that's a great one. Like if you're a data scientist or a practitioner or an engineer at like either a startup or a midsize company where I think the thing that she emphasized was these different tasks that we think about, like, you know, whether it's curating a data set or training a model or fine tuning a model or deploying a model. Sometimes at like a larger organization, those are functions in and of themselves. But when you're in this sort of mid-range organization, that's like a task you do, right? So to think

• 35:15Speaker 02

  about like those tasks as tasks of your role and time box them and understand like how to do all of those things well without getting sucked down into any one of those things. That was an insight that I found quite useful in my day to day as well as to sort of start to get a little bit of like spidey sense around, hey, I'm spending a lot of time doing this, but which probably means I'm like stuck in too much, like I'm making my ML ops too complicated, right, to track versions and like tie all this stuff together. Maybe I should just like do a simple thing and like paste a number in a Google sheet and move on or something.

• 35:56Alessio Fanelli?

  I think that's a good segue into some of the other work that you do. You run the datadan.io website, which is kind of like a different types of workshop and advising that you do. I think a lot of founders especially are curious about how are companies thinking about using this technology? There's a lot of demos on Twitter, a lot of excitement. But when founders are putting together something that they want to sell, they're like, okay, what are the real problems that enterprises have? What are like some of the limitations that they have? We talked about commercial use cases and something like that. Can you maybe talk a bit about, you know, two, three high level learnings that you had from these workshops? And like how these models are actually being brought into companies and how they're being adopted?

• 36:42Speaker 02

  Yeah, I think maybe one higher level comment on this is even though we see all these demos happening, everybody's using ChatGPT. The reality in enterprise is most enterprises still don't have like LLMs integrated across their technology stack. Right? So that might be a bummer for some people like, oh, it's not quite as pervasive. But I actually find it as refreshing, maybe because some of us feel like stuff happens every week. It's exhausting to keep up like, oh, if I don't keep up with this stuff, then like I'm getting left behind. But it takes time for these things to trickle down. And not everything, like we were talking about the stable diffusion use case and others,

• 37:26Speaker 02

  like not everything that's hyped at the moment will be a part of your like day to day life forever. Right? So you can kind of take some comfort in that. I think it's really important for people to, if they're interested in these models, to really dig into more than just kind of a single prompt into these models. The practical side of using generative text models or LLMs really comes around either what some people might call prompt engineering, but you know, understanding things like giving examples or demonstrations in your prompt, using things like guardrails or or RegEx statements.

• 38:06Speaker 02

  Or prediction guard to structure output, doing like fine tuning for your, your company's data. Like these things go, there's kind of a hierarchy of these things. I think, I think you all know Travis Fisher. He was, he was a guest on, on Practical AI and talked about this hierarchy from prompt engineering through like data augmentation to fine tuning to eventually like training your own generative model. I've really tried to encourage enterprise users and those that I do workshops with to think something like that hierarchy

• 38:47Speaker 02

  with these models, like get hands on, do your prompting, but then like if you don't get the answer that you want immediately, I think there's a tendency for people to say, oh, well, it doesn't work for my use case. But there's so much of a rich environment underneath that with things like Lingchain and Llama Index and you know, data augmentation, chaining, customization, fine tuning, like all this stuff that can be combined together. It's, it's a fun new experience, but I find that enterprise users just haven't explored past that very most shallow level. So I think, yeah, in terms of the trends that I've seen with the workshop, I think people have gone to ChatGPT

• 39:28Speaker 02

  or one of these models, they've seen like the value that's there, but they have a hard time connecting these models to a workflow that they can use to solve problems. Like before we all had intuition, like I'm going to gather my data. It's going to have these five features. I'm going to train my scikit-learn model or whatever. I'm going to deploy it with Flask and like now I have a cool thing. Now all of that intuition has sort of been shattered a little bit. So we need to develop a new workflow around these things. And I think that's really the focus of the workshops is kind of rebuilding that intuition into a practical workflow that you can think through and solve problems with practically.

• 40:12Alessio Fanelli?

  You have a live prompt engineering class? Prompt engineering overrated or underrated?

• 40:19Speaker 02

  Yeah, I think prompt engineering as like a term is probably too hyped. I think engineering and ops around large language models, though, is a real thing. And it is sort of what we're transitioning to. Now, how much you wanna say is like, that term gets used in all sorts of different contexts. It could mean just like, oh, I wrote a good prompt and I'm gonna like sell it on Twitter or something.

• 40:46Speaker 00

  Promptbase, the marketplace of prompts. I wonder how they're doing, to be honest, because they get quoted in almost every article about prompt engineering. They got really, really good PR.

• 40:55Speaker 02

  Yeah, yeah, I mean, if people can sell their prompts, I mean, I'll offer that, that's cool.

• 41:00Speaker 00

  I got prompts right here.

• 41:03Speaker 02

  But I think it goes, like, some people might just mean that. And I think that's maybe overhyped in my view. But I do think there's this whole level of engineering and operations around prompts and chaining and data augmentation that is a real workflow that people can use to solve their problems. And that's more what I mean when I'm referring to like, whatever, however you wanna combine the word engineering with prompting and language models.

• 41:29Speaker 00

  Yeah, I've just been calling it AI engineering.

• 41:31Speaker 02

  AI engineering, that's good.

• 41:32Speaker 00

  Rangle with the AI APIs, know what to do with them. That is a skill set that is developing that is a sub-specialty of software engineering.

• 41:40Speaker 02

  Yeah, yeah.

• 41:41Speaker 00

  It is what it is. And I think part of, something I'm really trying to explore is this, is this spillover of AI from the traditional ML space, like where you need a machine learning researcher or machine learning engineer. It's spilling over into the software engineering space. And there's this rising class of what I'm calling AI engineer that is specialized in, conversant in the research, the tooling, the conversations and themes.

• 42:03Speaker 02

  What do you think are the unique challenges that like someone coming from that ladder group, like engineers that are advancing into this AI engineer position versus like probably more like my background where I was in data science for some time and now I'm kind of like transitioning into this world. What do you, what do you think are the unique challenges for both groups of people?

• 42:25Speaker 00

  Oh, I mean, so I can speak to the software side and you can speak about the data science side. It's simply that we are, for many of us, dealing with the non deterministic system for the first time, that by the way, we don't fully control because there's this conversation about did GPT-4 regress in its quality and we don't know because model drift is not within our control because it's a black box API from, from OpenAI. But beyond that, there's this sense that the latent space of capabilities is not fully explored yet.

• 42:58Speaker 02

  Yeah.

• 42:58Speaker 00

  Right? Like there's 175 billion or 1 trillion parameters in the model. We're maybe using like 200 of them. It's literally where, is that meme where like we're using 10% of our brain. We are, we're probably using 10% of what is capable in the model and it takes some ingenuousness to unlock that.

• 43:17Speaker 02

  Yeah, I think from the data science perspective, there's probably a desire to quickly to jump to these other things around fine tuning or training your own models where if you really do take this prompting, chaining, data augmentation seriously, you can do a lot with models sort of off the shelf and don't need to like jump immediately into training. So I think that is like a knee jerk reaction on our end. And fine tuning is going to be around for the foreseeable future as far as I can tell, but data scientists have maybe a different, because we've been dealing with the uncertainty or non deterministic

• 43:59Speaker 02

  output for some time and have developed some intuition around that. But that's mostly when we've been controlling the data sets, when we've been controlling like the model training and that sort of thing. So to throw some of that out, but still deal with that, it's a separate kind of challenge for us.

• 44:15Speaker 00

  I just remembered another thing that we've been developing on the InSpace community, which is this concept of AI UX.

• 44:22Speaker 02

  Yeah.

• 44:22Speaker 00

  Right, that the last mile of showing something on the screen and making it consumable, easily usable by people is perhaps as valuable as the actual training of the model itself.

• 44:32Speaker 02

  Yes.

• 44:33Speaker 00

  So I don't know if that's an overstatement, to be honest.

• 44:35Speaker 02

  Yeah.

• 44:35Speaker 00

  Obviously, you're spending like hundreds of millions of dollars training models.

• 44:38Speaker 02

  Yeah.

• 44:39Speaker 00

  And like, you know, putting it in some kind of React app. It's not, it's not the biggest innovation in the world. But a lot of people from OpenAI say like ChatGPT was mostly a UX innovation.

• 44:48Speaker 02

  Yeah, I think like, leading up to ChatG, like when I saw the output of ChatGPT, it wasn't, I don't think I had the same earth shattering experience that other people had in believing like, oh, this output is coming from a model. Like that, sure, it came from a model, but the reception to like that interface and like the human element of the dialogue, like that was, so maybe it's both and, right? Like, it's not, like, you're not going to get that experience if you don't have the innovation under the hood and the modeling and the data set curation and all of that. But it can totally be ruined by like the UX. I typically give the example, like, one day in Gmail, I logged in and like I was typing

• 45:32Speaker 02

  my email and then like had the gray autocomplete, right? I did not get like a pop up that said like, do you want us to start writing your emails with AI? Like, it just like was so smooth and it happened and it made like, it created value for me instantly, right? So I think that there is really a sense to that, especially in this area where people have a lot of like misgivings or fear around the technology itself.

• 45:59Speaker 00

  And we're going to have Alex Graefly on in a future episode, but GitHub, when they had the initial codex model from OpenAI, they spent six months tuning the UX.

• 46:09Speaker 02

  Yeah.

• 46:09Speaker 00

  Just to get Copilot to a point where it's not a separate pane, it's not a separate text box, it's kind of in your code as you write the code. And to me, that's traditional, that's more the domain of traditional software engineering rather than ML engineers or research engineers.

• 46:22Speaker 02

  Yeah, yeah. I would say that is probably, yes. To circle back to what we were talking about, like challenges that are unique to like engineers coming into this versus like data scientists coming into this. That's something data scientists, I think, have not thought about very much at all. At the very most, it's data visualization that they've thought about, right? Whereas engineers generally, like, there's some human. I mean, unless you're just a very pure backend systems engineer, like, thinking about UI UX is maybe a little bit more natural to that group.

• 46:56Speaker 00

  You mentioned one thing, which is about dataset curation. We're in the middle of preparing this long overdue episodes on on on data sets 101. Any reflections on the evolutions in natural and NLP data sets that have been happening?

• 47:08Speaker 02

  Yeah, great question. I definitely like, I think, are you all familiar with Label Studio? And that is one of the most popular kind of open source frameworks for data labeling. And they've been, I think they've been on, we have them on the show, like, we try to have them on the show every year as like data labeling expert. Maybe it's time for that. It's just reminding me, so.

• 47:32Speaker 00

  They just released the, so Erin McHale is in the in the late space Discord. I think you had her on at.

• 47:37Speaker 02

  Yes, yes, she was at ODSC.

• 47:39Speaker 00

  Yeah.

• 47:40Speaker 02

  That's right.

• 47:40Speaker 00

  So they just release new tools for fine tuning generative AI models.

• 47:43Speaker 02

  Exactly, yeah.

• 47:45Speaker 00

  It's a good occasion.

• 47:45Speaker 02

  I think maybe the, that being an example of this is maybe a trend that we're seeing there is around augmented tooling or tooling that's really geared towards an approachable way to fine tune these models with human feedback or with customized data. So like, I know with Label Studio, a lot of the recent releases had somewhat to do with like putting LLMs in the loop with humans during the label process, similar to like, I think Prodigy has been doing this for some time, which is from Spacy. So this sort of human in the loop labeling and update of a model, they brought some of that in.

• 48:28Speaker 02

  But now, like, this new kind of set of tooling around specifically instruction tuning of models. I think before, maybe people, and I've seen actually this misconception, I was in a advising call with a client.

• 48:44Speaker 02

  And they're really struggling to understand like, okay, our company has been training or fine tuning models. Now we want to create our own like instruction tuned model. Like, how is that different from what we've been doing in the past? And kind of what I I tried to help them see is, yes, like, some of the workflow that happened around like reinforcement learning from human feedback is unique, but reinforcement learning is not unique. There's an element of training in that. There's data set curation in that there's pre-training that happened like before that whole process happened. So the elements that

• 49:25Speaker 02

  you're familiar with are part of that. They're just not packaged in the same way that you saw them before. Now there's this clear pre-training stage, and then the human feedback stage, and then this reinforcement learning happens. So I think the more that we can bring that concept and that workflow into tooling, like what Label Studio is doing, to make it more approachable for people to where it's not like this weird, like reinforcement learning from human feedback sounds very confusing to people, like PPO and helping people understand like how reinforcement learning works. It's very difficult. So the more the tooling can just have its own good UI UX around that process, I think the

• 50:09Speaker 02

  better and probably Label Studio and others are leading the front on leading the way on that front.

• 50:14Speaker 00

  I was thinking, like, so labels are one thing. And by the way, okay, I'll take the side tangent on labels and I'll come back to the main point.

• 50:20Speaker 02

  Yeah, yeah.

• 50:21Speaker 00

  I actually presume that scale would win everything.

• 50:23Speaker 02

  Yeah.

• 50:24Speaker 00

  And it seems like they haven't.

• 50:26Speaker 02

  Yeah.

• 50:26Speaker 00

  And and sorry, there's scale, there's snorkel, there's this generation of labeling companies.

• 50:31Speaker 02

  Like data centric AI.

• 50:33Speaker 00

  Right.

• 50:33Speaker 02

  Companies, yeah.

• 50:34Speaker 00

  Right. What happened? Like, how come there's still new new companies coming out? There's Label Box, there's Label Studio. I don't have a sense of how to think about these companies. Like, obviously, labels are important.

• 50:44Speaker 02

  Yeah. Yeah, I think also even before that, there was like tool, at least features even from cloud providers or whatever like AutoML like came before that, like upload your own data, create your own custom model. So I think that maybe it's that like companies that want to create this sort of custom models. And this is just my own opinion, I'll preface that. Maybe they don't want, like when they're thinking about that problem, they're not thinking about, oh, I need a whole platform to create custom models using our data. They're more thinking about like, how do I use these state of the art models with my data?

• 51:28Speaker 02

  And so it's still, if those statements are very similar, but if you notice like one is more model centric and one is more data centric. So I think enterprises are still thinking like model centric and augmenting that with their data, whether that be just through augmentation or through fine tuning or training. They're not necessarily thinking about like a data platform for AI. They're thinking about bringing their AI or their data to the AI system, which is why I think like APIs like Cohere, OpenAI that offer fine tuning as part of their API. It's sort of like people love that. It makes sense. Like, okay, I can just upload some examples and it makes the model better. But it's still like model centric, right?

• 52:13Speaker 02

  Yeah.

• 52:14Speaker 00

  I get the sense that OpenAI doesn't want to encourage that anymore because they don't have fine tuning for 3.5 and four. And then, so the last thing I'll do about data sets and we can go into the lightning round is I was actually thinking about enabled data sets for unsupervised learning or self-supervised learning, right? Like that is something that we are trying to grab our heads around, like Common Crawl, Stack Overflow Archive, the books. You know, like, I don't know if you have any perspectives on that, like the trends that are arising here, the best practices and like. As far as I can tell, nobody has a straight answer as to how, what the data mix is. And everyone's just kind of experiments.

• 52:50Speaker 02

  Yeah, well, I think that's partly driven by the fact that like the most popular models, you don't really have a clear picture of what the data mix is, right? So the people that are trying to recreate that and they're not achieving that like level of performance, right? Then they, one of the things they're thinking about is, well, what are all the different data mix options that I can try and try to replicate some of what's going on, right? So I think it's partly driven by that is like, we don't totally know what the data mix is like sitting behind the curtain of OpenAI or others. But I think there's, there's a couple of trends, I guess, which you've already sort of highlighted.

• 53:32Speaker 02

  One is like, how can I mix up all of these public data sets and filter them in unique ways to make my model better? So some, I listened to a talk, I believe it was at last year's ACL, and they did this study of common crawl, right? And they found that actually a significant portion of common crawl was like mislabeled all over the place, right? Like trash, yeah. So like, I think it was 100% of the data that was labeled as Latin character Arabic, so Arabic written in Latin characters, was not Arabic, like 100% of it. And there was like all sorts of other problems and that sort of thing.

• 54:15Speaker 02

  So I think there's one side, one group of people or set of experiments that you could think about as like, how do I take these existing data sets, which I know have data quality issues, or maybe other data biases or problems that I would like to filter out, like not fit for work data, that sort of thing. So how do I create my own special filtered mix of these and train a model? So that's one kind of genre. And then there's another genre, which is like maybe taking those, but augmenting them with this like simulated or augmented data, right? That's out of a model, like a GPT model or something like that. So I think you could combine those in all sorts of unique ways.

• 54:57Speaker 02

  And I think it is a little bit of like the Wild West because we don't. Totally have a good grip on what is the winning strategy there. And so I think that's where I would also encourage people to try a variety of models. So this is maybe a problem with benchmarks in general, right? Like you can see like the open large language model benchmark on hugging face. And like these models are at the top and you could come away with that and say, well, I'm like, anything below like the top three, I'm not even going to use, right? But the reality is that each of those had a unique sort of flavor of this data under the hood that might actually work quite well for your use case.

• 55:39Speaker 02

  So one example that I've used recently in some work is the camel five billion model from Rider. You know, it doesn't work great for a lot of things, but there are certain things around like marketing copy and others that it does a really good job at. And it's a bit smaller model that I can host and run. And I can get good output out of it if, if I put in some of that workflow and structuring around it. But I wouldn't use it for other cases, but that has a lot to do with the data. And you know, right, I'm guessing Ryder's focused on that copy generation and such. So yeah, I would encourage people specifically on this topic to maybe think about what's going on

• 56:20Speaker 02

  under the hood and also give some models a try for different, like gain your own intuition about how a model behavior might change based on like how it was trained and the mix of data that went in.

• 56:32Alessio Fanelli?

  Awesome. Let's jump into the lightning round. We have three questions for you. It's lightning, but you can say 30 seconds to answer.

• 56:40Speaker 02

  All right, cool.

• 56:41Alessio Fanelli?

  So the, the first question is around acceleration. What's something that already happened in AI that you thought would take much longer?

• 56:49Speaker 02

  Yeah, I think the thing that I was thinking about here was like how general purpose these large language models are beyond traditional NLP tasks. So it doesn't surprise me that maybe they could do like sentiment analysis or even like NLI or something like that. These are things that have been studied for a long time, but the fact that I can like, At ODSC, I was in like a workshop on fraud detection and they were using like some, I forget the models they were using, some statistical models to do fraud detection. I was like, I wonder if I just like do a bit of chaining and like insert some of the examples of these insurance transactions into my prompts if I can get the large language

• 57:32Speaker 02

  model to detect a fraudulent insurance client. And it seemed to like, like I got pretty far doing that. So that fact of like, you can do something like that with these models or that generalizable beyond traditional NLP techniques, I think is surprising to me.

• 57:50Speaker 00

  Awesome. Exploration. What are the most interesting unsolved questions in AI?

• 57:55Speaker 02

  Yeah, I think there is still such a focus on English and Mandarin. It's like that, like you're kind of large language model wise, if you look at the drop off in performance after you get past like English, Mandarin, German. It's Spanish to some degree, but German is actually better than Spanish because of how much it's been studied in NLP. And of course Mandarin has a lot of data. Spanish still does good, but like there's languages, even in the top 100 languages of the world that are spoken by millions and millions and millions of people around the world that don't like perform well in these models.

• 58:37Speaker 02

  So that's like thing one, but even modality wise, I know there's a lot of work going on in the research community around sign language, but like there's all of these different modalities of language. Written text is not, does not equal communication, right? Written text is a synthesis of communication into a written form that some people consume. But the combination of all of these modalities, along with all of these languages, there's just so much room to explore there and so many challenges left to explore that will eventually, I think, help us learn a lot about communication in general and the limitations of these models, but is an exciting area.

• 59:22Speaker 02

  It's definitely a challenge, but an exciting area.

• 59:25Alessio Fanelli?

  Awesome. And so one last takeaway. What's something or a message that you want everyone to remember today?

• 59:31Speaker 02

  Yeah, similar to when you were asking about my workshops. I think I would just encourage people to get hands on with these models and really dig into the new sets of tooling that are out there. There's so much good tooling out there to go from like a simple prompt to inject your own data, to form like a query index, to create like a chain of processing, even like trying agents and all those things. Like get hands on and try it. That's the only way that you're going to build out this intuition. So yeah, that's, that would be my encouragement.

• 1:00:04Speaker 00

  Excellent. Well, thanks for coming on.

• 1:00:06Speaker 02

  Yeah, thank you guys so much. This is awesome.