Quick, beautiful web UIs for ML apps

Abubakar Abid joins Daniel and Chris for a tour of Gradio https://www.gradio.app and tells them about the project joining Hugging Face. What’s Gradio? The fastest way to demo your machine learning model with a friendly web interface, allowing non-technical users to access, use, and give feedback on models. Sponsors: Fastly https://fastly.com/ – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorshipChangelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! Featuring: Abubakar Abid – GitHub https://github.com/abidlabs, LinkedIn https://www.linkedin.com/in/abid15, X https://x.com/abidlabsChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Gradio | We are joining Hugging Face! https://gradio.app/joining-huggingfaceGradio | Github https://github.com/gradio-app/gradioUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  But I think one of the very interesting developments that we're seeing, and we're trying to understand this piece better, is that people used to just have demos as kind of standalone things, but now people are taking demos and integrating it onto their websites. One of our users, for example, built this NFT search engine. The backend of it is a Gradio app. He hosted it on Spaces and just like embedded that Spaces on the homepage of his website. And so his home website, I mean, it has some surrounding information, but it's basically a Gradio demo. We've seen this in a few different examples. And this kind of raises the question, so now it seems like a lot of people are building very data-centric or machine learning-centric applications. That's like the focus of it.

• 0:46Speaker 01

  Hello, friends. Jared here to tell you about Change Log++. Our membership program for those of you who want to directly support our work. Your++ membership gets you closer to the metal with extended episodes, makes the ads disappear, and takes our audio to the next level with higher bitrate MP3s. You can join today at changelog.com/plusplus.

• 1:20Speaker 01

  Welcome to Practical AI, a weekly podcast making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join us at practicalai.fm/community and follow the show on Twitter. We're at PracticalAIFM. Thank you to our partners at Fastly for shipping our pods super fast all around the world. Check them out at fastly.com.

• 1:53Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist with S I L International. I'm joined as always by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris?

• 2:08Chris Benson?

  Doing very well, except I'm drowning in yellow pine pollen.

• 2:12Daniel Whitenack?

  Yeah, it's allergy season.

• 2:15Chris Benson?

  Yeah, it's everywhere. Other than that, doing great.

• 2:18Daniel Whitenack?

  Yeah, yeah, allergy season. I've been rotating between sneezing and then debugging some like Nvidia issue today. So it's like Nvidia issue, sneeze, come back, see if it's fixed, sneeze, blow my nose.

• 2:33Chris Benson?

  Is the Nvidia issue contributing to the sneezing?

• 2:36Daniel Whitenack?

  I don't, I don't know. I don't know.

• 2:39Chris Benson?

  Is there a correlation between the two?

• 2:41Daniel Whitenack?

  Let's say no.

• 2:42Chris Benson?

  Okay.

• 2:42Daniel Whitenack?

  On the record.

• 2:43Chris Benson?

  You've just inspired like the next line of Nvidia processors. That'll be like the sneeze or something, you know?

• 2:49Daniel Whitenack?

  Yeah, yeah.

• 2:50Chris Benson?

  They need good marketing and you're providing that for a free service.

• 2:53Daniel Whitenack?

  Yeah, I guess, I guess so. Yeah, speaking of, of practical things, I'm pretty excited about our guest today because, as you know, I'm always wanting to get into the practicalities of what we're talking about. And this really fits into that. I'm sure a lot of our listeners have heard about Gradio. And today we have with us Abu Bakar Abin, who is the Gradio Team Lead at Hugging Face. Welcome.

• 3:20Speaker 00

  Thank you so much, guys. It's awesome to be here.

• 3:22Daniel Whitenack?

  Yeah, yeah. As we get started, maybe just give us a little bit of a sense of like your background and how you eventually, you know, found your way into thinking about Gradio-like things or, or interfaces or apps for machine learning.

• 3:39Speaker 00

  Yeah, absolutely. So I've been doing machine learning for about a decade now. I did my PhD at Stanford and I worked a lot on building machine learning models for medical imaging and medical videos, that kind of thing. And during the course of my PhD, I would often work with collaborators who are not machine learning scientists, not machine learning engineers. There were doctors, clinicians, biologists, that kind of thing. And one of the things that we realized was, well, first of all, not everyone knows how to use machine learning models directly. Not everyone can code, you know, software engineering and all that good stuff. And we wanted to make it easy for other people to try out the model so that it could get feedback. Because you know, sometimes you you tell someone, hey, I I trained a model for you. It's like ninety five percent accurate.

• 4:24Speaker 00

  They're not gonna believe you. They're not gonna take take take your word necessarily. They wanna try it out themselves and you know, test it. And so we started off with a very simple library that was designed to make it easy to to test computer vision models. That ultimately grew into Gradio, which is an open source Python library to build machine learning models very generally. To build GUIs for machine learning models very generally, to build demos, uh, web apps for machine learning models. And then I actually finished my PhD at Stanford last year. And then soon after that, actually, Gradio got acquired by Hugging Face. And so now we're here uh at Hugging Face, which is a fantastic place to be.

• 5:03Speaker 00

  And really, we have this amazing mandate, which is, hey, let's, you know, let's make Gradio the default tool for machine learning practitioners. To build demos, to get their work out there so that other people can access their work. So that machine learning researchers can make their work more reproducible. Machine learning teams can easily collaborate and and kind of see what machine learning models are actually doing. And yeah, and and we're hoping to grow this uh a lot more in the next few years.

• 5:30Chris Benson?

  So just to to blow it out. So for those in our audience who are not familiar with Gradio, could you talk specifically about what that is? Since we've all been talking about it for a few minutes and I wanna make sure that they they have a clear image of what you're talking about.

• 5:44Speaker 00

  Yeah, absolutely. So Gradio is an open source Python library. So you can, you know, pip install it. And what it does is it takes a machine learning model that you've built. Or it could also be an API or any sort of function, but it takes something that you've built and wraps it with a GUI. So you have this like web-based GUI. So imagine like you have an image classification model. You can easily wrap a web-based GUI that allows you to drag and drop your images, see the model's predictions. And but the cool thing is that you can build this GUI entirely from Python, right? So in the past, if you wanted to build like a web application around your model, you needed to know, you know, Flask, but then also maybe Docker to containerize it. And then maybe some bash scripts and then you need to figure out web hosting solutions. And then you need maybe know a little bit of SQL to build a database to collect samples.

• 6:27Speaker 00

  And then maybe some frontend web development, HTML, CSS, JavaScript to kind of build a little UI around it. All of that in one framework in Python, which is already something that you know if you're doing machine learning.

• 6:39Daniel Whitenack?

  Yeah, so before the show, I was just running a few things in Gradio. And I have in the past, but I kind of revisited a few things that I was playing around with. Right now I've got an application running to do just like question answering, which is actually something that we do at S I L and work very closely with those sorts of models. Just to give people a sense, I created this application and I'm looking at it right now. It's 10 lines of code, three of those are blank lines. And in those 10 lines of code, imported question answering model. And like you say, wrapped it in this function, this question answer function. And then with that, when I do Python, you know, this app.py,

• 7:24Daniel Whitenack?

  then it spins up a web server and I can go into my browser and interact with the model. So, so yeah, it's pretty cool. I think it's like pretty interesting that it's almost like it seems like you can have superpowers. Like you get sort of superpowers to do this web stuff without having to actually know that stuff. I'm curious, from your perspective now, doing this for quite a while, seeing a lot of people use Gradio. In the context of like industry and companies, where do you think like the value add comes from this type of, I don't know if you'd consider it like a prototype or an app or a tool. Where does the value add come within the workflow of a typical like data scientist or something like that?

• 8:08Speaker 00

  Yeah, no, I think this is a really important question. So the way that I've seen machine learning done in, both in research kind of settings, but also in industry teams, is that there's almost like two cycles of machine learning. There's the prototyping slash exploration slash research cycle. And then there's kind of the deployment cycle. And the kind of the workflows and the tools that are used in both of the cycles are very different. So in the first one, you know, in the research exploration, you're mostly working in Jupiter notebooks, doing a lot of trial and error. You're building lots and lots of models oftentimes. And you're getting feedback, and this is really important, you're getting feedback from stakeholders. Because it's impractical and usually not the case that the machine learning developer knows everything about what kind of data, you know, how the model is going to be used.

• 8:53Speaker 00

  It's oftentimes end users, you know, sometimes quality testers, sometimes customers, sometimes business teams, who are going to be the ultimate consumers of these models. Right? So like, there's the producers of the models, there's the consumers, the consumer of the models. And traditionally, it's very hard for those two types of folks to talk to each other. And so what Gradio does, it kind of creates the kind of the bridge between these two teams. So let's say you're a machine learning researcher, scientist, developer, whatever your role is. You can easily take a model that you've built and then expose it so that it can be consumed by a variety of different people. And we've seen that be sometimes a quality testing teams. So one of the things you'll notice in a Gradio demo, by default, there is this flag button as well, which helps you catch incorrect predictions and so on,

• 9:37Speaker 00

  and store them in a local like database on your computer and stuff like that. So that that's very useful and it's really designed for like quality testing teams or even end users to try out your model and get back to you. The whole idea behind Gradio is it's so intuitive. Very few people know how to build machine learning models, kind of in the grand scheme of things. More people know how to like maybe load a model, you know, if you've given a model and maybe run some code against it. But way more people know how to use a browser, right? Like billions of people can, you know, literally drag and drop images into a browser and try it out. And what that does is it kind of lets you, it reminds you that, you know, your audience when you're building these machine learning models is very broad. And it lets, you know, it puts the control in their hands so they can they can try it out and give you feedback.

• 10:19Chris Benson?

  Is it kind of conceptualizing a little bit?

• 10:21Speaker 00

  Yeah.

• 10:22Chris Benson?

  If you're kind of making analogy to web development a little bit, and you have these frameworks out there to make it easier, like React and Angular and stuff like that. Is it a little bit like that? Or is it more focused on just the model? When you talk about creating that that user experience so that you can share it with people, what does a typical experience look like?

• 10:41Speaker 00

  Yeah, so really good question. So even when we started Gradio, and recently we've been having this conversation about how general a purpose do we want Gradio to be? Like, do we want it to be the way you create any sort of web application in, you know, directly from Python? That would be really cool. And I think we're working towards that goal. But we've intentionally started with these higher level abstractions that are designed for machine learning models. Like, really, like, if you have a machine learning model and you want to wrap it with a demo with a GUI, you can do it in like, less than like, three lines of code, as Daniel was just talking about. And that's because we've created these high level abstractions that make it super easy to like, plug in a model, plug in what kind of input you want, plug in what kind of output, creates that GUI for you. So really, like, the developer experience right now, as is, is designed, you know,

• 11:24Speaker 00

  really for like, machine learning kind of engineers and researchers. But, and this is one of the things we're working on right now, is we're exposing a low level API that can allow you to actually build more like, more complicated and more, you know, potentially a much wider array of web applications, whether they be for other use cases in machine learning, like, for example, labeling or annotation or a dataset exploration, or it could be for like, other things altogether.

• 11:48Daniel Whitenack?

  And partly because of great communities like Hugging Face and other communities, you know, we are seeing this rapid proliferation of all sorts of kinds of models and how they're used to do like, all sorts of interesting things. So could you describe a little bit with Gradio, like, maybe those main use cases where it's super easy? Like, I did the question answering thing. And then maybe other cases? How does like, the customization work? Maybe someone out there is thinking like, oh, well, I'm not quite doing the same type of like, object recognition or text summarization or something like that. How does that work?

• 12:28Speaker 00

  Yeah, absolutely. So Gradio started off actually designed for images. That was kind of the image-related tasks. But at this point, it covers a pretty much, I would say ninety, you know, five percent of machine learning use cases. And the way it does that, and this kind of goes back to a previous question, Chris, about who it's designed for. The way it does that, it comes shipped with pre-built components. Right? So the idea is, hey, I'm working with videos, I want a video input. And maybe my output is, I don't know, it's a heat map, it's an image, or it's a plot, or whatever it may be. So Gradio comes in shipped with all of these different components, making it, you know, super easy for. Basically, your, your use case could be anything from like, you know, I don't know, maybe I'm doing video activity detection.

• 13:11Speaker 00

  To anything to like more like, you know, traditional data science, maybe I'm working with. Data frames and other kinds of graphs and time series data. Gradio has components for all of that. And we've, you know, we're, we're always like adding new things. And so, for example, we just added automatic speech recognition, or like real-time speech recognition. So you can kind of speak and as you're speaking, you can get a transcription rendered real-time. We have a lot more things in the pipeline as well, some things like three D images and, and, and three D models and out objects. That's going to be released pretty soon as well.

• 13:42Daniel Whitenack?

  So I'm just looking at the documentation and I was also playing around with a couple of other things before this conversation. And it covers a lot, a lot of components. So like input components, including things like, you know, sliders and text box and video and audio. Output components, data frames, files, labels, text box. And so like I, I also work in dialogue systems with S I L and there's even like a chatbot output.

• 14:09Speaker 00

  Yeah.

• 14:10Daniel Whitenack?

  I actually didn't know that before I went into your, like went into your docs again after I'd used it in a while and I was like, that's cool. So I clicked on that and just like opened a notebook and had like a chatbot interface to like plug into my. N L U model, which was, which was really cool.

• 14:26Speaker 00

  You can stitch all of these together as well. So what we, you know, one of the things that kind of astounds me is. Sometimes people will build super complex, like a model will take like eight different inputs. And they will write all of like the Gradio code that's needed to create like this input that takes eight different things. And the GUI is so complicated. And I'm like, wow, you know, they read all this code, which isn't too much code, but still, you know, you have to get, you have to read the documentation, understanding, understand the parameters and all of that. And people do all of that because the alternative is, it's kind of terrible. The alternative is having to write all of this front end stuff.

• 14:59Daniel Whitenack?

  No one wants to do that.

• 15:00Speaker 00

  No one wants to do that. So having this ability, kind of this superpower to do it in, in Python, I think is quite nice.

• 15:43Daniel Whitenack?

  So you already mentioned the acquisition by Hugging Face and now you're, you know, lead of the Gradio team at Hugging Face, which is, which is super exciting. So our listeners can't see video, but I made sure in my, wore my new Hugging Face hat today for the interview. I don't know if you see that. Obviously, I'm, I have a little bit of bias on my side for, for Hugging Face, as our listeners will know. I'm jealous because I don't have a Hugging Face hat.

• 16:09Speaker 00

  How did you get one? I, I don't have one either.

• 16:11Daniel Whitenack?

  Well, I don't know. I saw someone on Twitter post like, hey, beta of the Hugging Face store. And I don't know if it's like public yet. Maybe I'm not supposed to be sharing this, but I saw it on Twitter, like beta of the Hugging Face store. And I was like, oh yeah. So I clicked there and then I, I ordered swag for my whole team. So I splurged a little bit. They won't be surprised, I guess, if they listen to this podcast. I'll try to get it to them before then. But yeah, anyway, with Hugging Face, like, so you've described Gradio. We've talked about Hugging Face on the show many times, but maybe you could just give us, from your perspective, like, what is Hugging Face? And why does it make sense to have like Gradio plus Hugging Face?

• 16:54Speaker 00

  Yeah, absolutely. Hugging Face does a lot of things, so it's hard to describe concisely. But basically, I think the overarching goal of Hugging Face is to make machine learning more accessible. Right? So previously, if you wanted to use state-of-the-art machine learning models, you have to wrangle with a lot of, you know, you have to read papers, you have to wrangle with a lot of, I think, malformed GitHub repos.

• 17:16Daniel Whitenack?

  I don't miss those days at all.

• 17:18Speaker 00

  Exactly. It's a lot of work. I remember that from my PhD as well. It's a lot of work. The way I see it is that Hugging Face offers various levels of access, right? So for example, there's Hugging Face data sets, which is designed for machine learning practitioners, really, to train their own models. Then there's Hugging Face models, which is designed for machine learning practitioners, but also software engineers, who don't want to have to think about like what's under the hood. They just want to use a really good, you know, image classification or really good question answering model and not really worry about any of the algorithmic, you know, implementation details. And so that, you know, opens it up to a lot more people. And then with Gradio and/or Spaces, which is what Hugging Face calls demos, that level of access opens up even more.

• 18:03Speaker 00

  So now, pretty much, anyone who can use a browser, like we said, you know, billions of people who have a browser and are connected to the internet, can now use state-of-the-art machine learning models, and they can do interesting things with that. And I'll just give you two quick examples. One is, we had this demo that someone built with Gradio called AnimeGAN. And we've seen many, many such examples, but AnimeGAN was this demo that you could put your own image, or put any image, and it would render it into this cartoonized, you know, like, almost like, I think it's called rotoscope, or like drawn version of that image. And someone built a demo hosted with Gradio, hosted on Hugging Face Spaces, which is a place you can host your Gradio demos for free. And then released it on Twitter, where it went viral, or you know. I remember it, yeah.

• 18:48Speaker 00

  Yeah, yeah. And we had thousands of people use the model, I'd say maybe tens of thousands. And you know, tried on their own profile pictures, pictures of their pets, you know, everyday objects. And they were interacting with actually state of the art machine learning, which is something that's never been done before. And that, that is really cool because now machine learning developers are thinking, hey, the audience for who's who's using my model is a lot bigger. Let me make sure that my models are robust, they can handle like diverse images, diverse inputs. And that leads to a level of, I think, you know, concern for the end user that wasn't really there before. I think that's really important because then issues of bias and accessibility are addressed as well. I want to give one other example as well, because I think this is interesting as well, because

• 19:30Speaker 00

  demos have a big purpose for education as well. And I remember one of the early days of Gradio, actually, we had built like this demo for an MNIST model. And I actually shared this, and you know, this MNIST model, you can draw handwritten digits and you can see what the prediction of the model is. And I shared it with my sister, uh, my younger sister, who has like no background in machine learning whatsoever. And she tried using the model and she, uh, you know, she drew a six and it predicted six and all that work. And then she just drew a little dot in the center. And I think it predicted like a seven. And she was like, uh, why did it predict a seven? That makes no sense. I just drew a dot. And then I, I told her, well, it kind of has to predict something, you know, and maybe the sevens were just the most common thing in the data set.

• 20:12Speaker 00

  And she was like, hmm, that doesn't seem right. So and I was like, well, what, what should I do? And she should, and she said, well, it should just like avoid making a prediction, you know, and it should kind of abstain from making a prediction. And so she had stumbled upon this idea of abstention, which is, you know, now a really important topic. A lot of people think about that. But you know, if you've never really like interacted with a machine learning model in this way, you might not even realize the importance of it. And you know, this someone has no background in machine learning. So I, I think demos, I can go a really far away in both accessibility and then also education as well.

• 20:42Chris Benson?

  So I love that. I actually like to dig in, having gone through those two examples and dig in just a little bit. It really begs the question of what typical workflows look like because you've kind of shown us two examples of kind of going out there and doing that. And that second one in particular, there was a big insight, you know, because we're all in this industry and I think there's some things we take for granted because we've been doing it for a while. But you had someone who wasn't someone important in your life, who was not in the industry, make you realize something. And I think we all have moments like that. What are some of the typical ways that you and your team and other people that you work with are using Gradio on a day-to-day basis that has like directly changed the way the workflow is? Where do you fit this in?

• 21:24Chris Benson?

  If I'm adding Gradio into my machine learning, I'm already maybe using Hugging Face, but now I want to use Gradio as part of that. What's changed? And how do I think about my workflow now?

• 21:34Speaker 00

  Yeah, absolutely. Let me give you one more example, I think, that might illustrate that, and then we can discuss that. So this was actually one of the early examples of Gradio, and where I realized it could actually have a big impact, especially in interdisciplinary teams. And so I was building a machine learning model to classify ultrasound images of the heart, so echocardiograms, if you're familiar with them. And so we had built this model, and it was getting really good accuracy, like 95%, AUC and all this good stuff, to predict like various things about the heart. For example, like, does this patient have a pacemaker in their heart or not? And so we had built this model and we shared it with a cardiologist, and the cardiologist was a little skeptical about, you know, how well the model was working.

• 22:18Speaker 00

  And so we built a Gradio demo around it, and we let him play around with it. And one of the things you can do with a Gradio demo is you can interactively edit the input. So the cardiologist can upload his own ultrasound image, in this case, and they can kind of edit it. And so they could, for example, remove the pacemaker from the image by like, you know, kind of white outing it. And so he did that. And so there was a pacemaker, ultrasound, the model was predicting this patient as a pacemaker, then the cardiologist white outed it, removed it, and then the predictions changed in real time. And when the cardiologist saw that and did this with a few different images, he was like, wow, this actually works, you know, this actually works. And even us, all of the machine learning people in the room, we all breathed a sigh of relief.

• 23:01Speaker 00

  Because one thing to see aggregate metrics, but it's another thing to have someone adversarially test your model. And then it's still, you know, it's kind of robust to that. Basically, I think there's two broad ways that Gradio can help. And one is if your model is good, it can help build trust in the model, especially for important stakeholders, because they can test it. But if your model is bad, and this may be more important, if your model is bad, it can help expose those issues of bias and other things that are really important as well.

• 23:29Chris Benson?

  I had a little follow up and you're starting to address it there at the end already. But I'm curious, and it's a little bit of a side issue. It's not a direct Gradio issue, but Gradio is obviously part of the solution to this. And that is that several times in our conversation, we've mentioned the idea of people being skeptical of model output and like, I don't know and all that. And obviously you've seen that and you're addressing it and you have, you've produced a really good tool for letting people get that. But I'm curious, as someone who's observed that repeatedly, what is it that's causing the skepticism in trusting the model? And obviously they get to a point where the tangibility of using Gradio allows them to get past that.

• 24:12Chris Benson?

  But what, what do you think is causing the challenge? The non-machine learning consumer of that model is facing upfront?

• 24:20Speaker 00

  Well, I think you only have to look at, you know, some of these very famous models. You look at something like GPT-3, for example, OpenAI released this model. It's supposed to be able to understand language and answer your questions. But people try it and they discover all sorts of problems, right? So you can ask it questions and it starts making very nonsensical kind of responses. Or, you know, even suggest very dangerous things. I remember there was a study like that showed that if you ask it medical questions, it could suggest things that could lead to self harm and all these terrible things. And I myself was part of a study and I did a study where we were looking at bias and kind of religious bias. And found that GPT theories, all these issues associating, for example, Muslims with violence and all of these kind of problematic things. And that's just one example.

• 25:03Speaker 00

  And so OpenAI did something interesting, which is that they did release it as a demo so that people could try it out. But that doesn't happen most of the time with research, right? So you see all of these really nice sounding numbers. And like, you know, state of the art this and that and and nice models published in Nature that claimed they'll solve this problem. I remember actually there was one person I talked to when I was a PhD student at Stanford. And she had she had made this really nice model to look at videos of people at ICU, kind of in the ICU. And like from those videos, you could tell if a patient had a particular disease, like based on how they were moving around. And as she published in Nature, I was like, oh, this is really cool. We should try to deploy this in the clinic. And then she looked at me, she was like, are you crazy? I would never trust this in the clinic.

• 25:46Speaker 00

  Because like, there's a big gap between, I think, what is, you know, publishable and sometimes what people can get away with. Versus, you know, what is actually usable in the real world for all of the reasons that we've talked about. Like there's usually models are trained are trained and tested on these really nice sanitized data sets. And, you know, we don't really expect to test them on on day on kind of real world settings that might be out of distribution. And so I think one of the cool things about Gradio, and we're seeing this more and more, is researchers. For example, at CVPR 2022, they published papers and they released a company in Gradio demos. And then the research community was testing them. And there were some models that did great, like people were testing them all sorts of difficult ways. And they were doing great. And then other models, people found holes relatively quickly.

• 26:31Speaker 00

  And so I think part of it is because, you know, the machine learning community is so accustomed to to training and testing on these very fixed benchmarks. That really, really stresses the need for something like Gradio to open that up, you know, open up that box and let other people tie with their own data.

• 27:10Daniel Whitenack?

  Well, I definitely want to follow up on some things related to Hugging Face plus Gradio, but maybe from a different perspective. I'm wondering, as you were running Gradio pre-Hugging Face, and you had the open source project, and in certain ways, you might not have known all the ways that people were using Gradio, but had some sense of how it was useful. And now you've got this Hugging Face scale of people using it, and an avenue for people to share things, and they're sharing a lot of things. What sorts of challenges have you faced as you've tried to integrate Gradio at Hugging Face scale?

• 27:52Daniel Whitenack?

  And tried to scale that up, make sure it runs well for people as they create their own spaces with all sorts of different models, some which might be really big and some which might be pretty easy to run? I just imagine that that's a really hard thing to do, but it seems like you've done it very well. So yeah, any thoughts there?

• 28:15Speaker 00

  Yeah, that's a good question. There's a lot here. So one of the things that I'm amazed by is just the diversity in the types of models that people are building. This is shocking. I think sometimes you can become an echo tamer. You think, okay, most people use machine learning for these types of use cases. But then you see users that just completely blow your mind. For example, people using GANs to generate Pokemon, or people using speech recognition in so many other languages. And you have to realize you have to support Unicode, this and that. It's just a lot of different use cases that show up, and you have to address that. I think in terms of... So one of the good things about Gradio,

• 28:58Speaker 00

  I talked earlier about how there's two different cycles of machine learning. There's the exploratory/research stage, and then there's actual production level types of stuff. So at Gradio, we tend to focus more on the exploratory/research side. And so even when you share a model, let's say on Spaces, or you share it temporarily, so I don't know if you've seen this, but... If you create a Gradio demo, you can pass in one extra parameter in the launch function, share equals true, and that creates this like temporary public link that allows anyone to access your model, which is super, super handy for prototyping. And so what we've like entirely focused on, we've said this, we're not trying to, you know, optimize for like production level of traffics or anything like that. We want to just focus on, hey, let a few people try out your model, get feedback, let a few other people try your model.

• 29:41Speaker 00

  And so that, because we've kind of focused, laser focused on that use case, even when a lot of other people are using Gradio demos, we have like always stayed with that expectation. So for example, people ask us, oh, my space is up on, on, on Hugging Face, I want to use it as an API. Well, we had a support for that, but we kind of made it clear, hey, this is not meant to be like a production level API. You can use it for testing and so on. So I think we've mitigated a lot of like traffic type related issues just by focusing on this stage of, of kind of the use case. And then the other thing that we've tried to do is leverage Hugging Face's existing infrastructure as much as possible. So for example, Hugging Face already has something called the inference API. So any sort of model that you can find on the Hugging Face hub, which at this point is, I think, more than thirty thousand different models,

• 30:26Speaker 00

  it comes with its own like inference API that you can just call. So Gradio also supports using any of the models in the Hugging Face hub pretty much off the shelf. So in like one line of code, you can build a demo for one of these models. And if you do it, it leverages Hugging Face's existing inference API rather than trying to create something ourselves. So by doing that, I think we've mitigated a lot of those like load issues. But what we definitely find is a lot of people using Gradio in like these different use cases that we wouldn't even even imagine. We see, you know, a lot of issues being raised and then we're doing our best to kind of support that. A lot of cool things, like people, people ask for new types of components as well. And so we're, we're, we're working on supporting that. A lot of people ask for like, we're using this space to test our model.

• 31:11Speaker 00

  We're seeing some, you know, kind of weird behavior. How do we kind of retrain the model based on what kind of issues have been observed? So we're thinking about how to fit Gradio into kind of this larger loop of, of training models again and making them better as well. So yeah, a lot, a lot of cool things there.

• 31:26Chris Benson?

  I'm synced with you pretty well, I think, because you're already going where I was going to ask you. And that is, as you talk about these new components that you're building, and you've talked about kind of focusing more on the exploratory side of that, but you've already acknowledged that there's the kind of the production deployment side. Is that where you want to go? Is it, it seems like it might be a natural workflow that if I'm already doing Gradio and getting my feedback with my demos and everything, that I might ultimately just want to deploy that in a variety of areas. And so as you build more components out, is that with that, that in mind of eventually you are robust enough in how you're making that model available so that you can, that it's Gradio all the way and forevermore?

• 32:08Speaker 00

  Yeah, it's interesting. We're thinking about that. And right now, we're actually leaning against that a little bit. And the reason for that is because that space is first of all, very crowded. There's a lot of like tools that are designed to help you deploy your, your, your model. And it's kind of an interesting, the issue is, it's one of those things, it's kind of like, I would say like, kind of like the Heroku type problem, which is that if people get big enough, they don't want to use your solution to deploy, they want to do it themselves. If people are small, like they want to, so there's a lot of people who are just prototyping. Great, so we've got that use case covered. When people are big, like kind of medium size, maybe they'll use like an off the shelf product to deploy your model. When they're very big, they're going to write something themselves or use something that's, you know, kind of very tightly coupled with one of the big cloud vendors.

• 32:53Speaker 00

  And so we're, we're actually thinking that rather than focusing on the production kind of, production use case, which is kind of crowded, what we should do instead is make it easier to build more kinds of web applications from Python itself. And so I think we're going to be leaning more in that direction. You'll see things like potentially solutions for, you know, maybe lightweight labeling solutions built out of Gradio, or maybe data set exploration tools built out of Gradio, trying to cover more of those use cases.

• 33:22Chris Benson?

  Love the focus there.

• 33:23Daniel Whitenack?

  And as you look towards that, I also note that I believe, if I'm not mistaken, the sort of main core of, of Gradio is, is open source. I don't know if there's certain things that, you know, are maybe integration things with Hugging Face and other things that, that aren't, but the main bit is, I'm wondering how that community has grown over time and the sort of codebase and open source community, what you're seeing in terms of activity and interest there?

• 33:52Speaker 00

  Yeah, that's been amazing and we obviously owe a lot to Hugging Face for that. So like, for example, we have a Discord server. I think if we had just launched our Gradio Discord server, I don't know how many people would have joined, but we're part of the Hugging Face Discord server, which helps a lot. And our community is honestly amazing. There's some folks, and this astounds me, there's some folks that use Gradio every day and they're like raising new issues. Like, I don't use Gradio every day, you know, so it's been really nice. People catch issues like that. Anytime we break anything, people let us know. But I think one of the very interesting developments that we're seeing, and we're trying to understand this piece better, is that people used to just have demos as kind of standalone things, but now people are taking demos and integrating it onto their websites,

• 34:36Speaker 00

  like as portfolio, you know, as part of their portfolios, but even just as like standalone websites. So one of our users, for example, built this NFT search engine using the backend of it is a Gradio app. He hosted it on Spaces and just like embedded that Spaces on the homepage of his website. And so his whole website, I mean, it has some surrounding information, but it's basically a Gradio demo, which is very interesting. I mean, you know, and this is not the only one. We've seen this in a few different examples. And this kind of raises the question. So now it seems like a lot of people are building very data-centric or machine learning-centric applications. That's like the focus of it. If there's enough use cases like this, maybe we wanna like focus on building, you know, kind of this one-stop-shop for how to build like a complete data-centric application. So we're thinking about that as well.

• 35:20Daniel Whitenack?

  I work with a lot of students from Purdue University, have a close collaboration there and other universities as well. And they're always asking me, you know, like as I'm going from like grad school into industry, like what can I do to set myself apart? I think for like those of you listening out there that are listening to this, like, this is like a really cool idea that is very, like can set you apart. Like if you're able to go past saying, hey, I ran this cool example in my Jupiter notebook and here's like, The GitHub repo with the Jupiter notebook, okay, it renders and I can see some images. It's a whole nother level when you can like point someone to spaces or to a Gradio app or embed that in a blog post or your website or whatever and have someone actually interact

• 36:05Daniel Whitenack?

  with it. I think that that like goes a long way. We get a lot of requests from people listening to the podcast and in our own lives about like people getting into AI and data science. And I think that's just a general like free tip out there. I think this episode has shown that.

• 36:23Chris Benson?

  It really differentiates that person from all the competition. And I know that in my own organization, we see that as well as people are coming in. If you, you know, walk in and you're a Gradio expert compared to people who are not able to show that, it's a huge differentiator for someone. So yeah, it's a great point you're making there.

• 36:41Speaker 00

  Yeah, that's a lot of fun.

• 36:43Chris Benson?

  It is a lot of fun.

• 36:46Speaker 00

  Yeah, just to be able to interact with your model, like it's just so much more real. You know, as we've talked about, you start noticing these things that you otherwise would just not have even paid attention to. Like a lot of things are just buried under these nice aggregate metrics that we like to take a look at. We just get way more insight when you can actually play it on with your model.

• 37:03Daniel Whitenack?

  And as you look to kind of the future of Gradio and Hugging Face and maybe like other things that are happening with those two things, what are some of the things that are exciting to you about maybe the AI space more generally and then maybe more specifically in the Hugging Face and Gradio world? Like what are you thinking about that you're super excited about the developments of and looking forward to the future?

• 37:32Speaker 00

  Yeah, absolutely. So I think one thing that really excites me is that we are moving away from this, like what we've talked about, you know, this benchmarking on these static data sets and that's it. Now a lot more people are interested in out of distribution robustness, right? Like we've trained a great model on our data set. What kind of guarantees can we give about how well it's going to perform in the real world? Obviously, it's a very difficult topic, but there's a lot more interest both on the academic side and then also on the industry side with practical tooling. And so this is where I think, you know, we're gonna see companies, you know, potentially including Hung Face also invest more resources in. Like you have some models out there. How can you effectively flag issues that the model is having so that other people who use

• 38:14Speaker 00

  it are aware of these limitations can contribute to the robustness of the model? Because I I do think at the end of the day, it's a it's a it's a problem that's probably going to be, it's very hard, I think, to formulate this problem in a clean way such that we can tackle it from a theoretical point of view and academic point of view. I think what we need are better tools, like to identify issues of behind models and and to like let people almost like, you know, just the way same way like GitHub, for example, is issues and PRs, you know, what is the equivalent of of that from machine learning look like? So that the people are kind of aware of the issues and can and can make things better. So that's something I'm personally very excited by. I think radio plays a role with that in helping discover these issues. But I think it's a much bigger problem just alone that Gradio can solve.

• 38:58Daniel Whitenack?

  Yeah, and you mentioned even, we didn't go into it in detail, but you do have this flagging feature within the app. Could you maybe tie in how that how you see that fitting into what you're talking about with this out of distribution input and that sort of thing?

• 39:12Speaker 00

  Yeah, absolutely. So this is one of like the core fundamental things that we added to Gradio early on, and we see people using it now very actively to this day, which is that Gradio lets you try out your own data in potentially someone else's model or your own model, right? You can put it, you can drag and drop your own image, you can type your own text, you can edit it, you can play around with it. And let's say you find something that the model isn't working well on. Let's say, I mean, just for an example, if you take a state of the art image classifier and you put in a picture of a of a bride who's wearing western attire, like American, let's say, you know, typical, it predicts bride, bride or ceremony, you know, pretty reasonable labels. But if you put in a picture of a bride from Pakistan, where I'm from, or India, or where

• 39:55Speaker 00

  it's, it usually predicts things like costume or performance or something that's, you know, could be that's wrong, maybe could be borderline offensive as well. You know, you find issues like this all the time with all sorts of machine learning models that we've talked about, they're very fragile. And so what this flagging button lets you do is it saves that sample and it stores it to a local database in like a CSV file, basically. And so the workflow, what this looks like in practice, and we see this often is people will create a Gradio demo and they'll share it because it's so easy to share that demo. It's like a, just like a link, like a Google Drive link. You share it with a bunch of people, they tie out your model, they identify issues, and that helps you, and then they can just click the flag button. That's all they have to do. And then you have this nice CSV of everything that's been flagged. And you can say, ah, okay, maybe I should retrain

• 40:37Speaker 00

  my model on these samples, or maybe I have some better understanding of what the failure points are. We have users who used to like, you know, have spreadsheets where they would send these spreadsheets back and forth. It's kind of a replacement for that and maybe a better way to do it.

• 40:50Daniel Whitenack?

  Yeah, I, I think this is super important. And yeah, I'm, I'm just really excited about the future with these sorts of tools and what you're doing. Keep up the good work. Really appreciate you taking time out of your, your busy Gradio hugging face life to, to let us know about these things. It was a pleasure talking. Hope to talk again soon.

• 41:09Speaker 00

  Yeah, it was great. Thank you so much for having me here and all the great questions.

• 41:14Daniel Whitenack?

  Bye, bye.

• 41:23Speaker 01

  All right. That is Practical AI for this week. If this is your first time listening, subscribe now at practicalai.fm or just search for Practical AI in your favorite podcast app. We're in there. And if you're a long time listener, please do share the show with your friends. It is the best way you can help Practical AI succeed. Thanks again to Fastly for shipping our shows super fast all around the world, to Brake Master Cylinder for the Beats, and to you for listening. We appreciate you. That's all for this week. We'll talk to you again next time.