🤗 The AI community building the future

Hugging Face is increasingly becomes the “hub” of AI innovation. In this episode, Merve Noyan joins us to dive into this hub in more detail. We discuss automation around model cards, reproducibility, and the new community features. If you are wanting to engage with the wider AI community, this is the show for you! Featuring: Merve Noyan – GitHub https://github.com/merveenoyan, X https://x.com/mervenoyannChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: The Hugging Face hub https://huggingface.co/New Hugging Face community features https://huggingface.co/blog/community-updateUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  I feel like the most important thing on Hugging Face is actually Hugging Face Hub. So basically at Hugging Face, we are trying to solve, you know, open source machine learning in general. And this involves a couple of problems, like one of them is reproducibility of your experiments. And also, how easy to infer your models are such that people can just go and stress test your models and see if it works for your use case. The essence of the open source ML in general, that can your model actually be used by someone else?

• 0:43Speaker 01

  Welcome to Practical AI, a weekly podcast making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join us at practicalai.fm/community and follow the show on Twitter. We're at Practical AI FM. Thank you to our partners at Fastly for shipping our pods super fast all around the world. Check them out at fastly.com.

• 1:17Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I'm a data scientist with SIL International. And I'm joined as always by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris? I'm doing okay, Daniel. So good to be talking to you today. It is. It's wonderful. It's a little bit of an odd day. I'm joining from a hotel in Dublin because I'm attending ACL, which for those that don't know is one of the big NLP conferences in the research world. And that's been fun. It's been tiring because I just forgot how tiring an in-person conference was.

• 1:57Daniel Whitenack?

  So

• 1:57Chris Benson?

  Is everyone else dragging around too? Are they all just kind of slumped over?

• 2:01Daniel Whitenack?

  I think so. There's, there's not consistent coffee in all the places, which is unfortunate. So that's, that's tough. But one thing that I have heard mentioned quite a bit at ACL consistently is Hugging Face. And, uh, and we're really privileged today to have, uh, Merva from Hugging Face with us. She's a developer advocate engineer at Hugging Face, really creating a lot of great content on the web and great tutorials and also making really significant contributions on the open source side. I'm really excited to have you with us, Merva.

• 2:39Speaker 00

  I'm so happy that you have invited me. Thank you so much.

• 2:43Daniel Whitenack?

  Yeah, great to have you here. Um, I'm wondering if, if you could kind of give us a little bit of the backstory of how you got connected with, with Hugging Face and NLP, maybe more generally. Like, what was the state of sort of Hugging Face when, when you joined? Cause we'll be talking a lot about like, kind of how it's progressing in this episode. But I'm wondering kind of what, what its state was when you joined and what, what got you excited about it at that time?

• 3:10Speaker 00

  Great question. So I have met NLP at the, at my senior year at the university, actually. And, uh, literally my first project was text mining and doing a classification using Naive Bayes in R about climate change sentiments, you know, if people believe it or not. I have scraped some tweets and, you know, classified them in a data science class that I have taken. And I was like, I'm going to make this my job. And then I later joined to bootcamps, did masters and, you know, started working somewhere as a machine learning engineer. You know, did everything I could. I worked two years as a machine learning engineer,

• 3:54Speaker 00

  doing mostly natural language processing and also a bit of analytics on the side. I was building chatbots, I was doing information retrieval tools and so forth, and I was using Hugging Face back then. But prior to that, prior to building information retrieval tools, and also like putting Hugging Face into my chatbot with the BERT embeddings and such. I, how I met Hugging Face was that I have watched Thomas Wolf's video on future of NLP, and I was like, he's explaining it so well, and I have become a fan. And then, once I have seen him posting about this sprint, community sprint on datasets, and I was like, can I join as well?

• 4:39Speaker 00

  And then I started contributing to Hugging Face, and then I later tried my chances with the audio sprint as well. And I was like, I, I met people over there. And I have learned a lot of things, like, things like, you know, like, they have taught me about C I C D styling, formatting, and contributing to open source, actually. Which is, it was my first time contributing to open source, and I was so excited when my first PR got merged. And I was like, you know, I kept helping other people out over there. And I was like, I'm going to be that person that is going to help people out in this sprint.

• 5:18Speaker 00

  So that's.

• 5:18Daniel Whitenack?

  That's so amazing. I, I love that story because it is very intimidating for a lot of people to figure out that process of committing in open source. And of course, some communities are more welcoming than others. I know Hugging Face is, is very welcoming, and there's a lot of great discussion that happens and all of that. But this is a little bit off topic, but I think it's on, on topic, at least as, as far as the open source side of Hugging Face. But what, what sort of advice would you give people that are looking maybe to start contributing to the open source side of data science or machine learning, or that side of, of the world, the, the open source community?

• 6:01Daniel Whitenack?

  That maybe not even just with Hugging Face, but there's so many great tools out there, whether it's, you know, Spacy or TensorFlow, it's, you know, itself, or all, all sorts of things.

• 6:13Speaker 00

  So basically, libraries like scikit-learn or, you know, Hugging Face Transformers occasionally have sprints in which the contributors are, you know, talking to you, you know, they are giving you issues or like if there is a, for instance, like if we are going to train models, there is a list of languages that the models need to be trained on and there is a dataset. So all you have to do is to actually train the model and, you know, improve it. So there are a couple of sprints, also same with scikit-learn, as far as I know, you know, you can get help from contributors actively rather than being in SING, in GitHub.

• 6:53Speaker 00

  And I would suggest to be, you know, aware of those sprints and community events. Like in Hugging Face, we have a lot of them. For instance, recently we had like a sprint about renewing the docs and adding tie pins and other stuff for the TensorFlow side of transformers, which was a good, um, good first contribution, in my opinion. So I think sprints are a good way to begin with. And other than that, it's just good first issues on the repository.

• 7:25Daniel Whitenack?

  You mentioned that you were kind of building chatbots before joining Hugging Face. Could you tell us a little bit more about that and like maybe how that shaped what you perceived as what was kind of needed in NLP tooling? It sounds like you found a lot of what you thought was needed in terms of Hugging Face and transformers. But how did that process of like trying to build a chatbot? What did that teach you or help you learn? Or maybe introduce to you in terms of challenges for people wanting to do that sort of thing?

• 7:59Speaker 00

  So it's heavily depends on what you're building, actually. So in my first job, I was building an automation box that was, you know, like talking to you and automatically creating an appointment for you or cancel your appointment in the background for us, for service companies. And over there, I was mostly doing machine learning parts. It is usually about how you uh solve the text classification problem and improve your data. I was using Racks Open Source. And if you are having a narrow domain chatbot, like, I don't know, pizza ordering chatbot or whatever, it's a very easy to solve problem because mostly you are solving a text classification problem at the end, trying to understand the end user.

• 8:44Speaker 00

  And like to iterate over your model and everything, it's it's easy. For my second job, it was really hard because like I come from an, you know, like applied math operations research type of background and I do not have any developer background. And I had to do the API side and, you know, learning flask and everything. I was doing both the backend and the chatbot itself. And I was also building this tool that was that helped. We had some researchers. The chatbot was about, you know, like, it was like a friendly chatbot, sort of like replica. But you could ask questions to it about your life

• 9:26Speaker 00

  standards, or like you can ask questions like, hey, I cannot sleep. What can I do about it? And we had like a researcher team that was looking into these answers in the papers and looking for statistical evidence that certainly is good for your health. But the chatbot was rather so hard to make because um basically conversational agents are divided into two. You have stuff like Blender Bot, Dialogy, PT, generative models that basically you can't talk about anything like T zero, whatever. The second part is chatbots that are based on intent and action, which is like um you have to write your own training data. It's not anything like zero shot or whatever.

• 10:10Speaker 00

  You have to define all of your actions to every single intent. It's hard to make a generalization over this and still be in control of what your bot is going to say. Because we know that these language models are a bit biased. You know, they tend to be sexist, racist, rude sometimes. So like, it's a hard problem to solve. So that's why I kinda quit on doing that. Because like, I kinda gave up. I would rather, you know, work in a, you know, chatbot that had narrower domain. Because like, it's not solvable basically without language models.

• 10:49Speaker 00

  And like, with language models, I would rather not put a language model in front of an end user freely with no filtering or whatever, so. And I also like, helped them out. Like, there was this research team and I have built a tool that would, you know, answer their questions from the research papers. And for that, I have used sentence transformers, which I was using through Hugging Face. Like, first time, I can, I cannot forget how I used pipeline for the first time. And I was incredibly amazed on, I was like, it's just one line of code and I can just get answer to my question. I just pass my model, I, this, I have fine tuned the model based on some, um,

• 11:32Speaker 00

  basically there was this biomedical birth and I have fine tuned it on some task for information retrieval. And I called pipeline. And I was like, is this for real? Like, does this actually work? And I was pretty amazed. And then later I looked into it and I was like, they made an abstraction over, you know, all of the, you know, preprocessing, inference and post-processing and put it in a box. And like, how smart is this? You know, it's like an engineering marvel, you know, like, it's just amazing.

• 12:05Chris Benson?

  So I had a totally different follow-up question into COVID. I'm actually wanting to ask you about this. I think a lot of the folks that we talked to on the show have come from developer backgrounds. And they're kind of, you know, they kind of already have that. And they're moving into other skills. And you've come in the reverse way from that. And you kind of had this, this moment there. What was the hardest thing as you were transitioning into this skill set? Where, you know, as you're talking about this history? And that was kind of an awe moment that you had. What's the hardest thing that, what, to, to kind of move into being able to be productive?

• 12:39Speaker 00

  So basically, in my previous job, I was just shipping stuff and, you know, like my, you know, quality of my code wasn't nitpicking and everything to, like, you know, my PRs weren't passing very, you know, in a long line. So in here, because we are working, you know, I'm working with, you know, like, very big teams and very big code bases. I can see how I can refactor things or, you know, like, how can, how I can improve things. So I'm mostly learning development in a way. And also, like, how UX matters, that's a very, like, I feel like it's a billion-dollar

• 13:19Speaker 00

  question, how you handle your UX and how you develop tools, because, like, most of my time at Hugging Face is actually passing with either developing a tool for people on the Keras side. I recently started working on scikit-learn as well. On the other side, I'm just, you know, building fancy demos to showcase people what transformers can do or like other, you know, like libraries can do for, you know, machine learning in general. So I have realized later on that UX actually is hard when you do not come from that background. And also, how you can improve your code, it's just endless.

• 14:00Speaker 00

  Like, there will, there will always be someone nitpicking your code and it's just the most beautiful thing because you keep learning from that. So I am just grateful to work here and I, I feel like I did improve myself from the start. But it was, like, at first it was hard because previously I was only optimizing my models and, you know, nobody questioned my code, you know, that much.

• 14:44Daniel Whitenack?

  Merva, you've already mentioned like a number of things that I'd love to dig into a little bit deeper. Because there's all sorts of pieces of the puzzle that fit into sort of what Hugging Face is and the ecosystem. I was wondering if you could kind of help us just like set the stage for this discussion with. You have Hugging Face, you have sort of model and dataset stuff, you have transformers, somehow Charis, and like you even mentioned scikit-learn. Could you just kind of give us an overview of like, you know, how you would see the Hugging Face ecosystem? And how the various pieces fit together?

• 15:24Speaker 00

  Yeah, sure. So like, what Hugging Face is working on, like if you were to ask to a random person in the company, I feel like the answers would differ. But I feel like the most important thing on Hugging Face is actually Hugging Face Hub. The reason why is because it's so basically at Hugging Face, we are trying to solve, you know, open source machine learning in general. And this involves a couple of problems, like one of them is reproducibility of your experiments. And also, how easy to infer your models are such that people can just go and stress test your models and see if it works for your use case.

• 16:06Speaker 00

  And another thing is, you know, like the essence of the open source ML in general, that can your model actually be used by someone else for their own use case? Which is not likely for the most of the, you know, like tasks, like tabular data related stuff. But you know, it applies for computer vision tasks, audio classification tasks, at least within language or.

• 16:32Speaker 00

  It applies for NLP because your features are usually universal. If not, it's uh language specific. But you know, for at least for computer vision, you can do. You can just go ahead and just pick a object detection or like image segmentation model and use it in your use case. So with the hub, we are actually trying to do this. And we want to have, we want to get people to declare the limitations of their model, declare the biases in their model, so that we can have good open source models on the hub. And we don't only have transformers on the hub. We have like various libraries. So for instance, we host the stanza models from Stanford NLP.

• 17:15Speaker 00

  We have Keras models, like we are integrating various libraries. L N N L P, Keras, uh PyTorch Image Models, you you name it. For instance, with spaces, what we want to do is we want to get people to see if a thing is possible. Like for instance, I can just demonstrate a very small thing, like a product, uh and do a P O T to my colleagues. There are like a couple of use cases and things you can use Hugging Face Hub for your end to end uh end to end workflows. But like my favorite thing inside is I think spaces right now because uh I'm a master's student.

• 17:56Speaker 00

  Most painful thing for me, and I know for the T A's and professors, is to actually reproduce my project. Then, you know, like setting up the environment, you know, running it. And you know, like you have to specify how to do that instead. Like, you know, I'm just sending them the spaces link of my project. For instance, this year I have, you know, like Fourier transform uh space with streamlit and I have just sent it to them. It's also good for, you know, like in my previous job, I was like a machine learning engineer. And I have built like in my first job, I was looking for ways to just put my model out there. And I had zero idea how to use Flask or, you know, like write front end

• 18:40Speaker 00

  or whatever, like it was so hard for me. Like, why would I be spending my time? Like especially if you're in a startup, like you do everything. Why would I spend my time just to, you know, it's it's, I'm not even putting this into production. I just want to showcase this to a client or like, you know, the end user. Like, why would I spend most of my time just trying to put this over there uh through, I don't know, like, just build a demo like that doesn't even look good with um, I don't know, Flask and just channel it with NG Rock or whatever. I think when I was on board, that being on board, that Spaces wasn't better and like,

• 19:23Speaker 00

  not so many people had access to it. When I discovered, you know, Spaces and Streamlit and Gradu and I I was like, this really touches many pain points on that side, especially if you're a data scientist. Most of the data scientists are actually statisticians or, you know, math folks who do not have development background but are working in startups. So it's actually very smart to just write like, five lines of codes and just drag and drop your app dot py file into Spaces and voila, you can just show it to your clients or your end user or your teacher or your family or your favorite pets.

• 20:07Chris Benson?

  I have I have a question that is you've been taking us through this and it kind of almost starts with the fact that as a, you know, kind of having gotten to the expertise that you've gotten to, but you've kind of taken us through this development as you've taken this journey of learning. And the ecosystem around Hugging Face has grown tremendously uh over that time and the and the tools are getting amazing. And as you are communicating this to people who are getting into it, you've got a big challenge just to communicate the ecosystem and all the things that are available. But how do you also, you clearly from from what you were just talking about, kind of remember that beginner's mind? And so as you're bringing new people into the community and teaching them

• 20:50Chris Benson?

  how to be effective and productive in what they're doing with uh within the ecosystem, how have you managed to stay grounded in that way so that you can accomplish both? You can you can sync with them at that beginner level and yet you can get them up to that point where they can run themselves?

• 21:06Speaker 00

  It's really hard, actually, because there is so many good stuff in the ecosystem. It's just understanding, you know, the user journey and like what they are going through and trying to touch, where you can fix their problems in their journey. So Hugging Face recently started to invest in tabular data as well. And like, because I was previously a data scientist, I know what an average data scientist does. And I think, uh, like, a couple of things you can do is, for instance, use data sets library to host your data sets. Which, in most of the platforms, you cannot host data sets that is more than a hundred gigabytes, by the way. And Hugging Face data sets allow you to do that, and you can even stream your data set.

• 21:50Speaker 00

  You know, like, take your data set, just do like an exploratory data analysis. You can, if you want to do a presentation, and if you don't want to show people notebook. Uh, you can do that through Streamlit or Gradle, like, you know, graphs about your data, or like, uh, profile. And then, after that, you can just train your model and push it to the hub and build a space for it. So that you can show what your model is capable of. You know, you can just put your baseline and see, like, let people test it so that it works. So I feel like the answer changes a lot according to what you are working on.

• 22:31Speaker 00

  And, uh, like, what side you are on, on the equation, like, are you a machine learning engineer? Are you a data scientist? Also, like, it changes according to the person you are asking to. But I am, I really like to ask people about their journey and see what type of problem we can solve with that. So, for instance, for end-to-end things, it's more like that. On the other hand, if you're a, you know, NLP person, you can, again, like, take a data set, train a model with transformers. Like, for the previous use case, you cannot do much with transformers.

• 23:09Speaker 00

  Because it's not used much in the tabular data, but like, we have a couple of integrations for the, you know, various libraries. And I can say, like, for instance, type of problems we are solving are like, for instance, we want you to reproduce your experiments. And like, we want other people to know that models have limitations and everything, so. For instance, currently what I'm working on, on the hub is for scikit-learn at least. We, I want to, you know, enable collaboration for scikit-learn. Like, I am currently designing automated model cards for scikit-learn in which it automatically

• 23:47Speaker 00

  produces model cards that has your uh models attributes, the, also the data sets attributes. I have done the same for the Keras. And for instance, I have put inside metrics, you know, from model history. I have put models architecture using graph viz, like you can also have like TensorBoard logs. And you write like one line of code to just push your model to the hub and let it host your uh TensorBoard logs and your model card over there. And if sometimes I, like if your model is working out of the box, then there is an inference widget uh as well.

• 24:29Speaker 00

  The same way you can just, with one line of code, you can just load your model. Yeah, like we want to tackle reproducibility this way so that people know that this models, this model has this metric, it has this hyper parameters that this, you know, training. We want to version them this way. So for NLP, again, you can just train a model, push it to the Hugging Face hub and just, you know, like the inference widget opens or like you can build a demo with again very few lines of code. Because like uh Gradio, for instance, Gradio has the same philosophy as transformers, I would say. It has something called pipe.

• 25:12Speaker 00

  It leverages Hugging Face's pipelines to load an interface. So when you call interface dot load on a model, it automatically knows what type of input that model takes, what type of output that model takes. It will just create the interface for you in one line of code. I'm always um amazed by the abstractions done to save your time as a developer, you know. I think every user has a different story, but like I would first get to know the person and tell them, you know, hey, you can do it like this and just. Like you can utilize Hugging Face hub like this because otherwise it's incredibly distributed, like there are so many things in the ecosystem.

• 25:55Speaker 00

  I have also done a project called Hugging Face Tasks. Uh, I am still maintaining it, so I have come up with this when I was on boarded. So basically, I have worked with a lot of software developers who wanted to build machine learning products. And I know that these people, they just need to know basic Python, like if they want to do a P O C to a data scientist to actually express what they want. Because they do not need to like for the, for the P O C, they do not need to know much about machine learning. So all they have to do is just to go to hub and, you know, filter the models, find the model according to their use case and just call pipeline or inference API on it.

• 26:40Speaker 00

  So most of the people do not know that and they also do not know what the tasks are capable of, like what you can do with an object detection model or what you can achieve with a named entity recognition model. So I wanted to show them, hey, if you want to build X products, then you can just filter for these models and just call pipeline on that model and use it and like check the model's metrics, you know, like if this metric is on this level, then this means that model is good. This model takes X as an input and outputs Y as an output. So that's why you can use it for, I don't know, information retrieval.

• 27:24Speaker 00

  It's also a bit complicated from the machine learning side, so many fancy things going on, but you actually do not know to need all of that. You just need to know which task is suited for you and you just need to call it. I have developed this with this beautiful team of developers and we have released that uh, I think it was around January or February. I just want to, you know, just go and tell every single software developer, hey, you just need to know this and you do not need to learn machine learning from the scratch.

• 28:28Daniel Whitenack?

  So Merva, I can definitely hear just the great respect that you have for your team and also this collaborative environment that you're obviously a part of. And I know that even just today, the day that we're recording this anyway, Hugging Face announced more collaborative features and community features on the hub. I was wondering, from your perspective and how you've grown to work internally with the Hugging Face team on different models and different tools and that sort of thing, what are you excited about in terms of collaborative features of Hugging Face? And what this might enable for the future of the hub?

• 29:08Speaker 00

  Good question. So we have announced pull requests and the community feature today, in which you can open up pull requests to someone else's repository. And this repository can be a model repository, which it contains model and the model files, like configuration or tokenizer if it's an NLP model. It has model card that improves reproducibility and open source machine learning. You have datasets repositories in which you have datasets cards and datasets themselves. Or it can be a space repository in which it has the application file.

• 29:51Speaker 00

  Or if you do not host your model on Hugging Face, it might have your model. So this way, people can improve each other's works like we do in GitHub. So in here, we do not want to actually duplicate work of GitHub. But given Hugging Face is mostly focused on models and the infrastructure as well, like we use Git's large file system to host models and datasets that are very big. Previously, we were versioning datasets and versioning models, dataset spaces. Like why not? Do pull requests on them. So this might mean like, for instance, I have a big TensorFlow model

• 30:33Speaker 00

  and people want to use in PyTorch, or for instance, you know, someone has a PyTorch model, they want to but I want to use it in the TensorFlow ecosystem because TensorFlow has like nice production level tools in the TensorFlow extended. So if I want to, like, I can just port it, but like if I also want to contribute those weights to the repository, then I can just do that, like I can just open a pull request in order to contribute those TensorFlow weights to that repository. Or if someone else has a space that is broken or needs to be improved,

• 31:14Speaker 00

  I don't know, like by means of anything, like needs a description or something like that, or like limitations that I have found in that space. If it has a bias that I have stress tested and needs to be declared, then I can just open a pull request or just, you know, discuss that, hey, I have found this bias in your model and like either like let's declare this or, you know, try to improve the model. Or if I have a data set, then I can just change, I can just tweak stuff in the data set itself and just contribute that. And also have like discussions regarding the models because like, for instance, like if someone has a specific model that has

• 31:56Speaker 00

  that is using a different, different mask token or whatever, they're, if their inference widget is problematic, I just want to go and tell them, hey, you can improve your model like this, or you can improve your space like this. If you were to cache this function, then your space would be faster. You know, I just want to go and tell them that, but I cannot, I wasn't able to do it because like, there was no way, except for, you know, like there's a Twitter handle on the people's profiles, which I think, like if I were to just go and tell them, it would be a bit creepy. So it's nice that we have like now, now we have a discussion section which I can, you know, just tell people, hey, if you were to do this, you know, your that would be faster.

• 32:41Speaker 00

  Or if I, I can just open a pull request and let them see how their space is improved. Because then they can just clone and just pull and just, you know, serve it on their local or just make another space and just see before merging that how their work would look like, how my work looks like on their space or model.

• 33:03Chris Benson?

  Yeah, I'm curious, you know, I think that'll be a really big change because, you know, you've referenced GitHub and if you think about what GitHub did for the open source world by coming into being Git was already there, the social aspect and that collaborative aspect, it fundamentally changed the community worldwide. Like, I mean, it was not the same thereafter.

• 33:25Speaker 00

  Yeah.

• 33:25Chris Benson?

  How, how do you envision the, the social changes or what do you aspire to or hope for, for the changes based on this? Do you think it will propel the community in the same, that same kind of massive shift that we saw in the, in the broader open source world?

• 33:42Speaker 00

  For GitHub, it's literally my favorite social.

• 33:48Chris Benson?

  Probably all of us.

• 33:48Daniel Whitenack?

  Good analogy, Chris. Yeah.

• 33:51Speaker 00

  I just love, you know, like how to follow people and, you know, see what they are starting, uh, you know, to see in interesting projects over there. And I feel like at some point we might evolve to that as well. You know, someone else starts a space and you know, like that's a really interesting one. So let me just go and look. And like even more like maybe like, I don't know, messaging or whatever. I am not in control of this. I just know that, you know, like we are also trying to somehow like increase the collaboration. And that's what GitHub actually achieved. Like, uh, there are awesome libraries out

• 34:32Speaker 00

  there where people contribute to. But for machine learning side, it's not the optimal thing to use. Like for instance recruiters or like technical interviewers, they wouldn't go to all of my GitHub. Machine learning projects, and even if they did, they will understand anything. But like, for instance, I have spaces in which someone could just go ahead and see that, hey, this person is actually doing in this space what I'm looking for, how did they achieve that? And maybe I should hire them. Or just, you know, hosting model weights, like very heavyweight model weights and just, you know, like cloning them is a pain. Like, why would I just want to clone everything in a repository?

• 35:16Speaker 00

  That is a, that is a model like I would love to see if that model works first through a widget or space and just do that. So for that side, it's more optimal and we are looking for ways to increase the collaboration and like give people more uh like a better UX in collaboration with features like this. And also excited to see uh what's next. I feel like the what's next is, you know, notebook viewers and such. I'm quite excited for that. So let's see what happens. I am like, I am not full in full control of the hub roadmap, but like it's mostly about the collaboration.

• 36:00Speaker 00

  Yesterday, there was another feature launched and it was so in model cards, uh you have a metadata section in which you define, you know, languages and everything. You can also define the models that are in a specific paper. It redirects you to the paper itself where you can see which model is actually there. So, you know, again, uh we are also investing in evaluation. So sort of like paper split code leaderboards in which you can see, you know, which model is state of the arts in the task in the Hugging Face hub, so you can directly use that model.

• 36:43Speaker 00

  It's, it's more about, you know, evaluating the model and doing a leaderboard style thing. It's mostly about uh I again, like open source machine learning, what we are trying to do rather than social media network. But it's might evolve that way.

• 37:03Daniel Whitenack?

  Yeah, I have a digression that includes an ACL story. I forgot who I saw on Twitter yesterday. I forget who from Hugging Face said, "We're gonna announce something tomorrow." And I said, "Oh, great, I'll, you know, I'll check at some point."

• 37:15Speaker 00

  I think it's Julian.

• 37:16Daniel Whitenack?

  Yeah, Julian, yeah. So I was like, "Okay, I'll check." But I was in talks most of the day, right? And the one talk, the last talk I went to today was called Quality at a Glance. It was from the Masakane Group, which works in African languages. Yulia and others. And they analyzed a whole bunch of open data sets, crawl data sets, that are on, you know, Hugging Face as well. But they're sort of used widely. And they looked at the quality of those data sets and found very, very interesting and disturbing things. Like, I think in Common Crawl Aligned, there's certain languages. All of the data in that language is not in that language.

• 37:59Daniel Whitenack?

  Like, 0% of the data labeled in that language is in that language. So one of those, I think, was like the Romanized Arabic. Includes 0% Romanized Arabic. I was like, thinking about this as I was leaving ACL. And then I was like, "Oh, yeah, I gotta check, you know, Twitter to see what Hugging Face's thing is." And then I looked and it was like, "Make PRs on data sets." And I was like, "Oh, I need to like circle back around and go right back in there and like talk to them about how we can get some PRs on like CC-aligned and other data sets." So it was just like perfect.

• 38:32Speaker 00

  Yeah, yeah.

• 38:33Daniel Whitenack?

  Perfect timing.

• 38:34Speaker 00

  Yeah, yeah. Someone else can just open a discussion about how that language doesn't count.

• 38:41Speaker 00

  We do, like, we have like a really great ethics team. Like, you probably know them. We have Meg, we have Sasha, we have Yasin. Like, they are just doing amazing work. We recently have Jada.

• 38:57Speaker 00

  And like every time something happens, like we see an inappropriate use case around humans. For instance, like the, the use cases around the like personal identifiable information is actually sometimes problematic. We do stress tests, uh, the spaces and models to, you know, reach out to the people. Hey, you know, your model might have bias. Would you declare it? So we do actually care about the limitations around models and also ethical restrictions regarding, you know,

• 39:36Speaker 00

  you know, biases, personal information and everything as much as we can. So, yeah. It's a hard problem to solve because it's all philosophical. Like in the end, like ethics is a bit philosophical, but like how you can actually put that in practice is a big question. And in case of Hugging Face hub, we do care about, you know, in the models that we train in big science or the models that we have on the hub, we do care that, you know, like if it has a bias, we declare it. We care about the data and everything.

• 40:12Daniel Whitenack?

  Yeah, so important. And yeah, I really appreciate Hugging Face really taking a stance there and putting a lot of effort into that. So a- as we kind of close out here pretty soon, you've mentioned a bunch of things that you either are working on or, or have worked on as part of like the open source ecosystem around Hugging Face. I'm wondering like, what's that thing that like keeps you up at night? Or the thing that's like on your mind that you'd love to do or dig into, but you haven't yet? What excites you or maybe like is, is something you want to dig into at some point in the future?

• 40:48Speaker 00

  So in Hugging Face, like, there is a certain group of people like me that do not really have specific things, like, in Hugging Face, we just, nobody, nobody actually tells you, you know, you should do X, Y, you just, you know, go and pick a responsibility and that's your thing from that moment. So what I did was so far, like, I did the tasks and I did, uh, like a Keras integration in which I have done model cards and TensorBoard and stuff. And like, I really become so happy whenever I see a Keras repository with a model card inside because I know that people actually find it useful and just keep using it.

• 41:32Speaker 00

  I am currently working on how we can, how we can use Hugging Face Hub for the tabular data. So I'm working with an amazing scikit-learn core contributor that is currently in Hugging Face. He's Edrin. Like, we are currently working on a package that is focused on how we can improve the production capabilities of scikit-learn because, like, you use, for instance, pickle, which is, which can run arbitrary code on your machine if you, you know, just pulled any pickle and just deserialize it. It's a bit hard problem to tackle. And we want to post whatever information we can have about the model.

• 42:15Speaker 00

  So like, currently, I'm working on that side. For instance, if it's a tree-based model, you can visualize a tree. If it's a clustering model, depending on what type of clustering model that is, it can be like a dendrogram or like a visual, like with the PCA. Or if you have like a, you know, linear model, you can put the hyperplane. So I am trying to, you know, put those stuff and also what model has learned through feature importance, ShapD values and everything. So what I want is, I want people to call one line of code and just push their models on the hub, which will create these model cards with, you know, several information.

• 42:58Speaker 00

  And I also am working, like, we are all supporting Gradio. And I'm also working on how we can leverage Gradio for tabular data stuff as well. Because like previously Gradio was mainly focused on the modalities like, you know, text or audio or computer vision uh and the components were are if you take a look at the documentation are focused on that. But like we can do, you know, like you can just drag and drop a dataset and just automatically visualize everything regarding that dataset. And that's quite magical. Like, I remember first time I used Pandas profiling and also data analysis baseline library.

• 43:39Speaker 00

  You know, like there are a couple of libraries that enable you to profile your dataset, train baseline models, do like an auto ML like teapots. I remember using them. I was like, this is witchcraft. Like, this is so good. You know, it's just, it it saves so much time. And I was like, you know, recently I realized we can actually do that on the hub. Currently, I am building like two tools. One is like profiling a data set. And like a Gradio, uh it's like a Gradio interface that does it. We have recently released in Gradio something called Blocks, which is more flexible than an interface.

• 44:21Speaker 00

  So you can, you have tabs and you have rows and you you can put multiple stuff inside. So I'm currently building two spaces. Like, as I told you, I either build like, I maintain or add something to the Hugging Face Hub library or something else or do demos. So currently, I'm working on something that is like an auto E D A, like a Pandas profiling. And another thing is an auto ML tool, sort of. So, yeah, these type of things also like save a lot of time and also lower the barrier of entry, I think. Because like you have a baseline model and you can just, it will push your best baseline model to hub.

• 45:05Speaker 00

  And maybe you can create like a space really easily on that, which you can later go and tell your local data scientist, hey, I want this but improved version. I I really love this, you know. I was a big fan of Hugging Face for like, I don't know, since I saw Thomas's video. I think it's been two years or something. I don't remember. And I'm still a big fan of Hugging Face. Like, I go and talk in, you know, Python conferences and people approach me. I say, "I'm a big fan of your conference." And I'm like, "Me too."

• 45:41Daniel Whitenack?

  That's great. Well, we're certainly big fans here and appreciate the way that you're building. The way that you're building really community and collaboration around AI and data sets and all of these things. So yeah, really appreciate your work, Merva. And yeah, appreciate you taking time to talk to us. It was fun.

• 46:02Speaker 00

  Thank you so much for inviting me. I enjoy like talk. I could talk about Hugging Face all day and I would, you know, worry that people would get bored of me. So it's nice to meet people like you who are fans of Hugging Face as well.

• 46:15Daniel Whitenack?

  Of course. We'll talk to you soon.

• 46:17Speaker 00

  Bye bye.

• 46:27Speaker 01

  All right. That is Practical AI for this week. If this is your first time listening, subscribe now at practicalai.fm or just search for Practical AI in your favorite podcast app. We're in there. And if you're a long time listener, please do share the show with your friends. It is the best way you can help Practical AI succeed. Thanks again to Fastly for shipping our shows super fast all around the world, to Breakmaster Cylinder for the Beats, and to you for listening. We appreciate you. That's all for this week. We'll talk to you again next time.