The world's largest open library dataset

Unsplash has released the world’s largest open library dataset, which includes 2M+ high-quality Unsplash photos, 5M keywords, and over 250M searches. They have big ideas about how the dataset might be used by ML/AI folks, and there have already been some interesting applications. In this episode, Luke and Tim discuss why they released this data and what it take to maintain a dataset of this size. Sponsors: Linode https://linode.com/changelog – Get $100 in free credit to get started on Linode – our cloud of choice and the home of Changelog.com. Head to linode.com/changelog https://linode.com/changelog OR text CHANGELOG to 474747 to get instant access to that $100 in free credit. Changelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! LaunchDarkly https://launchdarkly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Power experimentation at any scale. Fast and reliable feature management for the modern enterprise. Featuring: Luke Chesser – Website http://imluke.me, X https://x.com/lukechesserTimothy Carbone – GitHub https://github.com/timmycarbone, X https://x.com/timmycarboneChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Unsplash https://unsplash.com/The world’s largest open library dataset from Unsplash https://unsplash.com/dataThe Unsplash dataset on GitHub https://github.com/unsplash/datasetsUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Our original kind of vision for it is wouldn't it be cool if this was a two-way process in terms of we open up the data and then people who are much smarter than us are able to contribute back things built with that data, which then can be used by more researchers and it becomes this product in itself that continues to grow. And we just kind of provide the inputs to of photos and whatever we're kind of doing on our side. The V1 was like, let's get it out there and let's see what people start using because you know, Tim and myself were not, as you can probably tell, too well versed in machine learning. So we have no idea really how people are going to use this and what fields are going to be valuable. But get it out there, get some, you know, feedback on it, see what people are using. And then if in the future, we have that opportunity to make it a two-way process, we'd love to do that.

• 0:45Speaker 01

  Bama for change log is provided by Fastly. Learn more at fastly.com. Our feature flags are powered by Launch Darkly. Check them out at launchdarkly.com and we're hosted on Lino cloud servers. Get a hundred dollars in hosting credit at linode.com/changelog. What up friends? You might not be aware, but we've been partnering with Lino since 2016. That's a long time ago. Way back when we first launched our open source platform that you now see at changelog.com. Lino was there to help us and we are so grateful. Fast forward several years now and Lino is still in our corner, behind the scenes, helping us to ensure we're running on the very best cloud infrastructure out there. We trust Lino. They keep it fast and they keep it simple.

• 1:30Speaker 01

  Get a hundred dollars in free credit at linode.com/changelog. Again, a hundred dollars in free credit at linode.com/changelog.

• 1:55Speaker 01

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We're at Practical AI FM.

• 2:21Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist with SIL International and I'm joined as always by my co-host Chris Benson, who is a Principal Emerging Technology Strategist at Lockheed Martin. Happy Thanksgiving, Chris. How are you doing?

• 2:40Chris Benson?

  Hey, Happy Thanksgiving. Doing very well. I guess by the time this goes out, it will be just after Thanksgiving.

• 2:45Daniel Whitenack?

  Just after American Thanksgiving, yeah.

• 2:48Chris Benson?

  That's right. We haven't stuffed ourselves yet at this moment. So we're lively, lively today.

• 2:52Daniel Whitenack?

  Although I heard there was a general surplus of turkeys because people are at home and they're, you know, smaller groups, right? Of course, you and I, I don't know if you're, you go for the tofurkey. For our listeners that don't normally listen, Chris and I both don't eat meat. We always have a tofurkey and that's about the right size for us this year, I think. We're not going to have that many people around.

• 3:14Chris Benson?

  Yeah, it's good because my wife is really into like making sure that we have all the, we have the tofurkey and all the little things. But yeah, it's all vegan. And so, yeah, it's delicious. I gotta say, turkey, if you're vegan, you know, five years ago, it sucked to be a vegan on Thanksgiving. It really did. It sucked. But now it's good stuff.

• 3:32Daniel Whitenack?

  As far as like general meat substitutes, I would say it's pretty good.

• 3:37Chris Benson?

  They've gotten a lot better. Yeah, they really have. Yeah.

• 3:40Daniel Whitenack?

  I don't know if they're using AI to like analyze the recipes and get it right, but.

• 3:45Chris Benson?

  I was gonna say, I have no idea what this has to do with AI, but you know, there we go. We're coming into the holidays. It's time to eat.

• 3:51Daniel Whitenack?

  Yeah, yeah. Well, I'm pretty excited today about our conversation. You know, Chris, we've had a number of people come on the show and mention various opinions or ideas about opening up data sets. So open data, whether that be image data or speech data. We had a conversation with Mozilla Common Voice about that data. But generally, a lot of times when we get into these conversations, it's more from the perspective of consuming that data for some purpose, not from the perspective of people that have actually worked to open up data. So I'm really excited today that we have a couple of people from Unsplash with us.

• 4:30Daniel Whitenack?

  And Unsplash has released a huge, actually the world's largest open library dataset. And today, we've got with us Luke Chesser, who's co-founder of Head of Product at and Head of Product at Unsplash. And we've got Tim Carbonne, who is data engineer at Unsplash. Welcome, guys.

• 4:50Speaker 00

  Hey, guys. Thanks for having us.

• 4:51Chris Benson?

  Hey, thank you.

• 4:52Daniel Whitenack?

  Yeah, so before we jump in, why don't we just go into a little bit of each of your backgrounds and how you got interested in doing what you're doing now and eventually ended up at Unsplash? Do you want to start us out, Luke?

• 5:05Speaker 00

  Yeah, for sure. So my background was originally as a designer and I joined two friends to start a company, which was at the time an open marketplace, like a marketplace for hiring designers and developers. And by necessity, over the years, I ended up having to learn engineering and kind of worked my way into building products completely. And one of the side projects we started building inside of that company was a company called Unsplash. And it was just this little image repository that we open sourced. And that eventually turned itself into a full company, which is its own entire story. And now I get to oversee this team of really talented people. I'll pretend that I know a lot about coding and design and everything, but really, they're the ones that make me look really good now.

• 5:50Chris Benson?

  And before we go on, I got to ask, because the listeners only hear the audio. We're all on a Zoom call right now. And I see the guitar is hanging behind you on the wall. So you must be a musician, I guess, or just a fan or what?

• 6:01Speaker 00

  Let's say my coding is even better than my guitaring. So I don't know how good I would say this guitaring is. This is purely for show, I think, you know?

• 6:09Chris Benson?

  It looks good.

• 6:10Speaker 00

  Make myself look cool.

• 6:11Daniel Whitenack?

  Cool. Well, Tim, what about you? What's your background? And how did you end up working with Luke at Unsplash?

• 6:17Speaker 04

  So for my background, I have a classic computer science master that I did in France. And for the final master internship, I was led to Montreal, essentially. And that's when I started working in data. Couple of years later, I had a great opportunity to join Crew at the time, working remote, went back to France for a couple of years, and then went back to Montreal. So now I'm working for Unsplash. Since the transition, I've been working there for it's four to five years now. And I'm working as a data engineer, essentially building the whole data architecture, trying to get some stats in front of our contributors and get some useful insights for our business needs.

• 7:00Speaker 00

  Yeah, and also Tim's gonna be humble here, but Tim's the only person working on data inside of our company. And there's a lot of data. We have, you know, a whole stats analytic system that that serves our two hundred and something thousand contributors. We have, you know, massive search system, all these things. All of it's powered basically by Tim. So he's he's gonna be very humble, but he's he's very good.

• 7:20Chris Benson?

  You got some weight on your shoulders there, my friend.

• 7:22Speaker 04

  Thank you. And same thing, I'm not a machine learning engineer. So AI is very foreign to me. So yeah, mostly focused on data architecture warehouses and things like that. So you probably know more than me on on AI.

• 7:36Daniel Whitenack?

  Well, I think we're all learning and it sounds like from from what you're already doing, you probably have a good bit of knowledge in a lot of areas. So that's great. I'm curious a little bit, you mentioned a bit about how Unsplash came about and this data set came about. Could you give us a little bit of a history of just where this data originated and the nature of it? What's included in your data and why does it exist in the first place?

• 8:03Speaker 00

  Yeah. So going back to, I guess, the fundamentals, the the data exists because Unsplash is this repository of images that are contributed by a community of photographers all over the world. And they open up their images and say, hey, anyone can use these images for whatever they want to use them for. And traditionally, that's been, okay, I'm gonna download these images and I'm going to use them, you know, in a creative project. I'm going to make a graphic. I'm going to use them in an article. But there's so much more to images than just what they look like. And with two million images now in this community, we've been talking about for a while, the idea of opening that up to AI researchers or really just anybody who wants to use it in a different way, you know? And we've had this API traditionally where developers can come in and pull in images for different applications.

• 8:48Speaker 00

  So we power, you know, Medium's images, we power Google's, we power BuzzFeed, you know, a whole bunch of different companies use the images via the API. But the API wasn't set up to be used for the kinds of, you know, data intensive projects where you need to pull in a lot of information. And so. Tim and I have had this conversation, you know, a handful of times over the year. Wouldn't it be cool if we could open up our dataset and make it available to people who just want to play with the data and, you know, make cool things? And it was earlier this year, you know, during lockdown where we didn't have a lot going on, we were like, let's do something. And Tim took it upon himself then to think about, you know, what would be useful, what would be good for a V1 and how could we get that out there? And, you know, Tim's probably the best person to talk about what data we actually expose in them.

• 9:32Daniel Whitenack?

  Yeah, and I should mention too that one of the reasons why we're having this conversation is our editor at Change Log, Adam Stokoviac, one day in our messages during, like you're saying, during lockdown, he messaged me and said, hey, look at what was just released from Unsplash. And this would be a cool conversation. So thanks, Adam, for forwarding that along. Yeah, let's maybe then kick it over to Tim just to talk about what is actually in this dataset. So you mentioned images that people contribute. Is it just images? You know, what's the sort of domain of these and how are they represented?

• 10:07Speaker 04

  Yeah, so the dataset doesn't actually contain the photos. It's basically a couple of CSV files in which you will find the link to download the photo. So you could potentially parse the dataset and mass download everything if you want. And for each photo, you have a bunch of associated data and metadata. So you would have, for example, exit data from the camera. You would have photographer details, you would have geolocation for the photo. You would have Unsplash stats and you would also have things like tags and keywords data that were collected from third-party AIs, including like Google AIs and Amazons and things like that.

• 10:48Speaker 04

  You will also get some data about the colors that are present in the photo. And we were trying to include a bit more of an Unsplash content. So we added the notion of collections. So on Unsplash, when you go on Unsplash, you can create collections of photos. And that could say a lot, I'm guessing that I could say a lot to researchers as to which photo is related to another one or whatever. So we're including that collections data inside the dataset. And we're also including search conversion data that could be very interesting, I think, too. So you'd have people who search on Unsplash, who click on photos, who download photos, and that is also represented in the dataset.

• 11:30Chris Benson?

  Very cool. I guess as you were saying that, something came to mind. I was really wondering, and I'll probably throw it back to Luke for this, how is your business model accommodating that? Because what you're describing is wonderful to hear as a practitioner and a user, consumer of that. But how is your company organized so that you can support this kind of work? And what's the benefit? How does all that work in your world?

• 11:52Speaker 00

  Yeah, so the data set isn't open sourced, or sorry, isn't released in any way to be this commercial product. There is a potential for it to be, to have a commercial business behind it in the sense that right now it's under a license where we've released in two data sets. The smaller data set is able to be used for commercial and non-commercial uses. The bigger data set is more restricted in terms of commercial uses. So there is a possibility that it could eventually commercialize in that way. We're not necessarily banking on that, or that's not the reason why we're doing it.

• 12:25Chris Benson?

  Sure.

• 12:26Speaker 00

  Unsplash itself, though, has in its DNA this idea of sharing and openness. And we, as a business, get to benefit from the fact that people are opening up their images and sharing them and believing in this bigger kind of cause of when I share something out and it gets used, that's helping other people and that's great. So we wanted to do something similar. Traditionally, in terms of the history of Unsplash, we have always started with this idea of if we create value for people, we can find ways to commercialize it later and run a business on top of that. The way Unsplash makes its money is through brands uploading at images to Unsplash that then get this massive amount of distribution through all of our API networks, through the site, and end up in all these different kinds of places.

• 13:09Speaker 00

  So the data set doesn't need to be something where we make money directly off of it. And yeah.

• 13:14Chris Benson?

  Got it. That's very helpful. So kind of in the DNA, but I was wondering, because that's always an issue that we run into, because there's lots of companies that have different models for sharing. And I love the fact that sharing is part of your DNA, but you also have to maintain a company and do that. So it's always an interesting question to find out how people are approaching it.

• 13:32Speaker 00

  Absolutely, yeah. I think we're lucky and we're still a small enough company in terms of, you know, we're 25 people. We can make decisions that are, you know, long-term and don't need to directly impact the company, you know, tomorrow or something. And, you know, a lot of times we do things because we just want to do it and we think it's a cool thing to do. And I think the dataset has, you know, a handful of different motivations behind it. And one of them was just, wouldn't it be cool if we could do this? Absolutely. And when it's small enough, you get to kind of make some of those decisions. Cool, thanks.

• 13:58Daniel Whitenack?

  Yeah, it's interesting to me that there's been this progression in companies kind of built around open source software, so like open core companies or other companies that are built around some open source toolkit, at least on some level. It seems like there's not that many pre-existing examples of companies that are really built around like open data models. Is that true or am I just like sort of missing that?

• 14:28Speaker 00

  As far as I'm aware, I think, you know, if this ever has any commercial benefit, that'd be great. But at the same time, I think, you know, our primary motivation is, you know, let's see what gets out there. Let's see what gets used. And there is a reinforcing idea of, you know, if a lot of different researchers are using Unsplash Images, it reinforces the idea that Unsplash is the place for images. It's the place where you get images and it's one more benefit for our contributors who open up their images where they can say, hey, I contributed to these models that help, you know, self-driving cars or, you know, I have no idea how it's going to be used, but they can say, you know, there's one more benefit of me opening up my work and making it available to people.

• 15:23Daniel Whitenack?

  All right, so I'm kind of interested, I guess, maybe we can kick it over to Tim. I'm kind of interested in terms of like the, just some general stats in terms of the dataset size and the types of images that are included and also maybe like, how is that growing? Like you mentioned a lot about people uploading their images. What's the sort of trajectory of that and what's being added most in some of those sorts of stats?

• 15:51Speaker 04

  Yep, so the dataset, I think the light dataset is about Five gigabytes and the full size data set is more about twenty, twenty-five gigabytes, something like that.

• 16:02Daniel Whitenack?

  And that's the metadata about it, not the images themselves?

• 16:05Speaker 04

  Exactly, yeah, that's just the metadata. So we have about one point three million photos right now. And you mentioned which kind of photos are included. So at Unsplash, we have a manual curation process for every single photo that comes in. And so the photo can end up in kind of like different buckets. So you'd have photos that are featured, you'd have photos that are approved, and because it just matched the guidelines, so that they fit the guidelines. And so that distinction is also presented in the data set as metadata. But in the light data set, for example, only featured photos are showing up. It's a subset of featured photos. And I think they only concern nature photos. And in the full data

• 16:49Speaker 04

  set, everything's in there. So approved photos and featured photos. So yeah, so that's what's in the data set. I'm not sure I remember the rest of the question, I'm sorry.

• 16:59Daniel Whitenack?

  No, it's all good. Also, in terms of the trajectory of how the data set is growing, and obviously, that's also tied to, you know, the main part of your business and people uploading photos and that sort of thing.

• 17:11Speaker 04

  Absolutely. So yeah, we definitely want to make the Unsplash data set some kind of a product of Unsplash, and not just like a single time dump of data. So it's something we want to keep improving, upgrading, getting feedback on, and kind of like iterating over it. So we're getting thousands of new submissions every day. And I think the plan is to kind of like put those new submissions and those photos inside the data set from time to time. So we haven't decided on the frequency yet, but I'm guessing that every couple of months, maybe, or a couple of quarters, we'll be releasing a new version of the data set with new photos, and maybe some improvements. Couple of days after the release, we started

• 17:55Speaker 04

  getting some feedback, and we've pushed directly like a V one point one, a new version of the data set. So yeah, and we've been improving a couple of fields, improving data quality. So right after that, because we started, we started getting feedback. So yeah, we'll be reacting to feedback pretty quickly.

• 18:11Chris Benson?

  That's pretty cool. Is there any kind of thought for the future about like, since you have these open source projects and stuff that'll be pulling the data set down and running ML processes on it and stuff, maybe there's a feedback loop where some of those open source practitioners can help you do, like you mentioned manual curation and stuff like that. Any thoughts toward automating that curation with ML going forward?

• 18:35Speaker 04

  Yeah, absolutely. I think that's something that we talked about with Luke. It's also the fact that we have this base massive data set and we don't have the skills to leverage it properly. So if we put it out there, maybe it can help the research and we can leverage that research that was just made.

• 18:51Chris Benson?

  Yeah, that's part of being a startup, really. I mean, that's normal. You're always trying to find the skills for the next thing to go along and fund it. I totally get that.

• 19:00Speaker 04

  Exactly. So yeah, totally. And also I think that it can also be a feedback loop in the sense that some researchers can create kind of like new metadata for each photo that would be the result of a model. And we could include that result in the data set so that it can be helpful for other researchers. So it can be a feedback loop in that sense as well.

• 19:19Speaker 00

  That's exactly it, Chris. Our original kind of vision for it is, wouldn't it be cool if this was a two-way process in terms of we open up the data and then people who are much smarter than us are able to contribute back things built with that data, which then can be used by more researchers and it becomes this this product in itself that continues to grow. And we just kind of provide the inputs to photos and whatever we're kind of doing on our side. The V1 was like, let's get it out there and let's see what people start using. Cause you know, Tim and myself were not, as you can probably tell, too well versed in machine learning. So we have no idea really how people are going to use this and what fields are going to be valuable. But get it out there, get some, you know, feedback on it, see what people are using. And then,

• 20:02Speaker 00

  you know, if in the future we have that opportunity to make it a two-way process, we'd love to do that.

• 20:07Chris Benson?

  Just now, at the point where this is released, you have thousands of listeners who just heard that. So we should consider that to be a call to action to take advantage of the data set, but also contribute back there.

• 20:17Speaker 00

  Perfect.

• 20:18Daniel Whitenack?

  Yeah, it would definitely be cool to see out of this some examples that pop up on Google Colab or something of people pulling, pulling in some of that data and doing something and making that work itself open. I'm curious, as sort of a follow up to that, it sounds like you have started to get some feedback on this and some usage of it. So I guess one question is just like, how has that gone so far? And like, you know, what sort of influx of usage and downloads have you seen just as a result of this, this work?

• 20:50Speaker 00

  Yeah, so far, I'm not 100% sure on the numbers. We should have checked before we came on. I think it's done about 3,000 downloads. That's of the light set. I think for the full data set, it's probably done about a thousand downloads, probably. So it's been out for three, four months now. So that's, you know, it's a good start for us there. We haven't really been doing much marketing or talking about it. So hopefully something like this will, will get it in front of some more people as well, which will be helpful. The kind of couple of uses that I'm aware of that were, were cool and, and, you know, surprising for me. I mean, you can never predict how people are going to use this. And I think there was obvious things of, you know, people are going to train vision models, you know, and tagging models with it. But it's been cool to see some of the, the kind of more out there uses, one of them being, for example, somebody trained a model to detect

• 21:35Speaker 00

  human faces in, you know, everyday objects essentially. So like clouds and textures and stuff like that. And you could have given me like a million guesses and I never would have guessed that that was something people would do with the data set. So that was cool to see. And then you've got more serious kind of uses of it. So there's some researchers out of Cornell that are, they've written a paper. I don't think it's fully released yet, but it's basically trying to quantify and understand why people post to social media. So with everything going on in politics and across different social networks, one of their main focuses was understanding and detecting kind of the different motivations between why people post and, and trying to, I guess, find some authenticity in that. And so they've been using the UnSpatched data set to, to try and inform that. So that's been, you know, really cool as well to see.

• 22:19Chris Benson?

  You know, it'd be really interesting to see as practitioners use these, you know, for some of these use cases you're talking about and others going forward, they're going to have to go through the process of, of labeling the data in a lot of cases, you know, for any kind of supervised learning. And so it would be interesting to see if some of those might contribute back, some of the labeling for some of the photos to add to the data set, just to make future users easier. So that would be an interesting way for somebody to contribute back, I would say.

• 22:45Speaker 00

  Yeah, absolutely. I know the Cornell researchers were talking about contributing it back because they

• 22:49Daniel Whitenack?

  Oh, awesome.

• 22:50Speaker 00

  They spent a bunch of money labeling and so I think it'd be, I think they're open to that, which would be cool to see.

• 22:55Daniel Whitenack?

  Fantastic.

• 22:55Speaker 04

  Yeah, there's another use case that we have is people using the data set as kind of a, to get out stats for the photography industry in general. Because there's actually a lot of information in there. If you look at, for example, XZ data, you have camera model, camera, brand, stuff like that. And that can give you a lot of information about the state of the industry. And we've seen a couple of articles writing about that and leveraging the data set to get some insights about the industry already. So that's great.

• 23:25Speaker 00

  Yeah, and I think what was cool about those data sets is, you know, people have done that already with Flickr data sets and 500 pixels. But what's different about the data set with Unsplash is people aren't going and bulk uploading, you know, every photo that they take, which can skew the data set in a certain way. You know, if I go and I upload every photo of my family to Flickr, that skews the data set towards whatever I have. But with Unsplash, it's this different thing, which is kind of, what are the best photos being taken with? So is the mobile industry really like mobile photography? How much is that impacting professional or high-end photographers? And we're seeing different results when you analyze it with Unsplash versus, you know, a different story that you would get with Flickr,

• 24:05Speaker 00

  where the iPhone and, you know, Android smartphones are just completely dominating the camera industry.

• 24:12Daniel Whitenack?

  Yeah, it's interesting to me on that front. A lot of image data sets that you can download in sort of typical AI sort of tutorials, initial computer vision work and that sort of thing are actually fairly low resolution. So, you know, we're talking like maybe 227 pixels where, you know, something like that. It sounds like the data set from Unsplash is definitely that sort of higher quality in terms of the actual photographs, but then also, you know, scrutinized according to, you know, people not just uploading everything, but there's a curation process. There's photos coming in that are validated. Could you speak a little bit to that in terms of, you know, the properties?

• 24:57Daniel Whitenack?

  Generally speaking, the sort of properties of the photos in terms of resolution and then also like

• 25:03Chris Benson?

  Just the scrutiny that people give in terms of what they upload and the thought that goes behind that.

• 25:09Speaker 00

  Yeah, so I think that's the central, if we're going to start, what is the major difference between Unsplash's dataset and some of those other ones which exist out there? Because obviously, there's a ton of image datasets that already exist, and some of them have more photos in them than Unsplash. Fundamentally, the quality of these photos, from just a pure technical perspective, every image, I think, has to be a minimum of five megapixels. On average, I think the real size is above eight megapixels. You probably see like a cluster of images at like eight megapixels, and then a cluster at like 12 or 15 roughly. So it's right away, every image is high definition and contains a lot of pixel information in it.

• 25:49Speaker 00

  Then, as Tim hinted at before, we use a curation process where a human actually reviews every single image that comes in. So, right away, you filter out all the, let's call it, spam images from it. And then you also have a thing where on Unsplash, people don't upload photos that look exactly the same. They don't go and upload their entire camera roll, where they've taken 20 photos of the same thing from slightly different angles. It's really, they upload the best of what they have. And so, you're getting a potentially smaller dataset in terms of the number of images, but a much more broad dataset in terms of the types of photos you're going to see and the quality of those photos.

• 26:30Speaker 00

  And so, we think right away, that's something unique and can provide a different perspective. Especially if you're interested in looking at the idea of what makes a good photo, or what makes something beautiful, or how to help people take better photos. This is a great dataset right away for that.

• 26:46Chris Benson?

  You know, it occurs to me that you're almost creating a niche market in that you started Unsplash for the purpose of kind of offering creative assets out there that are very high quality. And maybe, you know, initial users were creatives who were looking for those amazing photos to include in their websites, or whatever it is they're working on. But now that you're making this available for ML, and you've developed a community around the ML Use cases, it might push the creative side as well in that you now have the potential of saying, if you move into, you know, AI-based curation and stuff, then you have with this somewhat unique data set, as you just were discussing,

• 27:27Chris Benson?

  like that high-end, high-quality imagery, then you could actually curate from a website level, for instance, that wants to refresh constantly to where every time you're there, you're getting new things and you could, you could have a model that's trained to pick out certain types of images and download those directly for there so that you can have refreshed creative. It may be a human that starts it off, but then you have ML kind of driving a user experience that's ever refreshing, which is a really interesting idea to me. And that, you know, you could, instead of just randomly picking something, you can have something that is curated at that level for, you know, a really nice website.

• 28:06Speaker 00

  Yeah, absolutely. Chris, you gotta come join our product team. You've been had like two, two of our ideas right away on this podcast. You're quicker than us. It took us months to get there. Yeah, we've got a review team who would love for us to be able to make their jobs a bit easier and do some stuff like that for sure. I think that's in the future for us for sure.

• 28:33Speaker 05

  Change log plus plus is the best way for you to directly support practical AI. Join today and unlock access to a private feed that makes the ads disappear, gets you closer to the metal and helps sustain our production of practical AI into the future. Simply follow the change log plus plus link in your show notes or point your favorite web browser to changelog.com/ plus plus. Once again, that's changelog.com/ plus plus.

• 29:24Daniel Whitenack?

  So because this is practical AI and as Chris knows, I'm always probably prone to jumping into details because I'm thinking there may be other people listening to this that are at companies that have sort of a wealth of data. And you know, I'm even thinking at SAL, we have so much data in our archives that's like so valuable for so many reasons, but it's definitely not, you know, available in this way. And the access patterns are not similar to what you were saying. Traditionally, you had certain access patterns for your data on Splash that weren't like downloading a million images at a time. I'm curious about,

• 30:06Daniel Whitenack?

  maybe this one is for Tim, in terms of thinking about, okay, we're going to open up a bunch of this data. How do we deal with the side of things about, you know, what if a bunch of people all of a sudden start to want to download millions of images at a time? And, you know, how do we support this sort of bandwidth and storage and infrastructure that's going to be required to provide this data in a reasonable way while not jeopardizing the sort of main functions that we do as a business? Any thoughts there? Do you want to take this, Luke?

• 30:44Speaker 00

  I should probably take this one. Because, yeah, I mean, I think our thing to start with is we always try to release things as broadly as possible and then work backwards to restrict it if we need to. You do get a lot of bad actors out there, unfortunately, especially at scale. And we've seen that over the years with Unsplash is it's, you know, our API start out as broad as we could make it. And then we've had to put in place certain restrictions in terms of, you know, don't use it to create competing products where you just, you know, resurface these images or whatever. The data set, you know, has that potential where people could go out and potentially, you know, Spam it and download it a ton of times. You're going to get access to all the image URLs that we have.

• 31:26Speaker 00

  And you could, you know, try and mess with our CDN and stuff like that. But fundamentally, we've seen over the years that those number of people that, you know, they make up a really small percentage of the community that's out there. And so we try and focus on the the good of it and starting with that and trying to make sure we we don't restrict the good uses as much as possible. In terms of how we got it through, in terms of, you know, how do you convince people inside your company to open up data? I think we're lucky in a lot of ways in the, you know, as I was saying, we're a very small company. And so it really is a conversation where it's like me and Tim were like, wouldn't that be cool? Yeah, let's do it kind of thing. So I get in, you know, other companies, it's a lot harder. But we do deal with that, you know, from an image standpoint is we work with a lot of big institutions

• 32:10Speaker 00

  who want to make their archives open and accessible. And getting that through the higher levels is is hard. People want to hoard on to data in whatever form it is. And I think we're making progress as a, you know, as a society towards this idea that openness can have second or third order effects that are harder to project and and put right in the bottom line right there. But they can have this greater effect on your goodwill, on your community, on a whole bunch of different things about your organization. And we're getting there. It's I think we're still, you know, there's a lot of people that want to lock it down and and hoard stuff. But I get to see, you know, people always talk about the advantages of data

• 32:54Speaker 00

  and how, you know, they're building a data advantage over people. I think for a lot of companies, there isn't really actually a data advantage. You can hoard on to that data, but but there's not a ton of value, at least commercially in that that data. And so opening it up to other people can can have a second or third order effect on your business. And so they should consider.

• 33:13Chris Benson?

  I was just gonna say, and it's not a question, just a comment to what you just said, is that working in a large industry for large companies, as I do, I'm seeing companies like yours, with the cultures that you're just now talking about, having that impact in larger companies. And companies that have typically been very large and very closed are looking at what kinds of companies like yours are doing, and they're starting to make those changes, not only in industry, but also in government, actually. So you are collectively having the impact that you're striving for. I just wanted to note that.

• 33:46Speaker 00

  I really hope so. And there's a lot of people out there doing a lot of work, and we're just one small piece of that. But I've noticed that with governments is, governments I think are coming around to it. And you're seeing it, you know, I think probably led first and foremost by open source software. Like, we've seen over the last 20 years or something, how roughly, you know, every major company now contributes to open source. And there's a selfishness to why they're actually doing it as a company, but the fact that we were able to get these companies to that point is, you know, testament to the work that people did back then. And I'm hoping that, you know, if you fast forward 10, 15, 20 years, you're gonna see people being in a similar position from data, where they can actually say, hey, if we do this, which is beneficial to everyone, it also has a benefit to us. And if we have to be the guinea pigs for that,

• 34:31Speaker 00

  we're happy to be the guinea pigs.

• 34:33Chris Benson?

  And it sounds like that you are instituting some sort of like versioning aspects to the data set that you're releasing. Tim, what was your thought process around that? And then maybe also, like, there's all sorts of different formats that you could release, you know, data sets in, you could just, you know, put up some archive files on S3 and have people download like a big thing. Were the decisions around how you

• 35:04Daniel Whitenack?

  Release this data related to how your infrastructure had, you know, in the past, supported people downloading images. What was your thought process around that?

• 35:15Speaker 04

  Yeah, so regarding the archive versus the links in the CSV, I think it's mainly because as a data engineer, I'm not directly working, I mean, because I'm not working in AI, I'm not directly working with the image itself. And so I'm working mostly with the metadata. And so the closest thing I can see from the image or the easiest thing accessible is just its link, right? And we currently have the infrastructure that allows us to basically power thousands of integrations on the internet. And so we might as well leverage that for people to download every single photo through the link in the CSV, right?

• 35:57Speaker 04

  So, so I think that was the main thought process. Regarding versioning, I think that's also kind of a way to stay flexible. So if you add the image to your archive, then it's kind of like frozen in time and you have that thing. Whether if you have a link, then it's much easier to say, I don't know, maybe today you're restricting at a certain amount of pixels and tomorrow you don't want to do that anymore. You have that flexibility and we can find that flexibility all across the data set. Yeah, and I've mentioned that we want to like keep adding more info, adding more things, and that's why versioning is kind of essential. It's super essential in like if we have that feedback loop model where

• 36:42Speaker 04

  where we have new contributions coming in, we need to have some kind of versioning in place. So yeah, that's the thought process, kind of like always staying the most flexible we can. And yeah, that's the idea.

• 36:53Daniel Whitenack?

  I noticed you have like the GitHub repo that talks about the data set. From the people that are making those downloads to access your data in a new sort of way or for new purposes, training models, have you got feature requests?

• 37:09Daniel Whitenack?

  In terms of the image metadata or like new access patterns or other things that maybe you didn't expect or fundamentally different from the ways that your users that are just accessing one image at a time or browsing are requesting.

• 37:25Speaker 04

  So right when we published the data set, I think it gained a bit of like traction and people started using it directly. So we quickly had feedback essentially. Mostly it was about some data quality points. So hey, you should watch this field because there's an unexpected value over there that we didn't find essentially. And so the feedback is really nice in that way. Regarding unexpected uses, I think for a V1, we tried to provide with kind of like our vision of what could be useful for machine learning and AI. So yeah, colors, keywords, geolocation, things like that. And the data requests that we had, I don't think there was anything super surprising

• 38:10Speaker 04

  at the time currently, but we had some feature requests, including data that we actually have and we didn't put into the data set. Things like pixel density, deeper color, or things like that. Some very specific data that we didn't plan to add. And so that gave us insight into, hey, the community might want this. So we might add that into the next version. And so, so we're trying to expose the versioning into our GitHub repo as well. And we're trying to expose the information of what's going to be in the next feature and what's going to be in the next release. And so if you have any suggestion, any, any feedback, you can just go on the GitHub repo and, and ask for it and, and we'll try to answer and plan it for our next release.

• 38:53Chris Benson?

  That's awesome. I think that kind of leads into a natural next question, which is, it's kind of the creative question. And so I'm going to add it. I want to ask both of you. I'm curious from both a business and a technical perspective. And I want you to wax poetic just a little bit and kind of like when you're sitting around talking together, what are some of the aspirations on where you want this to go? What are some of the cool ideas and you're not committed to it? Nobody in the audience is holding you to this. But what are cool things you can think of that you would like to see at any timeframe down the road, you know, going forward, whether you're able to do it, you know, whether that's a practical thing or not even a practical thing. I'd just love to hear what the creativity about how you'd see the future.

• 39:31Speaker 00

  The one that I'm really excited about, or the data that I think is really unique again with this dataset. And it's, I think the data that we have in there is a good start. We still have some ideas for where we want it to go. Specifically, the data around search conversions and the collections, I think, is really unique. There are datasets, again, out there that do this, but at the volume that we're going to get to, especially in the second or third releases of the dataset, I think it starts to become really a unique dataset there. And what's cool about it is people search on Unsplash for very abstract things. It's really, you know, when you look at the types of images that are on Unsplash

• 40:13Speaker 00

  and what they're downloaded and used for, it's not, you know, find me the dog photo. Find me the photo of the cup or the coffee cup in this image. It's like, you know, it's very abstract stuff like love or happiness or depression or like, you know, a whole bunch of things which are harder, I think, at the current stage at least, for vision models to really understand and quantify. But we're getting this real time feedback of millions of people going on Unsplash, searching these things, scouring all the different photos, collecting them into these different collections in different groupings, and then interacting with them in different ways. And we've exposed that data in kind of a V1 version, but we have, you know, a lot more internal data that we want to expose around that. And I think if we can expose that in the right ways,

• 40:58Speaker 00

  and again, I'm no machine learning expert, so I have no idea. But my hope is that that can start to improve the models that are out there around understanding that. And that would be, you know, a huge win, I think, because we experienced that ourselves is when we upload on, you know, images to Unsplash. The quality of the tagging for them is quite limited in terms of it can recognize what's in the images, but it can't represent what does this image actually mean? And so if we can help improve the industry there.

• 41:27Chris Benson?

  Kind of semantic meaning in that sense?

• 41:29Speaker 00

  Yeah, exactly. Like abstract semantic meaning. Like, I think that's the next step in vision learning. And I understand why it's super hard, but our hope is that maybe we can contribute back in some way to that.

• 41:39Daniel Whitenack?

  Very cool. Awesome. Tim, any thoughts in addition to that?

• 41:44Speaker 04

  I think that sums it up pretty well. I think that's fine. I think that's fine. I was going to go on like, go far back on a previous subject. That's not, let's not do that.

• 41:53Daniel Whitenack?

  All good.

• 41:53Speaker 04

  I stole your answer, Tim. I'm sorry.

• 41:56Daniel Whitenack?

  All good. I'm very excited that you took the time to join us on the podcast. I hope, and I anticipate that you'll get some usage out of this, and I hope that it connects with the AI community and they start investigating those more subtle things that are, that are included in the data set and other things that none of us on this call probably can even predict or expect that people will use it for. We'll include in our show notes a link to the data set and a link to Unsplash other resources. Please check it out and yeah, let us know in our Slack channel. You can join our Slack at changelog.com/community.

• 42:36Daniel Whitenack?

  And let us know in our Slack if you're downloading the data set and what you're using it for. And yeah, we're excited to hear. So, thank you, Luke and Tim for joining us. Really appreciate it and can't wait to see what happens with this data set in the future.

• 42:52Speaker 04

  Thanks for having us, guys. Thank you, guys.

• 42:58Speaker 05

  Come hang out with Daniel, Chris, and hundreds of other AI practitioners in our community Slack. It's a cool place to be, not a lot of noise, some great signal, and best of all, it's totally free. Check it out at changelog.com/community. And don't forget to follow the show on Twitter for AI news and links, highlights from past episodes, and more. We are at Practical AI FM. We love to have you following along. Thanks to Daniel and Chris for hosting Practical AI week in and week out. To the mysterious Breakmaster Cylinder for the excellent beats you hear on all Changelog podcasts. To our sponsors who have our back, Fastly, Linode, and Launch Darkly. And to you for listening. We appreciate your time and attention. That's all we have for you today. On

• 43:40Speaker 05

  the next episode, Chris and Daniel chat with the team at Microsoft, all about their new research to product milestones. That's coming at you, so stay tuned next week.