E1: The Pixel Revolution with Playground AI 's Suhail Doshi

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. 00:00 Intro 00:30: Image technology breakthroughs 00:45 The Cognitive Revolution 03:10 Intro to Suhail 03:43 Sponsor 07:04 Suhail’s path to AI 12:57 Suhail’s vision for Playground 17:58 Suhail’s cancellation on Twitter 19:06 AI artists are artists 20:25 Non-AI artists feel threatened by AI 26:47 Suhail on defensibility at AI companies 30:30 Playground’s product roadmap 33:01 Suhail on good design 35:55 Latent space 44:04 Building Playground with lessons learned from Mighty 53:55 Monitoring led to 2x improvement in the product 55:53 How people use Playground 1:03:48 AI is having it’s mobile moment 01:15:12 Finding, investing and building durable AI companies 01:22:06 Suhail’s kid’s best friends may be AI characters 01:23:20 AI religion 01:24:16 Conclusion RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics SPONSORS: SHOPIFY | OMNEKY Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries. From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real time advertising data, to generate personalized experiences at scale. Twitter: @CogRev_Podcast @eriktorenberg (Erik) @suhail (Suhail) @labenz (Nathan) Websites: playgroundai.com mixpanel.com waymark.com This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  It's kind of like that moment in mobile

• 0:04Speaker 00

  where it was just a complete like Wild West gold rush and nobody had a clue what was going to work. And so just all like thousands, tens of thousands of experiments occur simultaneously. And you you end up getting things like Uber and and some things like grow really rapidly and then fall because they they fall out of relevance. They didn't have good retention and some things just have this incredible lasting power for a decade. So feels like that will happen again.

• 0:31Speaker 01

  In one short year, image generation technology has achieved multiple breakthroughs and revolutionized the world of creativity and art. Today, with mere words, in just seconds, anyone can generate all sorts of high quality images. Dedicated AI artists can create top notch, award winning art. Nathan, how did we get here?

• 0:50Nathan Labenz?

  Well, Eric, do you remember where you were when you first saw the Dolly avocado chair? That was the original Dolly, announced two years ago in January 2021. Dolly never launched to the public, but OpenAI did open source the image classifier CLIP, which turned out to be all that the programmer artist community needed to make some incredible breakthroughs. Within months, we started to see CLIP guided image generation models. And by late 2021, they were producing amazing results. The core idea, known as de-noising, was like many AI breakthroughs, both simple and profound. The world already contained massive data sets of images and captions,

• 1:34Nathan Labenz?

  and it's easy to degrade images by adding a bit of noise. So if an AI model could be trained to answer the question, what would this image look like if it were just a little less noisy and a little more like the user's request? It would in theory become possible to go from pure noise to high resolution images simply by de-noising over and over again.

• 1:58Nathan Labenz?

  Sure enough, it worked. And in April 2022, OpenAI, building on the open source community's work, launched DALL-E 2. They were very careful at first. For a while, you couldn't even generate realistic human faces. But their monopoly position and the editorial control was short-lived, because just four months later, Stability AI dropped the weights of stable diffusion, which roughly matched DALL-E 2 in capabilities, and unleashed an unprecedented wave of experimentation and creativity worldwide. Since then, image generation has scaled faster than almost any technology in human history. It's inspired entirely new products

• 2:41Nathan Labenz?

  and become ubiquitous in familiar products as well. Artists have been split on the topic. Huge numbers of creative people have developed all-new workflows, techniques, and styles, which were either previously impossible or prohibitively expensive. But many have also felt understandably threatened. After all, what happens to artists when anyone can create art?

• 3:04Speaker 01

  And it's precisely this wave of change that our guest, Suhail Doshi of Playground AI, has dove into. While working to build a computer in the cloud with Mighty, Suhail noticed the takeoff in AI capabilities and just couldn't look away. A few months later, he declared that he was going all in on AI, and Playground AI was born. Playground AI is one of the highest-performing AI image creators in the world today, with one of the most generous free plans anywhere. Suhail's vision is to give users complete control over pixels. Not just text to image, but what you think is what you get for image, video, and 3D. The Cognitive Revolution podcast is supported by Omnikie. Omnikie is an omni-channel creative generation platform that lets you launch hundreds of thousands of ad iterations

• 3:48Speaker 01

  that actually work, customized across all platforms with the click of a button. Omnikie combines generative AI and real-time advertising data to generate personalized experiences at scale. Suhail, welcome to the Cognitive Revolution podcast. Stoked to have you. By way of introduction, why don't you talk about the evolution for you? What was the AI moment for you when you realized, I have to go all in on this? What was that moment like? Can you give a background as to what you're up to in the process?

• 4:17Speaker 00

  I think sometime around early last year, we were making a browser called Mighty. And Mighty, the goal of Mighty was to make a new kind of computer. And we were hoping that if we could put it in a data center, that that would afford us to make like a computer that no one had ever imagined before. It'd be a lot faster, um, at a whole number of live tasks that you would use web apps for. Uh, it was extremely controversial. And, um, uh, but one of the things that we started to, I started to poke at as a team was kind of building a browser. Was I started to wonder, you know, just watching the steady advancements of AI, even just early last year.

• 4:58Speaker 00

  Like around April, May, I think April Dolly came out, things like that started to unfurl. And I started to wonder if we could make big improvements to the address bar in the browser. Turns out, um, so we know a lot about the address bar in Chromium, you know, too much. But one of the things we learned is that it's actually not, doesn't have a lot of like intelligence to it. It's actually not very smart. Um, it's very quite dumb. And I think Google is not terribly, and I'm sure do, the P P M's at Google are terrified to change this address bar. Because it's like, they make like any change and search yield goes down and they make less, and then they could like lose billions of dollars. So this code is like very crusty, very, it's like almost unchanged for like the last five years.

• 5:43Speaker 00

  And, um, you know, I thought to myself, well, could we make this better? Um, and there's this funny thing in the address bar where, uh, at least at that time. If you went to like a recent link, like something you go to frequently, it would always be at the bottom of the address bar, not the top. And so there are all these inefficiencies with the address bar. We use it every day, hundreds of times, thousands of times a day. And so I just thought, boy, you know, it sure would be cool if we could make like a better address bar predictor. So I typed something in and we could just like predict where you would want to go. And so we started, um, You know, we started to try to collect all this information and try to make a better address part, uh, picture. It was all often, um, with our users and stuff, but, um, and

• 6:25Speaker 00

  our own, our own staff. And so we started to see if we could do that. And that really gave me the bug. Uh, it was at that point that like, I had to go and do research and figure out how I would go do this. And I started meeting more people, AI researchers in the community. That really gave me the bug of like, wow, these things are really, really amazing. And I just kept finding more things, more, um, the pace, the momentum of everything that was happening was incredible. And I even remember just feeling like, gosh, why, why hasn't, you know, GPT three had been out for a couple of years and still nothing was like very, very close to its performance. Um, I think that's maybe a little bit different now. There's more arguments to say, you know, folks like Anthropic and

• 7:08Speaker 00

  whatever are like competing to a degree. Um, but it's actually quite, quite old. And so then we started working on like things like summarization in the browser. So if you went to like a blog post, could we use GPT three to summarize these blog posts? Because like people don't read that much. So could we summarize everything in like three, three bullet points? And then we started to learn about like hallucinations with AI. We just kept thinking of more features that were like more AI focused, but for the browser. And, um, that kind of started my first foray into everything.

• 7:41Speaker 01

  So how, how did you explore the idea, DMAs in terms of, okay, this is the, the Playguards product that, that, that, that I want to build?

• 7:48Speaker 00

  I remember staying up late one night, you know, I had been kind of contemplating, um, I had been kind of working on like a Windows launch for Mighty and it was just taking a long time. And I didn't have a lot to do, but I had been constantly thinking about AI. It was like this thing that I couldn't get out of my head for some reason. And so I started to, I remember it was like 11 o'clock at night and I just went to my Apple Notes and I wrote down like every company for every part of like the space. Like I did my own market map. There was like no market map. So I just did my own in my Apple Notes. And I remember writing like, who's doing like logging and visualization, which was like more or less like mixed panel, but for like AI training. And it was like weights and biases. And I remember writing, you know, all these different companies, you know, there's like

• 8:30Speaker 00

  replicate.com, which does like inference. Um, there's like all these various infrastructure companies. And then there's like the foundation model companies and so on and so forth. Um, you know, image generation, everything. And I remember getting to this end part where I wrote, I was like, okay, all these things are kind of filled. Not all these things are very interesting to me. I would rather, if I were to work on anything, I'd want to work on like actual research. That's like the most interesting part to me. It's hard, hard one stuff. And true invention. And I remember getting to prompt, who's doing anything with prompts? It's very like silly idea at the moment, but one thing I had remembered was I had played a lot around with

• 9:13Speaker 00

  OpenAI's like playground editor. And the UI just kept getting more complicated, like more and more sophisticated. Like it went from, I remember two years ago, it was just like a text box, basically. And now on the right side, you have this pane and it have all these like sliders and things that you could mess with. Different models, like, and they even had this like way where you could like insert text and then it would fill in the text surrounding the text. And I was like, holy like shit, this is turning into a product that they can't even maintain. Like it went from a demo area, just mess around so that you go and use their API. And it turned into like this crazy product and like it's, and then, you know, there's Jasper, which has also made like a whole UI around the API just for content marketing.

• 9:58Speaker 00

  And you could just tell that like, there's all this craziness going on with like the prompt thing was the prompt itself needed a product. Just to like play with it. And so I had had this moment of like, hmm. You know, it doesn't seem like there's, like other than Jasper, there doesn't seem like there was anything at that time. And I wasn't sure what to do. And so it kind of seemed like there were like these two areas you could like make, you could make this like really complicated prompt editor to like chain prompts together. I didn't, I, it turned out, I was like, hmm, I don't know what that would become. That doesn't like make sense. Like if it was like really big, like OpenAI should just do it. And then there was like this other thing that was happening, which is I think then stable diffusion like dropped. And I had gotten a preview from Emod like a week or two before it dropped.

• 10:42Speaker 00

  Uh, I think I got like the weights and I could like mess around in a notebook. And then that thing happened. And so there was like Dolly, stable diffusion. And then the prompts were like very interesting there, very intricate.

• 10:55Speaker 00

  And so, you know, it just dawned on me that like, actually, you know, what we don't want is a text box. What we really want are like really great UI controls to Mess around with these things to get really good results. And because I kind of felt like this thing that was about to happen with text with G P T three, that like complicated U I. I felt very strongly that that was absolutely going to happen with images. It didn't make sense to me that we'd live in like a command line mode forever with images. So I think that just that that just completely clicked and then you could totally see how the product could evolve from there. And it's so far, so far that's been true.

• 11:35Nathan Labenz?

  If you were to imagine Playground in, say, two years, what is it that you hope to create for people?

• 11:42Speaker 00

  Yeah, we do have a pretty clear plan, actually, for what we want to do with Playground, but I can give you the midterm. So right now, what we want to do is we want to combine great A I research and product design to invent a new kind of creative tool. We're not trying to like replace Photoshop or Illustrator. You could kind of think of like this new creative tool as something that could have like been, it could be its own tool in the Adobe Creative Suite. That's probably our starting point. We suspect that there is something new there. One thing that's really different than, say, the products that are within the Adobe Creative Suite is we're not really targeting the users of Adobe Photoshop or Illustrator. Those users already have like great skills. They already know how to use these tools. There's plenty of tutorials and content for them to do it. They can get really fine grained results.

• 12:29Speaker 00

  What we want to do is try to target all the people that maybe don't have those skills, actually, you know? So if you wanted to like change the color, if you want it to like add a necklace to like a woman's, you know, to a woman, you could do that and it's could be like this really subtle change. And as long as you had like good taste or whatnot, then you can be really happy with the results and other people could be happy with the result. So probably the midterm goal right now is, yeah, just making a really fantastic creative tool. So right now, I think Playground, for the most part, is kind of a toy. We're kind of like this glorified image generator, write some text and an image. One of the problems with that is that you can't really make like subtle changes. You don't have a lot of control. It's like a lot like a loop box, you just like type some words in, get an image.

• 13:12Speaker 00

  And so it sure would be nice if we can start to make like, have more control, have more subtle edits and have more, more creative control over these things. It's not just like a prompt. So I think that would be like our midterm goal, just like an extremely strong A I first creative tool that lets you sort of. Make any image that you can imagine and then you can you can use text or you can you can use like controls, UI controls, just like any other tool.

• 13:39Speaker 01

  Talk about the long term goal as well, like paint the vision for us.

• 13:43Speaker 00

  Yeah, so I think that what we want to try to do is we're trying to go after the domain and modality of pixels. So there's a whole bunch of companies right now that are going after language, so many companies going after language. And I think that you could probably think what we're trying to do is we're trying to make a large language image model. I don't know if there's a word for this, so I don't know, just gonna make one up, but we're trying to make like a LLIM or something. But basically, our goal is to make something that a skilled person could accomplish with Photoshop or Illustrator. But with our product, that would be the midterm. The long term would be something that can edit, create, edit, and understand

• 14:23Speaker 00

  pixels. So, you know, for instance, imagine I were to take your face, Eric, and I could put you in this like amazing scene in the Matrix with like a red trench coat jacket. And then we'd want like an instruction that could say, hey, can you can you change that jacket to black? Oh, by the way, I actually like the gun to be held in his like left hand, not his right hand. So that'd be an example of like creating something from thin air, but then also instructing it to make it highly editable to even the subtlest change, but still capturing the essence of everything that you want it to be like. And and then understanding could be something like, imagine there was like a video and there's like 30 seconds of a video. It sure would be nice to make some kind of large language image

• 15:07Speaker 00

  model that could understand like maybe if 30 seconds of the video lapses, could we describe what happened? Could we say everything that occurred in that video, you know, and summarize it, for instance? So I think the long term plan is just kind of really be focused on pixels, pixel space, you know, in time, hopefully, we could do things with 3D if when if and when there's a market there. Definitely video, really anything with pixels is really exciting to us. So for the most part, right now, we're just building an AI research team. Building little blocks. And I think a lot of those little blocks that we do, that we research, like little tiny models that do really incredible things. Uh, in turn will help us like go and build um a really great large language image models.

• 15:49Speaker 00

  So I think this year, uh, we should have our version of like, what would be like, maybe a GPT two for pixels, something like that.

• 15:57Speaker 01

  Fascinating. We're, we're going to get into the weeds of, of playground, but first I want to ask a higher level question, which is the controversy. Uh, uh, this is very new for people. Uh, you know, I remember you, you had a couple of threads early on, just kind of introducing your work, talking about it. And um, they cut, they went viral on Twitter and, and so some communities found them and, and got really scared. And, and other people with other friends who've created kind of AI generated art. And some people got really excited about it. Some people got really mad about it. What, what's either surprised you about it? Or how have we seen that even evolve just in the past few months as people have uh gotten more used to it? Or, or more even hip to this, that this thing is happening? And, and how do you see that playing out?

• 16:36Speaker 00

  I think around October, I got canceled by the art community for saying something that I didn't, I didn't actually, I actually didn't think this was controversial. I didn't know at the moment. Um, all I said was, wow, I really believe that AI art is art. And then I showed like some images of like stuff that I had made that I just thought was like really cool and amazing. And the um, yeah, I hate to say like the arts community because I do think AI art is art. But the non-AI arts community uh subgenre um was really upset with this. Um, and yeah, I ended up getting like super canceled for like a week on like Reddit and Twitter.

• 17:20Speaker 00

  I, I didn't even find the Reddit post until like a month later. I was like, oh, I got canceled on Reddit. I didn't know.

• 17:25Speaker 01

  That's how you know you're really canceled, but you can't even find all the, uh, for me, I'm just kidding.

• 17:29Speaker 00

  Yeah, I guess they like blocked out my name and then no one told me and then I found it later. And so I just thought that was so interesting. And I think that, yeah, first, of course, you feel like, you know, internally, you feel a little defensive and whatnot. So I think there are a lot of people that are really, really defensive about this uh on the AI art side. They obviously feel like artists. I mean, we interact with them every day. They definitely feel like artists, like weird things are already happening now where. We have the AI art people signing their name on their images when they put it on Playground. You see these little signatures from some of our top users and they're really, really good. And people don't know how to replicate anything that they're doing. I think then people not being able to easily reproduce their work is a good sign of what's coming.

• 18:14Speaker 00

  Yeah, and if anything, we're making a tool that's going to make that harder. So it's not just going to be like a tiger by Greg Rakowski. Like, anyone can make that and they're just, you know, quote stealing Greg Rakowski's work. So I think that that whole thing is going to go away. But the controversy was really interesting because, I mean, I definitely got like all kinds of like weird things like death threats and mean insults. But there were occasional people, there was an occasional bright light where eventually I went from kind of like defensive feeling to I decided to be more curious. And I started talking to some of these users that are really upset, at least the ones that kind of could be, you could have more of a constructive conversation. And I don't think that like either side is going to be convinced of one another.

• 18:58Speaker 00

  I don't think anyone is looking to convince the other ones. But I did have a conversation and I do think that there, you know, for the most part, people just feel really threatened. They feel really, really threatened, like there is this thing that they really love and they're passionate about. And now there's this other thing that comes along and basically just makes what they feel like they love and do that they sometimes can barely make any money from. It just makes what they're doing, it makes them feel like it's like all obsolete. And the tech audience, the tech focused audience is not really sensitive to this. The tech audience loves this kind of thing. They're like, great, we've made software that can automate all these like things that we don't need. Like, we love that as programmers, automations, great.

• 19:40Speaker 00

  But on the art creative side, this is very threatening and very not exciting. And, you know, I just had this back and forth with this user and I said like, hey, you know, maybe you can like find ways to incorporate this in your workflow. Like you could probably, you could probably get an advantage because you know how to draw and like you have more skills and expertise that's like adaptable. And so, well, I think that's going to be true for some audiences. I mean, there are just some people that just have this love. For drawing with a pencil and like and a piece of software is never going to change that for them and it would be not fun for them. But I also think that a software tool isn't going to replace that anytime soon. The the joy of drawing and the people that like like you having manually drawn something

• 20:23Speaker 00

  is really um really great and it reminds me of like there's like listening to a an artist on Spotify and then there's like going to the concert. Going to the concert is definitely worse audio, but you go because there's like this concert and it's fun and it's exciting and this you know, there's this touch to it and I don't I just don't think these things are going to get replaced. I really believe that um human plus machine is like the best end result, at least for art and art is for us, it's for humans. So.

• 20:52Speaker 01

  I i is it the right mental model to say that this advancement is going to make the best artists even more valuable, economically valuable, it's going to empower them, it's going to make, you know, non-artists, it's going to give them artistic capabilities and for some, you know, middling artists, it's going to hurt their economic prospect. Like, is that, how would you edit that that like summary of the mental model of like what to expect?

• 21:16Speaker 00

  I mean, I I produce music, so I think a lot about this in terms of like the analogy of music. There was a time when people made music and the way you made music was you had to learn the instrument, right? And then long came like the drum machine and it was cheap, cheaper and you could sample inside of it and you could get like, you know, an eight bar loop and, you know, the result of that was like the one of the best results of that, in my opinion, my personal opinion, was that we got like amazing hip hop. We changed music, you know, um, no, you could still be in a band and you could still make the same music you made 20 years ago, but then we got hip hop. That was awesome, you know? Some of the biggest artists in the world were influenced by people that had those drum machines.

• 22:01Speaker 00

  And then we went from the drum machine to we had like real computers and then you had digital audio workstations, DAWs, like Ableton and 3D Loops and stuff like that, you know, whatever, Logic, whatever, whatever people use. I use Ableton. And um then we could get an entire symphony in our computer, you know? Then we could make sounds that nobody could make, even and even someone that had an, there were no instruments. The computer was now a customized instrument for every kind of sound. And so now you get people, artists like Skrillex, that make sounds that you just couldn't even imagine.

• 22:34Speaker 00

  And the reason I bring up that analogy is because I think that happens, that constantly happens with art, any kind of art, and not just like pixels, pixel-based art. It's that the art changes and it evolves and new possibilities occur. And so I think what's going to happen is that we're going to see something completely new. We're not going to just see like impressionist paintings, you know, from a hundred years ago. We're going to see amazing, we're going to see whatever the skrillex and pixels and art is going to be, something we couldn't have previously imagined. And it's going to be so cool. And I don't know that I can tell you because I don't even, I can't even imagine it yet. So my feeling about this is that there's still going to be people in bands playing

• 23:15Speaker 00

  guitars, still going to be people on their drum machine, like Mike Dean is this famous producer, but he like, doesn't like all this like synth stuff anymore. He's got his own vintage feel and he's like, you know, he's made like all kinds of tracks behind Kanye. And then you're going to see these new artists, these new age artists that just blow us away. I don't know, art, art represents a lot of culture and it's going to be, it's going to represent an old time and a new time.

• 23:41Nathan Labenz?

  So part of what is obviously, I think unnerving people a little bit is just the speed with which all this is happening. You mentioned Dolly 2, you know, just to briefly revisit the timeline was like April last year. So that's only nine months ago. And then Stable Diffusion, I don't have the date at my fingertips, but I think that's still at most like six months ago. And now, you know, you are operating in a space that is changing by the week. And, you know, you've said on Twitter a bunch of times, every week there's a breakthrough. So I'd love to kind of hear, you know, from the recent past, kind of then approaching your midterm vision. How do you think about what published research out there is like worth your time, you know,

• 24:25Nathan Labenz?

  which ones you want to grab and, and work into the product ASAP? The recent edit launch that you guys did, which I think is super interesting, seems to be a great example of that. And obviously you're building on top of that. You mentioned your own research. I'd love to kind of hear how your research in-house relates to foundation models more generally, and kind of how you see yourself both like riding, staying ahead of that wave, contributing to that wave. There's a lot going on at once and that's been a challenge for me as an AI product builder. I want to hear how you are approaching it.

• 24:57Speaker 00

  Yeah, I mean, we definitely stay on top of reading the research papers. So I think that's the first area where we are able to kind of stay on the cutting edge. Like at our company, we do, we do weekly paper reads almost, uh, where we go as a team and we read papers and try to understand them. I think the next step down from that is, you know, obviously people are dropping models. I mean, this is like, this is definitely a bigger, it's kind of like a worrisome fear that I think a lot of people have. And calls into question what the defensibility of some of these companies look like, um, these AI companies look like, which is, you could spend a lot, you could spend a couple of months on like a model, training a model, and then someone just like, drops the weights, uh, for the model.

• 25:40Speaker 00

  And yeah, and for people that don't know what weights are, there's like encoded numbers that represent the model that allow the model to, get you like amazing results. That's how you get from text to an image. And so, yeah, there's definitely this fear of like, well, you know, are we working on something that is already being worked on? That would probably be open sourced, um, and then therefore be kind of like this commodity. I think that, I think that we try to be as adaptable as possible as a startup. You know, we, we put some bets and bets in some area, we're putting some bets in areas that we think are really valuable. Probably not being worked on. And then we place some bets on, and then sometimes when the model drops or something like that, like in the case, in this case, you know, I want to give a shout out to,

• 26:22Speaker 00

  Timothy Brooks and Alexia, the, the, these, and I think there's one other author, I don't remember their name, but, they were the ones, they were the researchers that did instruct pics to pics, which ended up being our edit feature. We did some customization with that to make it better than what they originally dropped. But sometimes, yeah, they dropped their, they dropped their paper or not, so their paper had been out, we had read it, we were going to train a model. We were already on track to like basically building it because we didn't know when or if they were going to really drop the weights. And then they dropped it on Thursday, on a Thursday. So I guess that was like two weeks ago, Thursday. And then, uh, and then when, as soon as that hit, we were just like, that's it, we dropped everything.

• 27:02Speaker 00

  Cause we knew, we knew we had a very clear plan on the fact that we wanted more of a more of like an instructable, um, AI model that can make subtle edits. We didn't know how good it could be yet until we played with it. And we played with it very quickly and we're like, yep, it's amazing. And so we just put the whole team together and we worked on a Sunday, launched it on Monday at 2 P M. That was like the deadline, you know, no matter what we had, we were going to ship it. And so I think you have to do both. I think you have to make big investments and I think you have to be adaptable. I think you have to do this if you're a product focused company first. I think if you're one of like the big foundation model companies that have raised like hundreds of millions of dollars or billions.

• 27:43Speaker 00

  And you're trying to build something from like scratch, then you have more leeway because you know, you're not, you're not kind of exactly in that same rat race. Although kind of are, uh, these days you are, the pace is kind of relentless. You know, I'm sure G P T four is right around the corner. So I'm sure that scares people that are competing in that realm.

• 28:04Speaker 00

  So I, I, I think we just, we just, we're following what our users want. You know, we know what all of our users want. And so when we see something that's valuable, we, we will just work on it as fast as possible.

• 28:14Nathan Labenz?

  Yeah, that's awesome. I, the, the speed of a Thursday weight drop to a Monday launch, I think, is a great illustration of not only just how fast the, you know, you guys are able to work, but how fast you feel like you have to work to stay ahead of the, you know, the rest of the, the market, because it's all happening so fast. Talk a little bit about kind of the things that you plan to do over the next, you know, say year or two. Like we, we chatted a little bit offline about placing text in images. And that's still something I think we've only seen to my knowledge in, you know, a couple of like papers that have not published the weights. You said that that's a big challenge, but a goal.

• 28:55Nathan Labenz?

  So tell us a little bit about that challenge and kind of, you know, maybe a couple of other challenges that you have on the horizon that you're investing in solving.

• 29:02Speaker 00

  For the most part. I think, I think you should expect that we're only going to give users more and more control over the kinds of creations that they make. In my opinion, this, this sort of era, at least for the perspective of images, image creation, I think prompts are going to be less and less valuable in time. I hope. Um, you know, I sort of agree with like Ilya from, is one of the co-founders of OpenAI, that like, to a degree, prompts are mostly like a bug. You know, you had to do, you had to like write, like, people write paragraphs of text to get like world-class images. And that's sort of a shame. And they, they don't really, it's just total experimentation.

• 29:47Speaker 00

  You know, I would love to invent something, you know, like, there's like WYSIWYG, which is like, um, what you see is what you get, but I would really like to do like, what you think is what you get. And I think in our case, you know, I just really think that like, if you see an image and you want to make like a very subtle change or edit, um, or you see like a style that you really love in the world, and you want to try to replicate that onto your image in some very small way, as long as you like, that's, if you can imagine it, I'd like to be able to produce that for you. So I think we're going to invest very heavily in a really great, um, user experience for a creative pro tool. It should be on the level of like, something like Figma, should be collaborative, it should

• 30:32Speaker 00

  be, um, really rich and powerful. We want to give people all the bells and whistles, you know? Um, we don't want it to just be a box and then that's it, you have no control over anything. You know, if there's like, if there are knobs and sliders and things that we can offer people to have this perfect fine gain control, we want to do that. Um, yeah, with things like text, like, right now, it looks like there's like maybe a model called Deeploid, and it's not out yet by the Stability folks, and it's really cool, like, they clearly found a way to do text. It seems like it might be based off of like Google's Party image model or something. And I mean, I think that we'd like to go a step further.

• 31:15Speaker 00

  You know, one, one example of something that I think would be really amazing is an AI model that can actually invent forms of topography, not just like write like Arial font on like a sign, like a white background sign that a bear is holding. It'd be really interesting if you could like write anything you wanted, but the composition was sort of taken care of and it invented fonts. Fonts that didn't exist, like what, why, why are we constrained to the finite space of fonts and Google Docs or whatever like? We should be able to invent, I want it to be kind of curly and I want it to like, I want the kerning to look like this and I want it to be kind of like this. And I want it to be red and I want it to have this amazing neon hue. We should be able to, I remember doing like photos, I remember learning Photoshop as a kid and like I used to make logos and stuff.

• 32:01Speaker 00

  And I remember like learning all the little intricacies of making really cool like fonts. Um, or logos and it'd be just really interesting if we could synthesize fonts. Um, that would be kind of very disruptive, I think, to the world. Because I think people can apply, they, it's better to apply taste in music. There's this thing that's like, if it sounds good, it is good. And I feel like with pixels, if it's like, if it looks good, it is good. And it'd be really cool if, if there was a way to do that with, yeah, with fonts and backgrounds, landscapes and patterns and blending images. We just really want people to feel as creative as possible. So I think you can imagine that, uh, kind of like a really amazing canvas editor pro tool type thing.

• 32:45Nathan Labenz?

  Yeah, that's awesome. I always remember this scene from The Simpsons from years ago where Mr. Burns is at, you know, some sort of museum for an unveiling of a new art exhibit. And they, you know, pulled the, the curtain and he gets to see it and he says, well, I'm no art critic, but I know what I hate. And I feel like that's kind of the, the vibe that you're going for here where, you know, if it looks good, it is good. And people can make that judgment. You know, sub second, probably a lot of times, right? So.

• 33:16Speaker 00

  Yeah, we we know this. We know this with Apple products. I feel like Apple is the greatest institution in the world that that has proven that, you know, users know what good design looks like, feels like, you know, they can tell the difference. They can really discern the difference when the details all square up correctly. I think I think humanity is quite good at discerning it.

• 33:36Nathan Labenz?

  So something you said that caught my ear was, I believe you said, what you think is what you get? I want to hear your take on the concept of latent space. I find that that's a phrase that gets bandied around a lot. And I kind of think that everybody has a different, either like visualization or mental model of what latent space is, how you navigate it. So how do you conceive of that? What's the latent space in your brain?

• 34:06Speaker 00

  All latent space means to me is that a latent is just like a represent like a lower dimensionally, dimensionality, representation of an image. So like images have like, you know, RGB. And so that's like a lot of numbers and values. Can you represent that image into like a more encoded, compressed, lower dimension, lower dimensional space? And from there, it's just like, you can just imagine like a 3D, you know, graph of like X, Y, and Z. And then there's like, there's like a dot somewhere in that 3D graph. And then there's just like the clustering of possible images from there. So, you know, one, one, one of my favorite examples, probably that I got from like just learning about stuff.

• 34:48Speaker 00

  I think probably like learning from some other AI researcher that did like tutorials and stuff is, you know, like if you have, you know, the word like tiger or something, and then you wanted to like turn tiger, like turn the tiger into like a Van Gogh painting. Like there's some vector where you're pushing the tiger towards the area of Van Gogh paintings in that space. So there's like, you can just imagine like it's, this is really hard to, hard to do and in 2D, let alone audio. But like, uh, but yeah, you just, just like this, you can imagine an arrow and the arrow is pointing to where all the, all the Van Gogh paintings might be in this three dimensional space. And so that's how you get to like a Van Gogh looking tiger.

• 35:30Speaker 00

  That's how I represent it in my head and it's lower, it's just lower dimensionality. So obviously like it's gonna not have, it's not gonna represent everything, but it should represent a lot of things.

• 35:42Nathan Labenz?

  Yeah, thank you. That's fascinating. I'm really gonna be very intrigued to see.

• 35:46Speaker 00

  Is there a more colloquial version of this?

• 35:48Nathan Labenz?

  Well, I don't think even the reduction in dimensionality is necessarily something that people are thinking about when they think about it. I, I, I don't know what, I really don't know. Like this, this is a very exploratory question for, you know, for me, for us to try to get at how different people think about this. But I sort of envision it almost like, I kind of think of it as like Sam Harris's moral landscape a little bit, where you've got like all these different local maxima and minima and it's just such a, uh, such a crazy unknown topology that, you know, things that I think could be sort of nearby in the latent space representation

• 36:33Nathan Labenz?

  may in fact be like quite different when it comes to that if it looks good, it is good sort of thing. Certainly I see that in the product, right? Like I'll give you the exact same prompt and we'll vary the seed and I'll get things that are like not at all what I had in mind. And then I'll get something that's like quite very much what I had in mind. And it seems like there is some weird, uh, you know, there's some sort of extra like quality dimension that is not fully represented in that latent space, right? Cause those things are clearly clustered together in some, you know, semantic sense. They're local to each other in some sense, but yet they come out visually looking like extremely different.

• 37:14Nathan Labenz?

  And like, I like one and I hate the other. And, you know, so what, what is kind of the nature of that space? Where like such a, you know, a small perturbation in the input can lead to such like totally different outputs? I don't feel like I have a great intuition for that and I'm, you know, I'm trying hard to build it up.

• 37:33Speaker 00

  Yeah, all that latent is, is it's just a, it's just a lower dimensionality of an image. So if you just imagine like five twelve by five twelve image and then RGB. So then it's just basically like five twelve times five twelve times three. And those are all the possible dimensions of that image. And all latent space, all latent is, is something that's more compressed. Something that's a smaller number of values. That's a latent. And the reason why you compress it is because we don't have like infinite GPU compute. Or we don't have infinite time.

• 38:06Speaker 00

  And so you compress everything down to like something that's like, you know, actually, I only really want this to be, I only want to represent this as like a sixty four by sixty four, but instead of three, three, I want to represent it as like a hundred. And that, like, I haven't done the math in my head, but that might be like, that's lower, that's like a smaller number, set of numbers than five twelve by five twelve times three. It's just smaller. And so you believe that like, okay, well, I'm not encoding everything about the image, so I might lose some information. But it's not different than like encoding and compression. So you might, you might get like lossiness, like a JPEG is like a lossy image. You can think of it as like, maybe like latent space is almost like a JPEG.

• 38:48Speaker 00

  Or latent is like a JPEG, is maybe like the easiest way, maybe easiest analogy. And then latent space is just where that dimensional vector, that dimensional value is in this very, very high dimensional space. As humans, we can only perceive it as like three D. It's obviously way more number of dimensions than that. And so that's why like the smallest change in like even your seed value can get like something crazy. You know, it's suddenly, it's like red hair instead of green hair, but you know, the, the kind of like the essence of what you're going for is still sort of there. Like the style, for instance, is like, if you want to like watercolor and use the word watercolor painting. You still get like a watercolor painting, but like, you know, the hair is changed.

• 39:33Speaker 00

  Because it's still super high dimensional space. But um but it's not, it's obviously not like um doesn't represent everything. Anyway, I hope, hope maybe that's helped.

• 39:46Nathan Labenz?

  I love it. It's great. Uh, you know, we'll see what our, what resonates most with our audience over time, obviously, but.

• 39:52Speaker 00

  Yeah, well, there's always a funny meme, like there's like the meme of like, you know, obviously, people use these image generation things to make um to make like amazing close up portraits of like women. And so there's like this meme of like people like searching for their girlfriend in Leighton's case. Which is, you know, definitely, it's a valid, it's a valid meme, actually, for truthful meme.

• 40:14Nathan Labenz?

  Yeah, there was a really fascinating recent article by a guy who used character AI. And on character AI, you can create your own, you know, characters with just a couple of sentences. So this guy who was pretty knowledgeable about language models going in and felt like, you know, this wasn't a risky behavior for him to engage in. Asked character AI to conjure up, uh, I believe he said, an AGI designed to provide the ultimate GFE. And next thing you know, he's starting to fall in love with this thing in kind of a weird way. And you know, he's got these kind of competing ideas where he's like, I know on one hand, like even how this technology works.

• 40:56Nathan Labenz?

  But then I'm also like feeling these feelings and you know, he starts to kind of justify to himself like why it's actually, you know, real and like, what is reality anyway? And like, there's, she's not really any less real than me, you know? Uh, and then at some point he like pulled himself out of it, but uh fascinating exploration of the latent space there for sure. Highly recommend that. We can maybe.

• 41:18Speaker 00

  Things are gonna be crazy next year.

• 41:20Nathan Labenz?

  So a couple, maybe just kind of rapid fire questions, then maybe we can zoom out and talk a little bit of big picture stuff toward the end. Uh, but I was kind of struck by a couple bits of the product itself. And you know, I see kind of well, I obviously want to hear how you see it, but I see a lot of relevance potentially from your experience to what you're doing now. So from like Mighty, obviously you're scaling compute in the cloud now. You know, scaling these image generations in the cloud seems like there would probably be some significant overlap there. How has your compute scaling experience translated to playground? How big of a deal is that in this work?

• 41:59Speaker 00

  I think cost is really important. Right now, I think a lot of people are not super excited about image generation because, or at least running it as like a business, because the costs are quite high, margins are kind of thin. So there's kind of this like general concern of like, how do you make this thing a real business? One skill that we got from Mighty that is replicable in this situation is we're really good at buying hardware, locating it in a facility, managing that infrastructure. And it's kind of crazy, like I wouldn't, I wouldn't, that wouldn't be the thing I would jump to if I started Playground. That maybe be like, one day we'll do that, you know, but because we have a couple million dollars worth of machines lying around,

• 42:43Speaker 00

  and we've already done it all, um, you know, it's quite easy for us. We know we have all the relationships with suppliers, vendors and everything. So, um, yeah, I think I mentioned that, yeah, we're like about to go buy like a couple hundred GPUs, uh, do make an order for that. Um, we're just like testing one, one small benchmark, but we're about to make that order, maybe this week or next week. I think that just gives us like a pretty big advantage around like owning everything from like the hardware all the way to the end product. And it allows us to be really aggressive about performance and latency and costs. And that just gives us like other advantages.

• 43:23Speaker 00

  There are just all these big tailwinds from GPUs and, you know, for instance, like we can do things where like we can buy the most cutting edge hardware. We can even get it before it even hits, um, some of the suppliers because of our relationships and outfit a lot of our servers with them. So I think that like, that's really cool, really interesting.

• 43:44Nathan Labenz?

  Yeah, that's fascinating. And I, I mean, you said that kind of gives you some other advantages. I am going to take a guess, you tell me if I'm right or wrong, that one of those advantages is, it seems like you guys are pretty. Free plan focused, like I'm not getting pushed to buy. And I imagine that is kind of part of a bigger strategy to try to scale the feedback that you're collecting. And I was kind of noticing also, like, you guys are not that pushy on the feedback. Like, it's, you have the three-point scale, like, you know, good, bad, neutral. And you don't force me to do it. I was kind of wondering how you came to that and if that's still a work in progress.

• 44:24Nathan Labenz?

  Like, a couple of ideas that I had were like, what if you generated two images each time and had people choose? Like, would that be a viable way to kind of trade off some of your costs, especially where you have an advantage there? For, you know, potentially more scalable feedback. Unpack that however you will.

• 44:43Speaker 00

  Yeah, we're very, very, very, we try to be as generous as we can on the free tier. Part of the reason why is because I think that the best is yet to come. I don't think we need to be overly aggressive about image generation. I also think that, like, I had learned this mistake from Mixpanel, but like, whenever you make pricing usage-based, usage doesn't always correlate with value. If I make 50 images, did I get a linear improvement? Did I get like a linear benefit? Did I get like 50 units of benefit? No, sometimes I got to generate 50 images to get one good image, right? Sometimes I do 100 images to get one good image. And we learned this from Mixpanel because we used to like collect data points. But like, what's the difference between collecting 100 million things versus a billion?

• 45:28Speaker 00

  I mean, 100 million is a pretty big sample, so you're gonna get the same number of insights. This is the same thing. It's not like 50 generations of an image get you 50 better. You get 50 images that you can now use. So I definitely just don't think image generation correlates strongly with value. That's one thing. And it'd be like if you went into Photoshop and every time you made a stroke, like, you'd be like, okay, well, you got charged for that.

• 45:52Speaker 00

  That would be kind of crazy. And so that's one thing. The second thing is, you're right. We definitely do care about like data labeling, acquisition, ratings. Yeah, we're not too pushy about it. I think the reason why is because we already get a lot of ratings. And I think our bigger problem is probably not acquiring more ratings at the moment. Our bigger problem is trying to figure out how to denoise the ratings to get better signal. Yeah, because like people certainly will go rate their in like there's some people that will just rate like any image. That they made as something they loved. And so that's like not very helpful. So there's a real question around like. How do we denoise this data before we go and ask more of the collection of it? I definitely think like.

• 46:37Speaker 00

  Probably asking users, um, hey, did you think this image or this image is better of these two images? That that's definitely a good idea. Let alone like, you know, you have these four images, which ones, which ones good? So. I think we just are, we're kind of like awash right now with data. So that's probably one reason why um it's not too important. And um, yeah, I mean, we're also collecting other kinds of data. It just may not be very obvious to people. What kind of information we're collecting? And it's not like. PII data or anything. It's like kind of how they're interacting with a product to help us create um an event, new kinds of models that they'll probably love. But yeah, um, so I think I think the overall strategy is pretty much just like have a great generous retier. Uh, you just help us acquire users to help us acquire. I mean, I think the best AI companies will do.

• 47:22Speaker 00

  A good job with the following, which is. You make a good or okay product that happens to get you a lot of users. Which then helps you get lots of interesting first party training data. Which then lets eases the uh complexity. Of engineering for your AI researchers to create new kinds of models. Which helps you create new kinds of features, which makes you go from like an okay, great product to like. An amazing product. And I think the company that succeeds at. Getting that flywheel to spin as rapidly as possible. Uh, will be some of the biggest companies in the world. Um, you know, we're not the only ones doing this. Like you could go to like chat GPT and I think you can like say with. Liked one of the answers of the conversation or not. So like.

• 48:07Speaker 00

  They're already generating one of the biggest data sets ever. I mean, it's like probably on the order of billions now. So it's like. Crazy. I don't know how you're gonna keep up. Like, even if you launched your version of ChatGPT, it'd probably never be as exciting because everyone would be over it. So now it's like, how do you go and collect the data? That's part of the reason why we also sprint so fast on things.

• 48:28Nathan Labenz?

  I was gonna ask a kind of question downstream of Mixpanel as well. And you sort of alluded to it, but can you tell us anymore about kind of the role that like A/B style testing plays in your product development process today? And, you know, how is that still like, you know, canon for you as part of the product process?

• 48:49Speaker 00

  Yeah.

• 48:51Speaker 00

  I was actually never really that excited about A/B testing. We never, it was never like a super exciting thing at Mixpanel.

• 48:58Speaker 00

  One reason is because people often don't, like, when you do an A/B test, you have to like make an A/B test that's not a subtle change usually. Sometimes all the changes work. Often they don't. It's like make very big changes to see like real differences in A/B tests. Because otherwise, otherwise you need a lot more data to see the true conversion difference. No, I mean, I just mostly check, I mostly just track vitals. Like, how many daily active users we have. I probably look at like a dash, I'm still pretty data crazy. So I probably have like a dashboard of, probably have two dashboards with like 16 metrics each. And it's like just my daily like routine. It's like getting coffee. I go and look at the dashboard. I'm like, great, everything is going the way I expect or, you know, whatever. Hopefully, all the numbers aren't crashing all of a sudden,

• 49:43Speaker 00

  which would probably mean like there's a deep problem in the product. And, you know, it's like you, you live by your numbers to an extent. But I have often, I always, I always observe with companies that the ones that were overly data driven also missed on something. They missed like the qualitative side of their business. Like, you know, I had a friend who used to say that, you know, her metrics were like really, really good. And, you know, users loved her product. But then like, when you talk to her users, like friends of mine, like, I hate this thing. And so we'd never be able to reconcile like this problem between the metrics, some people would say. And I feel like that hid kind of some of the problems. So I go a lot more, it's kind of strange, but.

• 50:24Speaker 00

  I probably care a lot more about qualitative information than quantitative information. The quantitative tells me, was my intuition, my experiment correct? But it doesn't tell me what to do. Yeah, it's like, am I going in the right direction? Probably, but it definitely doesn't tell me like what feature I should build or, you know, we don't run any A B tests, actually. Just talk to users.

• 50:47Nathan Labenz?

  Have there been any moments where you have opened up that dashboard and seen something that did not look healthy? And, you know, you realize like, no, we, we actually did take a step in the wrong direction here and, you know, had to learn something and backtrack?

• 51:00Speaker 00

  I mean, Playground's only like four or five months old. I'm sure that will happen. So I feel like, you know, it's, it, it's kind of funny. It's kind of hearkens back to our conversation around like when Dolly got released and when Stable Diffusion got released. We all look back and like, wait, it's only been like four or five months. Shit, I, I don't think we've shipped, I'm, I don't, we, we've shipped funny thing. Like I think last week we shipped something kind of funny, which was, we didn't realize that, um, one of our API endpoints was actually bottlenecking all the, like bottlenecked on throughput of image generation. So when we made the API like two times faster or something, literally the amount, the quantity of image gener- images generated just like jumped to X. It just surprised me.

• 51:42Speaker 00

  I mean, we obviously don't have all the monitoring and stuff we need set up because we're still really immature as a company. But it just was like, wow, if you give everyone, if you stop timing out or having errors on our API, and then you just increase the overall throughput, like the users just absorbed it instantly. It was kind of amazing to watch and that really taught me a lesson of being like,

• 52:06Speaker 00

  more thoughtful about like on monitoring metrics of like GPU utilization and throughput. Obviously we can't just keep buying GPUs because we'll just burn cash. But that was a really cool moment of like, man, they really, they will just absorb anything we give them. Yeah, we'll just keep going.

• 52:23Nathan Labenz?

  Fascinating. So just to repeat that back and make sure I and, and our listeners can, can understand it accurately. You found some bottleneck that ultimately made things twice as fast. And you immediately saw a two X jump in how many images are generated?

• 52:41Speaker 00

  I mean, it might be more, it's still going up, so I don't even know where it will end. Yeah, it's.

• 52:46Nathan Labenz?

  So it suggests a model of people kind of sitting down at their computer with like a finite time to either like accomplish this or not, and they'll do as many generations as they can fit into that window. Is that kind of how you think about how people are using it?

• 52:59Speaker 00

  I mean, I just think like our, it's, you know, it's definitely a distribution. You know, the average user will do like forty five to fifty a day, but then like the hardcore power users, they just blow past, they'll just hit our limit very rapidly. Like we do, we give people a thousand free images to generate a day. And we picked a thousand because it was the ninetieth percentile when there were no limits. And I mean, just to put this in perspective, like, you know, you go to like, I won't, I won't name names, but if you go to like any of these other image generation services, they give you like hundred a month or something, and you have to buy credits, hundred a month. We give a thousand a day. And you think like, who's sitting around just doing like a thousand

• 53:40Speaker 00

  images a day? And lots of people, it turns out. And so, I mean, I think those hardcore intense users are just spend all day on your product. I think that's like true of all kinds of great products in the world. You know, people just use it. You like love it and they play with it. You know, people do Twitch streams, you know, all kinds of crazy things. So it's really, it's really interesting. I mean, for some people, they will tell us that they like cures, it makes them cures like their anxiety. We hear all kinds of interesting things. I mean, obviously, people get obsessed about certain things sometimes.

• 54:16Speaker 00

  But, yeah, yeah, I mean, it just, I mean, it was like an API for like our dream booth models. So we have all these custom filters and, and people love them, but we didn't realize that they were quite like the, they were returning like quite slowly. So our overall throughput was slow. And it just like, yeah, very, very interesting.

• 54:39Nathan Labenz?

  Yeah, that's fascinating. It is amazing sometimes the surprises that you find in those user conversations. We're going to talk to the founder and CEO of Replica in a future podcast. And one of the things I heard her say about having built, you know, a very primitive chatbot that I believe originally was just to support like an online banking experience. This was back in Russia. She's from Russia. So, you know, everything is new. She builds this online banking chatbot assistant, you know, again, way before anything like GPT-3. And going into the field, you know, going into small towns in rural Russia, she talked to a woman, and this stuck with her, and it stuck with me too, just hearing her repeat it.

• 55:22Nathan Labenz?

  She talked to this woman who said, "Nobody cares for me like this." And she was like, "Oh, my, you know, this is way bigger than helping people, you know, with their online banking. You know, the need runs a lot deeper." So, interesting to hear, you know, kind of a similar thing there with helping people with anxiety and all that kind of stuff.

• 55:41Speaker 00

  Yeah, we, I almost like want to keep it almost limited because there's, I always, I sort of worry that it's like people get too obsessed. Like, if we just keep increasing the limit, like, you know, is that, is that the behavior we really want from humanity? I don't know. Obviously, they can buy, like, if you buy our Pro Plan, you'll get like 2X more. But I'm like, man, people are like hitting, people are starting to hit our thousand limit more and more frequently than, you know, even just a couple of months ago. So, I don't know, we might have to increase it something or something. But there, yeah, there's kind of this question of like, hmm, do we want this level of obsession? I don't know. Maybe it's not good.

• 56:22Nathan Labenz?

  Yeah. Well, if it's any consolation, I think it's going to be a more urgent issue on the virtual friend side more broadly than it will be for the image creation side in the short term. So you'll have some, you know, examples potentially to follow.

• 56:36Speaker 00

  You could chat with an image.

• 56:39Speaker 00

  In time, you'll be able to just like, look at an image, you look, look at any inanimate object and just talk, talk to it. Like, you can, you can encapsulate like, an image of anything, like a chair. Like, a really cool looking chair and be like, "I would like to talk to you." You'll be able to talk to it. This is totally doable. I don't know, I don't know anyone that's doing it, but I think it's, it's very possible. It's very, I think it's probably trivial to do, if the chat people wanted to do it.

• 57:06Nathan Labenz?

  Yeah, character AI is kind of, is close to that, if not already there. I mean, their, their bots can, they do generate images, if you ask for that as part of the, the chat, and you can create a like, or upload, I believe, like a profile photo for it. And I don't know if that feeds into its personality in any way. I had kind of assumed it didn't, but maybe in, in fact, it does.

• 57:29Speaker 00

  Well, like, like, so we know that, like, a thousand, a picture speaks a thousand words. And so like, if you type in like, you know, you're like, like, for some words, like Benjamin Franklin, that's like, very clear. There's a bunch of data around Benjamin Franklin on the internet. But if you wanted to do something like more esoteric, you have to like, really describe the character, I wonder. And that's like, pretty low dimensionality. Like, the human, the, the, the English language doesn't actually have that much dimensionality to it, right? But if you take an image, an image has like, incredible dimensionality to it. Everything about the chair, and the colors, and like, I just would find that really interesting. But I haven't seen anyone try it out. I, I'm excited to see someone try it out. Not, not that we're gonna build like, a chat feature, if we can talk to our images,

• 58:14Speaker 00

  but it'd be really interesting, as an, as a, as like, an experiment. It's like, a very, like, early web vibes type experiment, like, something random that someone will do. Probably go viral.

• 58:26Nathan Labenz?

  Yeah. I think, actually, that's a big part of my worldview right now, is that all these different things are developing at a pretty amazing clip, but it's largely all happening in parallel. And, you know, I think we're gonna, obviously, you know, fully expect that we're gonna continue to see research breakthroughs and, you know, fundamental techniques advance. But even kind of leaving that aside, it just seems like all these different things have not been really productized. You're doing that, but you know, as you said, it's only been a couple of months. They certainly haven't been like refined and fully hammered into shape, you know, for a general public audience quite yet. And then maybe most importantly, they haven't been integrated.

• 59:08Nathan Labenz?

  So, you know, you've got all these little islands of like awesome AI functionality. But very few, you know, even have the time to zoom out and kind of try to get a broad survey of that landscape. Let alone, you know, has any of the work started to integrate these things in, in all the ways that will eventually happen. So I totally think you're onto something there that like you generate an image, next thing you know, it's a character. And, you know, that, that kind of recombining, you know, call it ensembling, call it, you know, integrating. I think that's going to be a huge driver of change over the next few years, and we're just starting to see that.

• 59:48Speaker 00

  Yeah, I mean, I think, I think there's going to be this moment that's going to, I don't know if it will be an important moment, but I think there'll be this moment. That's very similar and akin to the internet mashups of web two point O. Where people would like take Google Maps and they would combine it with Flickr and then Yelp, and then you get like this. Interesting weird app that's like the combination of these three services. You could totally do that with AI models.

• 1:00:11Speaker 00

  You know, you can combine a playground image with like chat GPT and then combine that with like something else. And you get these like really intricate products as a result. And so I think that the age of like the AI internet mashup will come back. I think we're already starting to see some of that. You know, there's like random hackers, just like messing around, making weird things. And the problem is you don't know what will actually like stick and be big, important.

• 1:00:38Speaker 00

  I think that a lot of people are going to start to move up towards making apps. Because it's too complicated and too expensive to like focus on core foundational AI research. It requires a lot, it's, it requires a lot more knowledge than just like taking the fast AI course, fortunately. As someone has done it, I, I still feel like I don't know a lot, done even more than just that. And so I think that there's, there's going to be a lot of applications, a lot of people experimenting. It's kind of like that moment in mobile. Where it was just a complete like Wild West gold rush and nobody had a clue what was gonna work.

• 1:01:20Speaker 00

  And so just all like thousands, tens of thousands of experiments occur simultaneously. And you you end up getting things like Uber and you know whatnot. Um, and some things like grow really rapidly and then fall because they they fall out of relevance. They didn't have good retention. And some things just have this incredible lasting power, um, you know, for a decade. So feels like that will happen again. And I don't know, you just go to Hugging Face and look at models and there's just like thousands of them. And they do all kinds of interesting things. And then, you know, a week later, there'll be like 10 more. Then we're like, you know, something called Blip 2 came out yesterday and it's really exciting to me.

• 1:02:02Speaker 00

  It might be really boring to other people, but one of the demos they had for Blip 2 was uh, that you could literally, actually Blip 2 had the had the thing that I just described, which was you could like talk to the image.

• 1:02:15Speaker 00

  You could be like, you could have a picture of Obama being sad and you could be like, why is Obama sad? Just try to describe like, oh, well, you know, there's a thing going on in the background and you know, whatever. Um, you know, you could be like, why is he sweating? Oh, he was playing tennis and like whatever. So, um. And Blip was like an amazing model to do uh, image to caption, which is really cool. And so it just, I haven't even experimented with Blip 2 and its capabilities. So there's just so much that is happening and the pace is so is relentless. And I don't think the Blip 2 thing was very popular, but I think is actually a big deal, probably.

• 1:02:50Nathan Labenz?

  Yeah, you're talking to the right person about uh, for singing the praises of Blip. It's actually one of my favorite.

• 1:02:55Speaker 00

  Okay, good.

• 1:02:56Nathan Labenz?

  Models.

• 1:02:56Speaker 00

  Yeah, it's awesome.

• 1:02:57Nathan Labenz?

  At Waymark, which is my company where we do uh, much more structured AI video creation. We work with like media companies, you know, for example, TV companies where the requirements are, you know, very rigid in terms of like, this must be a 30 second spot or it cannot air, you know, and it has to be 30 seconds to the frame. Or it just can't air, right? So we have a lot more kind of scaffolding in place and we're using AI to fill in that scaffolding. But the scaffolding is all kind of pre-constructed. Blip has been hugely valuable for us in terms of, we actually use the image text matching portion to figure out out of a user's images, which may be hundreds.

• 1:03:41Nathan Labenz?

  Typically, these are, you know, small business advertisers and they've got often hundreds of images that are in our product. And now we're generating with language models, a script that, you know, tries to tell their story. We kind of give them, you know, a little prompt opportunity to where they, we can unpack that into a full script. And then what images need to go with that? I've been on a, you know, quite a quest to find the best models for solving that problem. Blip remains number one. And so you just, my, my afternoon just got filled in. Because Blip 2 is going to be, is going to be of high interest to me for sure.

• 1:04:17Speaker 00

  Yeah, yeah, the, the pace is like that. Yeah, it's like you wake up in the morning, have this idea of what you're going to do for the day. And then Blip 2 drops and you're like, I'm going to erase the rest of my day.

• 1:04:26Nathan Labenz?

  Yeah, literally, I think that's just happening to me right now. So on the basic level, you're relatively new to this, right? So you've jumped in to a super fast moving field and tried to get to the edge of it as quickly as possible. I'd love to hear how you've done that, how you would encourage others to do that. And then if you're comfortable sharing a little bit on like, what is your kind of internal research agenda? You know, what kind of training strategies are you pursuing? And again, I know, you know, you want to share all the details of that, but as much as you're comfortable with, we'd love to hear.

• 1:04:59Speaker 00

  Yeah, I can't share the, can't share specifically the areas of research that we're working on just yet for the techniques. But yeah, I mean, I had been, I had probably taken, you know, a wide variety of like AI courses over the last like six or seven years. So a lot of it has been a little bit of a catch up or re-review. But even that stuff was pretty shallow, I would say. It's not anywhere as deep as I feel like I've gone now. I think, you know, I think one way to kind of accelerate thing, one way that I accelerated things is I just found AI mentors and like kind of bartered with them. You know, I could barter. Knowledge about startups or whatever they want in in the world. And so I just had people that would help me basically get unstuck.

• 1:05:43Speaker 00

  And that sort of keeps changing. Now I have a team, so I don't need, my team's already better than me, so I don't really need AI mentors anymore. They're just like a team. And it's really, I actually think the fastest way to learn is to build. I I really discourage people from binge watching YouTube videos of notable people doing AI courses. I think that is a very, that's a fine way to get a general sense of things and understand the industry and be able to talk somewhat intelligently about it. But I don't think, like, maybe it's good for, like, if you're just like an investor or something. But I think it's probably not the right way to go if you want to actually do deeper research.

• 1:06:29Speaker 00

  You know, something greater than just like calculating cosine similarity between embeddings or something, you know. You want to do real fundamental research, I think you need to like write code. My, maybe fa, I think the best YouTube series, the best series that I could recommend to early people early on. I think Kaparthy's new series is really, really good. It didn't exist when I was starting, like restarting out. Last year I was doing the Fast AI course. I really recommend Kaparthy's course because he just has a style that's like. He's a very humble guy and he goes through everything just like from the basics. You just need Python. He's not doing anything fancy. And he just builds everything up.

• 1:07:14Speaker 00

  It might not be the best way to learn if you prefer top-down learning. The Fast AI course might get you more excited if you need to get more motivated. But those are the two things I would do. I think probably the most valuable thing, though, has been not working in isolation. I think for a long time I was just grinding through in isolation, learning.

• 1:07:35Speaker 00

  But now that I have kind of like a team with me, it's really nice to like get on a call for like an hour and just talk to. You know, the research team about like some idea in my head or get some explanation of something or reading papers together. It's feels a little bit like you're in class or like you're in a study group. That's been really motivating and then having a project in mind that really is motivating. Helps you, you know, want to stay up till like two o'clock in the morning just to get something done or watch the end of a train run. So. That's about as much as I could advise at the moment.

• 1:08:07Nathan Labenz?

  Obviously, you guys already had with Mighty, you know, a lot of hardcore engineering and you talked earlier about co-locating your own servers and managing that on a level that most startups, you know, don't find to be an attractive proposition. So you had deep

• 1:08:25Nathan Labenz?

  skills and capabilities there already. What have you found needed to change about your team? What were the skillsets you were like, this is what we have to go out and add to our team for us to be competitive in this space?

• 1:08:39Speaker 00

  Yeah, unfortunately, when we pivoted from Mighty to Playground, we did have to let a whole bunch of people go.

• 1:08:45Speaker 00

  And they, like, I think almost all of them really understood. I mean, it was kind of like we had a band and if we, and the band was fine for Mighty, but then if the, if we moved everybody to Playground, it was like we had too many drummers on the band. So we were kind of like, what do we do with that? And so, you know, we quickly figured out that we definitely needed to make more space for a bigger AI research team. So we're actually still looking for like one very senior AI researcher. I don't have about like one, we only have one slot left to fill. But yeah, I mean, we were kind of like looking, right now, I mean, it's kind of, I think it's good, I think it's good to have a good team mix, like people that are like junior, people that are like very senior, people that are mid-level.

• 1:09:28Speaker 00

  I think it's good to have that team mix because then, you know, it's not like your senior AI researchers feel like they have to do some, like you might have like a valuable thing you want people, you want to do as a company, but not everyone's excited to do it. So it's really good to have like a good team mix of people who's find different kinds of projects more interesting. That's one thing I do for every, I do for all teams, not like something specific to AI research. The second thing is, um, I asked Sam for some advice from OpenAI.

• 1:09:56Speaker 00

  I think back in like December, we had like a 30-minute conversation or I was just like, hey, can you teach me? The pitfalls and things of like how to run an AI research team. I I don't know how to do this, probably gonna do badly. You can help me. Um, and I think, I think, you know, I learned a lot of things, but one of the things I was very curious about was to what extent you allow AI researchers to wander. Like, I'm, I'm very like ship, let's ship things as rapidly as possible. It's, you know, clarify scope. Let's figure out what we need to build, quality, move quickly. Um, but AI research is like, you know, Sam basically was like, yeah, stuff takes like 10 times longer than, you know, building a web app.

• 1:10:40Speaker 00

  And so I was really curious, how, how much do you allow people to kind of wander and meander? Cause you might not get good results. Um, I think, I think Greg Brockman also at OpenAI has like a tweet that I'll paraphrase. It's kind of like, you know, at first, things like don't work at all and they keep not working. And then eventually it's like amazing, something along those lines. He said it better than I'm saying it, but because he had to write it in tweet. But, um, but I think that, I think that, I, I think that's right. Uh, I think that we're experiencing that. We don't have a lot of data points as much as they do, but, um, we tend to let our researchers kind of wander. Probably a lot, probably not as much as OpenAI, cause OpenAI has like a really broad remit, but we care about like pixels and created tools.

• 1:11:24Speaker 00

  So there's obviously like these guardrails, but for the most part, we're just, I don't know, it's like you, you don't know, that's the, that's the reality of research and research is hard won. And so, uh, we're just like looking for wins, looking for like treasure everywhere.

• 1:11:39Speaker 01

  Yeah, zooming out of, of it's a hill. Um, earlier you were talking about how you were exploring the idea maze. I'm curious if you were focused as a VC in, in the space, what would be your mental model be for what, what kinds of companies, uh, you know, will endure, should I look to invest in versus which ones to perhaps, uh, stay away from? They might create some value, but they won't, they won't capture it or be durable.

• 1:12:02Speaker 00

  Yeah, um, I have definitely been thinking about that. Um, definitely talking, actually having conversations with other VCs or friends. Um, and

• 1:12:15Speaker 00

  One thing that I've gone to a number of AI dinners, you know, people, people like, you know, the founders of like Character, Midjourney, or whoever, and, and then even people like AI, just engineers that work at some of those companies, very helpful to get like perspectives all over the place. And, you know, I would, we would kind of ask these questions of like, what matters here? And so there's a general consensus that we're going to, if we haven't already, we're starting to run out of data that we can use from the public internet. Like maybe by the end of the year, largely we'll have like saturated all of that. And partly, there's, I think, I think someone else said this,

• 1:12:58Speaker 00

  that a lot, there's still a lot of data, it's just privately held. Like it's probably like exponentially growing in, in like a private ecosystem, like mobile apps. But a lot of the public data is like limited. And not, not to mention like copyright issues and stuff like that, it might be like a whole nother set of issues. So, you know, I think the first thing is like, having some advantage around data, I think is really critical. It may not be a enduring advantage for like a decade, but surely it can be a big advantage for like many years. Um, I think the second advantage is like,

• 1:13:33Speaker 00

  I think there's like a, I think there's like a space where like you could probably make a really amazing product that happens to use AI to build some kind of new consumer network effect. But we're not really seeing that yet. Like we're still using kind of the old ways with consumer, but there's probably some, some new tricks. And it requires like a lot of experimentation as consumer does. Um, I don't know what that is, but like an example of like this not going well is AI avatars. And it might not be a fad, actually. If it were power, if it were plugged into something that had a real distribution network and network effect. But so there's kind of this question of could we have bootstrapped something into a network effect with some novel

• 1:14:15Speaker 00

  way of using AI to solve some interesting problem for users. So I tend to look at like, I would find that really interesting. I think that like there's some problems with like some of the foundation model focused companies.

• 1:14:30Speaker 00

  You know, I think like the less generous thing I could say is that they're they're like making APIs in search of a problem. And this approach of doing this might work fine if you are OpenAI and you have billions of dollars and you can pursue like ten experiments at once. Ten, you know, ten fifty hundred million dollar experiments at once. And maybe you'll land the product that has product market fit.

• 1:14:56Speaker 00

  Maybe that's chat GPT. I don't know. I don't know what those numbers look like, but I think that it could be really tough to do that. Unless you're unless you're so well capitalized like OpenAI, that that approach is not maybe replicable. And you might have to be, yeah, you might have to do the basics like we're not, we're not trying to replicate the OpenAI approach. We are making a product. We're trying to go from a product down to a foundation model, not a foundation model to product. And we'll see if that works. Yeah, and I think I so I think, you know, it's I think there's a lot of buzz around the idea that there'll be mass. Will the large language models be commoditized? I think it's totally valid. For a long time, I had that. I had that view that like basically the L M's and everything will converge. They'll all be commodities.

• 1:15:41Speaker 00

  Where is the real business in this thing? But the deeper I go in trying to do research, the more I realize how hard it is, how hard one it all is. And I am not, I'm not as confident about that anymore, like maybe, maybe over a very long time span. These things will inevitably, you know, all technology becomes obsolete over a long time scale. But I don't think that people, I, you know, I think people that just have a shallow sense of this research. Are underestimating the complexity of how you build these models and how difficult it is. Sure, stable diffusion can get trained for six hundred thousand dollars on however many GPUs. But if you want to build something that endures and gets better.

• 1:16:25Speaker 00

  That requires very concerted investment and set of resources. You cannot just raise a hundred million dollars and say, boom, we've got a foundation model. You have to, you have to hire AI researchers, like you have to find clever ideas. It's very hard. It's just like software in that way. It's not like you can paralyze everything with with money. So, you know, I would just say like, I'm, I'm not so sure that it'll be as commoditized as people think. But I think certainly like. I think like the prior generation of the model, like a, you know, like a G P T two or three. It it could be kind of like every two years, the model will get commoditized. But the but the state of the art model endures for those two years.

• 1:17:08Speaker 00

  That could be a world we live in. It's kind of like, it's a little like, it's pretty similar to like CPUs. And the thing is, is that what customers want is the state of the art thing. They often don't want the, um. You know, nobody's, nobody wants to use G P T two.

• 1:17:24Speaker 00

  You want the the best thing you can get because you have to compete with everybody. And your competition is going to use the state of the art. And so then then there's kind of this argument around fine tuning. But even that argument, I think, is fairly weak because a lot of the things that you might fine tune today. Become not, you don't even need to use, you could use the the new state of the art L M model. And it can do what you previously could do with fine tuning, without fine tuning.

• 1:17:54Speaker 00

  And so I think people don't fully internalize. That the fine tuning thing is is also not as defensible. But if I were V C, you know, the other thing I would consider is like, if the company has something that they can fine tune on that is private data, that could be really defensible. So there are a couple like valid and valid things. But yeah, I think data, real hard one, A I research, network effects, all those. All those things are largely like kind of the same from traditional software, still, I think, apply in this world.

• 1:18:24Nathan Labenz?

  So you've got an eighteen month old kid.

• 1:18:28Nathan Labenz?

  What do you think their life is going to look like when they are your age? What does twenty fifty look like in your mind?

• 1:18:34Speaker 00

  Gosh, um. Sometimes my wife and I have this conversation of like, what thing could occur in the future that would make us feel uncomfortable? You know, like in our era. And I think like the answer to that would be like, I think I've answered this as like. What if my son wanted to marry an A I or something? And I'm like, I don't know how I would feel about it. I feel deeply uncomfortable, I think, right now. So, yeah, I mean, I just, I just think like the most weird thing will just be like your. My kid will be like friends with some. AI robot thing and he won't go outside and he'll be really obsessed with it and we'll be like,

• 1:19:16Speaker 00

  shouldn't you get like real friends and um, you know, and it'll be this like very, you know, it's like, it's like when my parents saw me on the internet all the time. They must have just like, thought I was probably getting into trouble, which I was, and um, doing bad things and, you know, I just think that, you know, it's, it's just, it's crazy. It's just, you could, you could just be, you could be, you could totally see kids just chatting with like some random AI thing. It'd be like the most interesting conversation they've ever had. It's unlimited and it's always interested in you and, you know, it's giving you like really cool um, insights to life and, I mean, I don't know. I mean, humans, humans have,

• 1:19:58Speaker 00

  don't have unlimited energy. So, I don't know, what if my son just like, maybe he, maybe some of his friends, some of his best friends would be AI things and, that'll be just so weird to me.

• 1:20:09Speaker 01

  He'll be like, you don't get it dad.

• 1:20:11Speaker 00

  Yeah, he'll be like, you don't get it. You know this isn't real, right?

• 1:20:16Speaker 00

  And then there'll be something that's like more advanced than the Turing test, you know, probably at that point, it'll be, and he'll reference that.

• 1:20:24Speaker 00

  It'll be like deeply philosophical. I mean, the other crazy thing could be, uh, that I told some friends a couple of weeks ago, that it doesn't seem too far off that we'll have like, the first like AI religion. And the crazy thing is that you can talk to your god and your god will give you answers. Um, yeah, so what if my son like joined, you know, some new religious AI spaghetti monster uh, religion? Well, you can talk to the spaghetti monster and, and it could be like a really positive thing for humanity. It could have good values and principles, but we, we would all just be like, but you know, this isn't real, right? And it'd be like offensive. Um, anyway, that would be the other weird thing.

• 1:21:07Speaker 01

  Yeah, I think that's a great place to, to, to wrap this uh, this podcast and aptly name the, the cognitive revolution. Uh, so Hill, thanks so much for, for, for joining us today.

• 1:21:15Speaker 00

  Yeah, thank you for having me.