How can we make sure that everyone has access to AI? Can small models outperform large models? With Stability AI’s Emad Mostaque

AI-generated images have been everywhere over the past year, but one company has fueled an explosive developer ecosystem around large image models: Stability AI. Stability builds open AI tools with a mission to improve humanity. Stability AI is most known for Stable Diffusion, the AI model where a user puts in a natural language prompt and the AI generates images. But they're also engaged in progressing models in natural language, voice, video, and biology. This week on the podcast, Emad Mostaque joins Sarah Guo and Elad Gil to talk about how this barely one-year-old, London-based company has changed the AI landscape, scaling laws, progress in different modalities, frameworks for AI safety and why the future of AI is open. Show Links: Stability. AI Stable Diffusion V2 on Hugging Face Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @EMostaque Show Notes: [2:00] - Emad’s background as one of the largest investors in video games and artificial intelligence [7:24] - Open-source efforts in AI [13:09] - Stability. AI as the only independent multimodal AI company in the world [15:28] - Computational biology, medical information and medical models [23:29] - Pace of Adoption [26:31] - AGI versus intelligence augmentation [31:38] - Stability. AI’s business model [37:44] - AI Safety

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:07Speaker 00

  What if the route to AGI is not one big model to rule them all, trained on the whole internet and then narrowed down to human preferences, but instead millions of models that reflect the diversity of humanity that are then brought together?

• 0:23Speaker 00

  I think that is an interesting way to kind of look at it because that will also be a human-aligned AGI, rather than trying to make this massive elder god of weirdness bow to your will, which is what it feels like at the moment.

• 0:41Sarah Guo?

  This is the No Priors podcast. I'm Sarah Guo.

• 0:44Speaker 02

  I'm Alad Gheil.

• 0:45Sarah Guo?

  We invest, advise, and help start technology companies.

• 0:48Speaker 02

  In this podcast, we're talking with the leading founders and researchers in AI about the biggest questions.

• 1:01Sarah Guo?

  AI-generated images have been everywhere over the past year, but one company has fueled an explosive developer ecosystem around large image models, and that's Stability AI. This week on the podcast, we'll talk to Ahmad Mustak, the founder and CEO. Stability builds open AI tools. They're most known for Stable Diffusion, the unreasonably effective AI model where a user puts in a natural language prompt and the AI generates images. But they're also engaged in progressing models in natural language, voice, video, and biology. We'll talk about how this barely one-year-old London-based company has changed the AI landscape, scaling laws, progress in different modalities, safety, and why he thinks the future of AI is open. Ahmad, welcome to No Priors.

• 1:42Speaker 00

  Thank you for having me on, Sarah. You lad?

• 1:43Sarah Guo?

  Let's start with a personal story. You have a background in computer science and you were working in the hedge fund world. That's a hard left turn, or it looks like it, from that world to being a driving force in the AI of state-of-the-art. How did you end up working in this field?

• 1:57Speaker 00

  Yeah, I've always been interested in AI and technology. So on the hedge fund, I was one of the largest investors in video games and artificial intelligence. But then my real interest came when my son was diagnosed with autism and I was told there was no cure or treatment. And I was like, ah, well, let's try and see what we can do. So I built up a team and did AI-based literature review, this was about 12 years ago, of the existing treatments and papers to try and figure out commonalities. And then did some biomolecular pathway analysis of neurotransmitters for drug repurposing, and came down to a few different things that could be causing it. You know, worked with doctors to treat him, and he went to mainstream school, and that was fantastic. Went back to running a hedge fund, won some awards, and then I was like, let's try and make the world better. And so

• 2:42Speaker 00

  the first one was, uh, non-AI-enhanced education tablets for refugees and others. And that's Imagine Worldwide, my co-founder's charity. And then in 2020, COVID came, and I saw something like autism, a multi-systemic condition, the existing mechanisms that extrapolated the future from the past, wouldn't be able to keep up with, and thought, could we use AI to make this understandable? And so I set up an AI initiative with the World Bank, UNESCO, and others, to try and understand what caused COVID, and try and make that available to everyone. Then I hit the institutional wall in a variety of places, and realized that the models and technologies that had evolved were far beyond anything that happened before.

• 3:25Speaker 00

  And there were some interesting arbitrage opportunities from a business perspective, and more than that, a bit of a moral imperative to make this technology available to everyone, because we're now going to very narrow, superhuman performance, and everyone should have access to that.

• 3:42Sarah Guo?

  It's an amazing journey, and congratulations on all the impact you've already had. So, as you say, or as you imply, the AI field in recent years has been increasingly driven by labs and private companies. And one of the most obvious paths to performance progress is to just make models bigger, right? Scaling data parameters, GPUs, which is very expensive. And then in reaction, just to set the stage a little bit, there's been some efforts over the previous years to be More community driven and open and build alternatives like Luther. How did you start engaging in that and how did stability change the game here?

• 4:20Speaker 00

  Yeah, so when I was doing the COVID work, we tried to get access to various models. In some cases, the companies blew up, other cases we weren't given access despite it being a high profile project. And so I started supporting a Luther AI as part of the second wave. So you know, Stella and Connor and others kind of led it on the language model side. But really, one of my main interests was the image model side. I have a fantasia, so I can't visualize anything in my brain, which is more common than people would think. In fact, a lot of the developers in this space have that, like we've got nothing in our brain.

• 4:51Sarah Guo?

  You just see words, what's in there?

• 4:53Speaker 00

  Just feelings.

• 4:54Sarah Guo?

  Okay.

• 4:55Speaker 00

  So like, again, I thought it was a metaphor. Imagine yourself on a beach. I was like, okay, I feel a beach. No, apparently you guys have pictures in your heads. It must be like just disconcerting. But then with the arrival of clip released by OpenAI a couple of years ago, you could suddenly take generative models and guide them to text prompts. So it was VQ GAN, which is kind of the slightly mushy, more abstract version first. But I built a model for my daughter while I was recovering ironically from COVID. And then she took the output and sold it as an NFT for three and a half thousand dollars and donated to India Code Relief. And I was like, wow, that's crazy. So I started supporting the whole space at Luther and beyond, giving jobs to the developers,

• 5:36Speaker 00

  compute for the model creators, funding the various notebooks from Disco Diffusion to these other things. You know, giving grants to people like Mid Journey that were kind of kicking this off.

• 5:47Sarah Guo?

  Just personally?

• 5:48Speaker 00

  Just personally, they were doing all the hard work. And I was like, can I capitalize this because this is good for society? Then about 15 months ago, I was like, well, these communities are growing. It'd be great if we could create this as a common good. And originally I thought you got communities, you got to make them kind of coordinated. Could a DAO work or a DAO of DAOs? And that's how stability started. After about a week, I realized that was not going to work and it was incredibly difficult. So then I figured out commercial open source software could be the way to create a line technology, not just an image, but beyond. That would potentially change the game by making this stuff accessible. Because as you said, one of the key things, this is in the state of the AI report.

• 6:29Speaker 00

  This is in AI Index as well, is that most research has been subject to scaling laws and other things. Transformers seem to work for everything. And so it was moving more and more towards private companies. But the power of this technology is double edged. One is that there are fears about what could go wrong, so it's not released. And the other one is, why not keep it for excess returns, right? So you've had this massive brain drain occurring and no real option. You work in an academic lab, you have a couple of GPUs, or you go and work at Big Tech or slash OpenAI. Or you set up your own startup, which is very, very difficult, as you guys know. So I wanted to create another option, and that's what we did with Aloutha and Stability and the other communities that we have grown and incubated.

• 7:13Speaker 02

  Could you talk more broadly about why you think it's important for there to be open source efforts in AI? And what your view of the world is? Because I think Stability has really helped create this alternative to a lot of the closed ecosystems, particularly around ImageGen, protein folding, a variety of different areas. And those are incredibly important efforts. So I'd just love to hear more about your thoughts on, you know, why is this important? How you all view the participation of the industry over time? And also what you think the world looks like in, you know, five years, 10 years, et cetera, in terms of closed versus open systems?

• 7:42Speaker 00

  So I think there's a fundamental misunderstanding about this technology. Because it's a very new thing, right? Classical open source is lots of people working together with a bit of direction. It's a bit chaotic, but then you've seen Red Hat and other things emerge from this. There aren't many people that train these models, right? We don't invite the whole community and you have 100 people training a model. It's usually five to 10, plus a supercomputer and a data team and things like that. And the models, when they come out, are a new type of programming primitive infrastructure. Because you can have a stable diffusion that's two gigabytes, that deterministically converts a string into an image.

• 8:19Speaker 00

  That's a bit insane, and that's what's led to the adoption here. You know, on GitHub stars, we've overtaken Ethereum and Bitcoin cumulatively. It took them 10 years. We got there in like three, four months. If you look at the whole ecosystem, it's the most popular open source software ever, not just AI. Why? Because again, it is this new translation file, and you do the pre-compute, as it were, on these big supercomputers, which means the inference required to create an image is very low. And that's not what people would have expected five years ago, or to create a chat GPT output. So as infrastructure, I think that's how it should be viewed. And so my take was that what would happen is everyone would be closed because you needed talent, data, and supercomputer, and those would be lacking, as it were. So it'd be the big companies only. They would go four,

• 9:04Speaker 00

  five years, and then someone would defect and go open source, and it would collapse the market as they commoditize everyone else's complement. So similar to Google offering free Gmail and all sorts of stuff around their core business. But more than that, I realized that governments and others would need this infrastructure. Because if a company has it privately, they will sell to business to business, so maybe a bit of B to C. But we've seen the Cambrian explosion of people building around this technology. But who's building the Japan model, or the India model, or others? Well, we are. And then that means that you can tap into infrastructure spending, which is very important because it needs billions. But the reality is, that's actually a small drop in the ocean. Self-driving cars

• 9:44Speaker 00

  got 100 billion of investment. We have 300s of billions. 5G, trillions. And for me, this is 5G level. So from an ethical, moral perspective, I was like, we've got to make this as equitably available as possible. So this model perspective, I thought was a good idea as well. But I thought we would held here inevitably. So I decided to create stability to help coordinate and drive this forward in what's hopefully a moral and reasonable way. Like, you know, the decisions that we make have a lot of import, and they're not easy. But we are trying to be kind of Switzerland in the middle of all of this, and provide infrastructure that will uplift everyone here.

• 10:22Speaker 02

  What do you think this world looks like in five years or 10 years? Do you think that there's a mix of closed and open source? Do you think the most cutting-edge models, the giant language models, are going to be both? Or do you think like capital will eventually become such a large obstacle that it'll make the private world more likely to drive progress forward? And I know you have plans in terms of how to offset that, but I'd just love to hear about those.

• 10:43Speaker 00

  The reality is, we have more compute available to us than Microsoft or Google. So I have access to national supercomputers and I'm helping multiple nations build excess scale computers. So to give you an example, we just got a seven million hour grant on Summit, one of the fastest supercomputers in the US. And like I said, we're building excess scale computers that are literally the fastest in the world. Private companies don't have access to that infrastructure because governments, thanks to us, are realizing that this is infrastructure of the future. So we have more compute access. We have more cooperation from the whole of academia than all of them do because their agreements tend to be commercial. There's no way that private enterprise can keep up with us and our costs are zero as well, when you actually consider that, whereas they have to ramp up tens of billions of dollars of compute.

• 11:28Speaker 00

  So my take is that foundation models will all be open source for the deep learning phase because we're actually got multiple phases now. The first stage is deep learning. That's creating of these large models and we will be the coordinator of the open source. The next stage is the reinforcement learning, the instruct models, Flan Palm or Instruct GPT or others. That requires very specified annotation and that's something that private companies can excel in. The next stage beyond that is fine tuning. So actually, let's give a practical example. Palm is a 540 billion parameter model. It achieves about 50% on medical answers, right? Flan Palm is the instructed version of that and that achieves 70%. Med Palm, they took medical information, they fed it in, this is a recent paper from

• 12:13Speaker 00

  a few weeks ago, achieved 92%, which is human level on the answers. And then the final stage for that is you take this Med Palm and you put it into clinical practice with human in the loop. For me, the private sector will be focused on the instruct to human in the loop area and the base models will be infrastructure available to everyone on an international, generalized and national basis, particularly because when you combine models together, I think that's superior to creating multilingual models. So that's quite a bit there and I'm sure you want to unpack that.

• 12:45Speaker 02

  Yeah, that's very exciting. Yeah. Could you actually talk about the range of things or efforts that are going on at Stability right now? I know that you've done everything from these foundation models on the language side, protein folding, image gen, et cetera. If you, if you could just kind of explain. What is a spectrum of stuff that Stability does and supports and works with? And then what are the areas that you're putting the most emphasis behind going forward?

• 13:08Speaker 00

  Yes, I think we are the only independent multimodal AI company in the world. So you have amazing research labs like FAIR, Meta and others, and DeepMind, doing everything from protein folding to language to image. And there are cross learnings from all of these. Basically, we do everything from audio to language, coding, models, any kind of almost private model. We are looking at what the open equivalent looks like. And that's not always a replication, right? So with stable diffusion, for example, we optimized it for a 24GB VRAM GPU. Now, as of the release of Distill's stable diffusion, it will run in a couple of seconds on an iPhone. And we have neural engine access because our view of the future is creating models

• 13:51Speaker 00

  that aren't necessarily bigger, but that are customizable and editable. So this is a bit of a different emphasis. And we think that's a superior thing than scaling. I think things like the Chinchilla paper, that's the 67 billion parameter model that's as performant as GPT-3 at 175 billion, are important in that because it said that training more is important. And actually, when you dig into it, it actually said data quality is important. Because now we're seeing that the first stage, the DL stage, is it where the deep learning stage is, let's use all the tokens on the internet, you know? But maybe we can use better tokens. And that's what we see when we instruct and use reinforcement learning with human feedback. And we've also been releasing technology around that. So our Carper Lab, representative learning, we released our instruct framework

• 14:35Speaker 00

  that allows you to instruct these big models to be more human. The way I kind of put it is that our focus is thinking, what are the foundation models that will advance humanity, be it commercial or not? What needs to be there and what's raised susceptible to this transformer-based architecture that takes about 80% of all research in the space? Making that compute and knowledge and understanding of how to build these models available to academia, independent research and our own researchers. And then from a business perspective, really focusing on where our edge is. And our edge is in two areas. One is media. And so this is why image models, video models and audio models have been a focus. 3D soon as well. And the other area is private and regulated data.

• 15:18Speaker 00

  Because what's the probability that a GPT-3 model weight or a palm model weight will be put on-prem? It's very low. Versus an open model, it's very high. And there's a lot more valuable private data than there is public data. So it is a bit of everything, but like I said, there are certain focuses on the business side on media. And then I think on a breakthrough side, computational biology will be the biggest one.

• 15:44Speaker 02

  That's really cool. And on the computational biology side, I guess there's a few different areas. There's things like protein folding and then to your point, there's things like MedPalm. And so are you thinking of playing a role in both of those types of models in terms of the medical information?

• 15:56Speaker 00

  Yes, we will release an open MedPalm model. Well, Med, stable GPT. And then protein folding, we are the, one of the key drivers of OpenFold right now. So we just released a paper on that, much faster ablations than AlphaFold. We're doing as well DNA diffusion for predicting the outcome of DNA sequences. We have bioLM around taking language models for chemical reactions. And that's an area that we will aggressively build because there's a lot of demand from the computational biology side for some level of standardization there. There have been initiatives like Melody and others looking at federated learning, but there is a misalignment of incentives in that space that I think we could come in and fix. And I think that's where we really view ourselves.

• 16:37Speaker 00

  How can you really align incentive structures and create a foundational element that brings people together? And I think that's where we are most valuable. Cause private sector can't do it that well, public sector can't do it that well. A mission oriented private company that has this broad base and all these areas could potentially.

• 16:54Speaker 02

  Yeah, I think also the, the global nature of your focus is really exciting because when I look at things like medical information or medical models, you know, ultimately the big vision there, which a number of people have talked about for decades at this point, is that you'd have a machine that would allow you to have very high access to care and medical information, no matter where you're in the world. And especially since you can take images with your phone and then interpret them with different types of models and then have like an output, you know, you should, if you have a cardiac issue, you should have care equivalent to the world's best cardiologist from Stanford or, you know, you name the. Center of excellence available to anybody in the world, whether they're rich, poor, developing country, not, et cetera. And so, you know, it's very compelling to see this big wave of technology and sort of the things that may be able to enable, including some of the

• 17:39Speaker 02

  things that you mentioned around AI and medicine. So

• 17:42Speaker 00

  Yeah.

• 17:42Speaker 02

  It's very exciting stuff.

• 17:43Speaker 00

  I think it's very interesting as well, because this technology is being adopted so fast. I mean, let's face it, Microsoft and Google, two trillion dollar companies have made it core of their strategy, which is crazy insane for technology that's basically five years old, let's say two years old, really breaking through. Because it can adapt to existing infrastructure, you know, like it sits there and it absorbs knowledge when you fine tune it. But then my thing is, I look to the future and I'm like, that best doctor, which bits of that should be infrastructure for everyone and which bits of that should be private? And so that's how I kind of oriented my business. I look to the future, I come back and I think, what should be public infrastructure and

• 18:25Speaker 00

  how can I help build that and coordinate that? And that's valuable. And then everything else, other people can build around.

• 18:31Speaker 02

  How do you think about the traditional pushback that's existed in the medical world around some of these technologies? So for example, you know, the first time an expert system or a computer could actually outperform Stanford University physicians at predicting infectious disease was in the 1970s with this Mycin project, where they literally trained an expert system or designed an expert system to be able to predict infectious disease. But here we are, almost 50 years later, with none of that technology adopted. And so do you think it's just, we have to do a lot of human loop things and it's a doctor's assistant and that'll be good enough? Do you think it's just a sea change? There aren't enough physicians? Like, what's the, what do you think is the driver for the technological adoption and something so important today?

• 19:10Speaker 00

  So I think the infrastructural barriers are huge for adoption of technologies, particularly when private sector. I think there is a new space of open source technology adoption that could be very interesting. And a willingness now that people kind of understand this, which wasn't there even 10 years ago, you know, the nature of open source. Now it runs the world's servers and databases. And I think there's another level of open source, which is open source complex systems, as it were. Previously in other discussions, I've talked about our education work. So right now, we're deploying four million tablets to every child in Malawi. By next year, we'll have hundreds of millions of kids, hopefully, that we deploy to. It's not just education, it's healthcare. And it's working with the governments, it's working with multilateral to say, can we build a healthcare system from the bottom

• 19:54Speaker 00

  up that can do all of these things? Without an existing infrastructure, because they don't have an existing infrastructure. It's one doctor per thousand kids, ten thousand kids, one teacher per four hundred kids. I am certain that system will outperform anything in the West within five years, which is crazy to say. But then our Western systems can then take bits of that and adapt to it. Because I think this competitive pressure is required. Because Western systems are very hard to change. And in the UK, we've done that with HDR UK, the genomic banks and others. And that was a massive uphill battle, as you know, to get these technologies adopted. Because I mean, it should, there should be barriers to adoption of this technology when it comes to things as important as healthcare. But at the same time, I think now is the time to open it up.

• 20:37Sarah Guo?

  Yeah, I think there is an interesting loose analogy to different pace of adoption of different technologies in different geos in the past, right? So one that comes to mind is today, I think it's very commonplace amongst consumer internet investors to look at what's happened with mobile in East Asia as a precursor to interactions that might happen here. And, you know, mobile technology advanced much more rapidly in China, Korea, many other places. One, because of private-public partnership. And two, because, you know, there was more, I guess, green field in terms of access to information and different infrastructure that supported mobile as the primary communication medium. And I, I could certainly see that happening with some AI native products.

• 21:23Speaker 00

  I think that's an excellent point. I agree a hundred percent. I think just as they leapfrog to mobile, a lot of the emerging markets, Asia in particular, will leapfrog to generative AI or personalized AI. And I can see this because I'm having discussions with the governments right now. What is the reaction? Over the Christmas holiday, I was getting a few hours of sleep finally. I got like six calls from headmasters of UK schools saying, "Emad, what is our generative AI strategy?" And I was like, "You what?" And they were like, "All our kids are using ChatGPT to do their homework."

• 21:54Speaker 00

  And so it's kind of one of the first global moments, an amazing interface that OpenAI built. It's going mainstream. And I was like, well, get good, you know, stop assigning essays. So now in some of the top private schools in the UK, they actually have to write the essays during the lessons without computers, which I think is wrong. Because my discussions in an Asian context, for example, with certain leading governments that are about to put tens of billions into this space. They're embracing the technology and they're like, how can we have our own versions of this? And how can we implement this to help our students get even better, right? Because also it's very, even though there might be bureaucracy in some of these nations, if they want to get something done, they get it done. And this technology is very different in that the costs are not continuous, like a 5G network.

• 22:38Speaker 00

  Uh, like the capex profile and other things are very different. Like, you know, you can say it costs $10 million to train a GPT. It doesn't cost that much anymore. That's really valuable if you can have a chat GPT for everyone, like the ROI is a huge. So yeah, I do think that a lot of these nations, like the African context is one that we're driving forward with education as a core piece. And right now we're teaching kids with the most basic AI in the world, literacy and numeracy in 13 months on one hour a day in refugee camps. That's insane. That's already better. It's going to get even better. But I think Asia in particular, they're going to go directly to this technology and embrace it fully. And then we have to have a question. If you're not embracing this in the West, in America and the UK, you're going to fall behind.

• 23:22Speaker 00

  Because ultimately this can translate between structured and unstructured data quicker than anything.

• 23:26Sarah Guo?

  I'd like to see what pace of adoption we can have in the United States of this technology as well, but I can see uh the the prediction coming true. If we just go back to the core, I guess not the core necessarily, but the most advanced, like mature use case with instability. And as you said, media as an advantage, what does the future of media look like? And actually, even if we go back before that, you know, you're involved in sort of early ecosystem efforts with Luther and such. How did you even identify that this was an area of interest for you versus everything else going on across modalities?

• 24:01Speaker 00

  So you know, I've always been interested in meaning, like semantic is even part of my email address. And that's my religious studies as well around epistemology and ethics, ironically. The way that I viewed it is that the easiest way for us to communicate is what we're doing right now via words, right? And that's held constant. But now we can communicate via phones and podcasts or whatever. And it's nice. Writing was more difficult and the Gutenberg press made it easier, but visual communication is incredibly difficult, be it a PowerPoint, which is visual communication, or art, which is visual communication. And then you have video and things like that, which is just impossible. Now you have TikToks and others making it easier. And I saw this technology and I was like, if the pace of acceleration continues, visual communication becomes easy for everyone. Like my mom sending me memes

• 24:45Speaker 00

  every day telling me to call more or kind of whatever. And I'm like, that's amazing. Because that creation will make humanity happier. Like you see art therapy, that's visual communication, and it's the most effective form of therapy. What if you could give that to everyone? So there was that aspect to it. But then I saw movie creation and things like that. So my first job was actually organizing the British Independent Film Awards and being a reviewer for the Rain Dance Film Festival. So, you know, every year I put a movie on for my birthday and we give the proceeds to charity. Get to see my favorite movie with my friends. It's pretty cool. And then I was the biggest video game investor in the world at one point. So these types of communication and interaction are really interesting. And I thought that people really misunderstood the metaverse, UGC, and the nature of what

• 25:29Speaker 00

  could happen if anyone could create anything instantly. It's not going to be a world for everyone, or a world that everyone visits. It's going to be everyone sharing their own worlds and seeing the richness of humanity. And again, I thought that was an amazing ethical slash moral imperative for making humanity better, but also an amazing business opportunity. Because the nature and way that we create media will transform as a result of this technology. And we're seeing it right now. We have amazing apps like Descript, right? Where you could take this podcast and you can edit it with your words live, you know? You have amazing kind of gaming things come out where you create assets and instances. Or, you know, some of this new 3D nerve technology where you can reshoot stuff. We are working with multiple movie studios at the moment who are saving millions of dollars

• 26:13Speaker 00

  just implementing stable diffusion by itself, let alone these other technologies. And that was for me tremendously exciting to allow anyone not to be creative because people are creative, but to access creativity and then allow the creatives to be even more creative and tell even better stories.

• 26:29Sarah Guo?

  I believe Sam from OpenAI has said they don't think image generation is kind of like core on the path to AGI. It's obviously really important to you personally and to stability. Tell us about your stance on AGI and if that's part of the stability mission.

• 26:45Speaker 00

  Yeah, I don't care about AGI except for it not killing us. They can care about it. What I care about is intelligence augmentation. You know, this is the classic kind of Memex type of thing. How can we make humans better? Like, our mission is to build the foundation to activate humanity's potential. So, but AGI is fine. Again, we have to have some things around that. I do believe that they are incorrect around multimodality being where images being a core component of that. But I think there are two paradigms here. One is stack more layers, and I'm sure GPT-4 and Parm 18 and all these things will be amazing, stacking more layers and having better data as well. But one of the things we saw, for example, stable diffusion, we put it out together and then people trained hundreds of different models.

• 27:29Speaker 00

  When you combine those models, it learns all sorts of features like perfect faces and perfect fingers and other things. And this kind of is related to the work that DeepMind did with GATTO and others that show that auto regression of these models and the latent spaces becomes really, really interesting. So what if the route to AGI is not one big model to rule them all, trained on the whole internet and then narrowed down to human preferences, but instead millions of models that reflect the diversity of humanity that are then brought together? I think that is an interesting way to kind of look at it because that will also more likely to be a human-aligned AGI rather than trying to make this massive elder god

• 28:12Speaker 00

  of weirdness bow to your will, you know, which is what it feels like at the moment.

• 28:17Sarah Guo?

  Yes, we're going to have a hive elder god instead. You've mentioned that stability is still working on language. The application of diffusion models to image is a really unique breakthrough and it's not as computationally intensive as like the known approaches to language so far. I think you've said that the core training run for the original stable diffusion was 150,000 in 100 hours, which is like not that huge, that grander scheme of things. What can you tell us about your approach to language?

• 28:43Speaker 00

  So yeah, so via the kind of Luther AI side of things in our team there, you know, we released GPT Neo J and X, which have been downloaded 20 million times. They're the most popular language models in the world. You kind of basically either use GPT-3 or use those. They got to about 20 billion parameters. And like I said, we've released our TRLX from the Kappa Lab, which is the instruct framework. They're training, you know, multiple models up to 100 billion parameters now. I don't think you need more, chinchilla optimal, to enable a chat, open chat GPT equivalent, you know, enable an open Claude equivalent. I think that will be an amazing foundation from which to train sector-specific and other models that then again can be auto regress. And there will be very interesting

• 29:24Speaker 00

  things around that. Language requires more, not necessarily because of the approach and diffusion breakthroughs, like recently, Google had their newspaper where they showed a transformer actually can replace the VA. So you don't necessarily need diffusion for great images. It's more because language is semantically dense, I think, versus images. And there's a lot more accuracy that's required for these things. That, I think there are various breakthroughs that can occur, like we have an attention-free transformer model basis in RWKV that we've been funding. We've got a 14 billion parameter version of that coming out that has showing amazing kind of progress. But I think that the way to kind of look at this is we haven't gone through the optimization

• 30:04Speaker 00

  cycle of language yet. So OpenAI, again, amazing work they do. They announced Instruct GPT, their 1.3 billion parameter version, outperformed 175 billion parameter GPT-3. You look at kind of Flann T5, the Instruct version of the T5 XXL model from Google, the 3 billion parameter version outperforms GPT at 175 billion parameters in certain cases, you know? These are very interesting results, and it's one of those things that as these things get released, it gets optimized. So like with stable diffusion, leave aside the architecture, day one, 5.6 seconds for an image generation using an A100, now 0.9 seconds. With the additional breakthroughs

• 30:48Speaker 00

  that are coming through, it'll be 25 images a second. That's 100 times speed up, over 100 times, just from getting it out there and people interested in doing that. I think language models will be similar. And I don't think that you need to have ridiculous scale when you can understand how humans interact with the models and when you can learn from the collective of humanity. So like I said, a very different approach of small language models or medium ones versus let's train a trillion parameters or whatever. And I think there will be room for both. I think it will be use these amazingly packaged services from Microsoft and Google. If you just want something out the box, or if you need something trained on your own data with privacy and things like that, that may not be as good, but may be better

• 31:30Speaker 00

  for you, use an open source base and work with our partners at SageMaker or whoever else, you know?

• 31:35Speaker 02

  Can you talk more about that in the context of your business model and your approach? You mentioned that you think that some of the areas at stability will be focused on as media and then proprietary and regulated data sets. So I know there's things you can share right now in that area, if not, no worries. But if you can, it'd be? Interesting to learn more about, you know, how you view the business evolving.

• 31:53Speaker 00

  Sure. So like now we're training on hundreds and soon thousands of Bollywood movies to create Bollywood video models with our partnership with Eros and that is exclusively licensed. We'll have audio models coming as well. As you have your A or R model or whatever, you know, we're talking to various other entities as well. And this is why we have the partnership with Amazon and SageMaker. So there'll be additional services that can train models for your behalf of most people. Our focus is on the big models for kind of nations, the big models for the largest corporates, who will need to train their own models one day. And that's really difficult. There's only like a hundred people who can train models in the world. Like it's not really a science, it's more an art, like losses explode all over the place when you try to do something. And so we're going to make it easy for them.

• 32:36Speaker 00

  And we're going to be inside the data centers, training their own models that they control. And our open source models then become the benchmark models for everyone. Like again, we have access to the neural engine, dedicated teams at Intel and others kind of working on optimizing these. That is the model, the framework and the open model is optimized and then we take and create private models. And again, I think that's complementary to the APIs and other things you will see from Microsoft, Google, et cetera. Because yeah, you would want both.

• 33:03Speaker 02

  Yeah. Some of the other areas that you focused on, or at least talked about, I think in interesting ways, is about how AI can be used to make our democracy more direct and digital. A little bit more about, you know, broader global impact. Could you extrapolate a bit more there?

• 33:17Speaker 00

  Yeah. So I think, you know, you have to look at intelligence augmentation, right? Like information theory in classical Shannon ways, information is valuable in as much as it changes the state. And we've obviously seen political information become more and more influenced by manipulation of stories and things like that. So the divide has been grown. What if we could create an AI that could translate between various things, make things easier to understand and make people more informed? I think that would be ideal with some of these national and public models and interfaces being provided to people. And then that can be very positive for democracy and allowing people to really understand the needs. Like you can already with ChatGPT, when you train it on nature of yourself, it can summarize all your perspectives. You know, that's amazing thing, right?

• 34:01Speaker 00

  You can talk like a five-year-old or a six-year-old or an eight-year-old or a 10-year-old. Once it starts understanding Sarah and Alad, that'll be even better. And again, you don't need some open students to do that. The OpenAI embeddings's API is fantastic. But I think there'll be more and more of these services that allow there to be that filter layer between us and this mass of information on the internet. That will be amazing. I think if we build the education systems and other things correctly as well, this young lady's illustrated primer that we're going to give to all the kids in Africa and beyond, like, again, let's really blue-sky think, how can we get people engaged with their communities and societies? Because it will be a full open source stack, not only education and healthcare and beyond. That's super exciting. And I think, again, that's the future of how we come together. Because you want to come together to form a human colossus, like in the Wait But Why

• 34:46Speaker 00

  style, where you get shit done, pardon my language. And I think this is one of the best ways for us to do that, leveraging these technologies.

• 34:53Sarah Guo?

  Okay, we don't have commercial sponsors.

• 34:55Speaker 02

  There's actually a book called Lady of Mazes. It's an AGI-centric book from like 10 years ago. And basically, the idea is sort of what you mentioned, where as different AGIs gain models of how a subset of the population thinks about certain issues, it substantiates into a virtual person who's basically representing them in some house of representatives equivalent. So you don't actually have to vote. The AGI just kind of synthesizes group opinions and then turns it into representatives.

• 35:22Speaker 00

  Yeah, and you have to think about, you know, with the advances like Meta's amazing work on Cicero, for example, beating humans on diplomacy. They used eight different language models combined. And again, I think this is the future, not just zero shop, multiple models interacting with each other is the way, full stop. The issue in mechanism design's perspective of kind of the game theory of our current economy is that there is no central organizing factor that we trust. Like, what is the trust in Congress? Like, I think they trust Congress less than cockroaches. No offense to Congress, please don't bring me up. Like, it's just a poll, right? People will erred towards trusting machines, as it were, and machines are capable of load balancing. Now they're capable of load balancing back some things. And so we have to be super careful as we integrate these things, what that looks like, because they will make more and more decisions for us.

• 36:08Speaker 00

  That could be for our benefit, having AI that speaks on our behalf and amalgamates. But then we need to make sure that these aren't too brutal and fragile as we cede more and more of our own personal authority to them because they are optimizing. This is also one of the dangers on the alignment side. As we introduce RLHF into some of these large models, there are very weird instances of mode collapse and out of sample things. I do say these large models as well should be viewed as fiction creative models, not fact models, because otherwise we've created the most efficient compression in the world. Does it make sense? You can take terabytes of data and compress it down to a few gigabytes with no loss. No, of course you lose something. You lose the factualness of them, but you keep the principle-based analysis of them. So we have to be very clear about what these models can and can't do because I think we

• 36:53Speaker 00

  will cede more and more of our authority individually and as a society to the coordinators of these models.

• 37:00Speaker 02

  Could you talk more about that in the context of safety? Because ultimately, one of the concerns that sort of increased in the AI community is AI safety and there's sort of three or four components of that. There's alignment, you know, will bots kill humans or whatever form you want to put it in.

• 37:14Sarah Guo?

  They'll farm us, not kill us.

• 37:15Speaker 02

  Farm us, that's a good point. They'll just build a giant RLHF farm on top of us or something. There's the concern around certain types of content, pedophilia, et cetera, that people don't want to have existence in society for all sorts of positive reasons. There's politics. You know, there's concerns, for example, that AI may become the next big battleground after social media in terms of political viewpoints being represented in these models with the claims that they're not political viewpoints. And so I'm sort of curious how you think about AI safety more broadly, particularly when you talk about trust of models. To your point, part of it is fact versus fiction, but part of it may also be, well, it looks like it's political. And so therefore, maybe I can't trust it at all.

• 37:54Speaker 00

  Yeah, I don't think technology is neutral. So I'm not one of the people that adheres to that, especially with the way we build it. It does reflect the biases and other things that we have in there. I did. Kind of follow the open source thing, because I think we can adjust that. On the alignment side, you know, it was interesting, Alouth are basically split into two. Part of it is stability and the people who work here on capabilities. The other part is conjecture that does specific work on alignment. And they're also based here in London. And I think it's not easy, right? I think that everyone is ramping up at the same time. And we don't really understand how this technology works, but we're doing our best, you know? You have people like kind of Riley Goodside and others, prompt whisperers. They're like, wait, like, what on earth? It can do all these kind of things.

• 38:37Speaker 00

  Um, I think that there needs to be more formalized work. And I actually think there needs to be regulation around this. Because we are dealing with an unknown, unknown. And I don't think we're doing good enough kind of tying things together, particularly as we stack more layers. And we get bigger and bigger and bigger. I think small models are less dangerous, but then the combination of them may not be safe. But again, we don't know this yet.

• 38:59Sarah Guo?

  You've mentioned before this support for the idea of regulation of large models. What would be a productive outcome of that regulation that you can imagine?

• 39:09Speaker 00

  I think that a productive act from that regulation is anything above a certain level of flops. Needs to be registered similar to, well, bioweapons and things that have the potential for dual use. I think there needs to be a dedicated international team of people who can actually understand and put in place some regulations and how do we test these models for things like, you know, the amazing work Anthropic recently did with constitutional models and other things like that. We need to stop pooling this knowledge as opposed to keeping it secret. But there is this game theoretic thing of one of the optimal ways to stop AGI happening is to build your own AGI first. And so I'm not sure if that will ever happen, but we're in a bit of a bind right now, which means that everyone's having their own arms race. When governments decide, and I don't believe they've decided yet, that having an AGI is the number one thing,

• 39:53Speaker 00

  tens of billions, hundreds of billions will go into building bigger models. And again, this is very dangerous, I think, from a variety of different perspectives. So I prefer multilateral action right now as opposed to in the future. So I put that out there, but I can't really drive it. I'm already dying from all the overwork, because it is. But I do believe that should be the case. I think going on to kind of the next one, as you said, the political biases and things like that. We can use this as filters in various ways. And I think one of the interesting things, and the other thing I've called regulation of, maybe I should do it a bit more loudly, is you have a lot of companies that have ads as one of their key elements, right? And ads are basically manipulation and these models are really convincing. They can write really great prose. My sister in law created a company, Senantik, that can do human, realistic, emotional voices.

• 40:37Speaker 00

  She did like Val Kilmer's voice for his documentary and stuff like that before selling to Spotify. It's going to be crazy, the types of ads that you see. And we need to have regulation about those soon because you're going to see Meta and Google and others trying to optimize for engagement and manipulation fundamentally. And I think that those can then be co-opted by various other parties as well on the political spectrum. So we need to start building some sort of protections around that. What was the final one? Sorry, Elad.

• 41:04Speaker 02

  I was just asking about, and I think Sarah asked the question around, you know, where do you think regulations should be applied? And what are the, what would be positive outcomes of that versus negative outcomes?

• 41:13Speaker 00

  Yes, I think, you know, there should be these elements around identification of AI, especially on advertising. I think that there should be regulation on very large models in particular. European Union introducing a C mark slash generative AI restrictions, where the creators are responsible for the outputs, I think is the wrong way. But there are other ones as well, like I would call for opt out mechanisms. And I think we're the only ones building those for data sets. So we're also building some of these data sets and trying to figure out attribution mechanisms for opt in as well on the other side. Like right now, the only thing that is really kind of checked is robots dot text, which is kind of thing on scraping. But I think, again, it's evolving so fast that people might be okay with scraping, but they may not be okay with this.

• 41:58Speaker 00

  Legally, it's fine, but then I think we should make this more and more inclusive as things go forward.

• 42:03Speaker 02

  So that's for example, if an artist doesn't want their work represented in a corpus that a machine is trained on, for example.

• 42:09Speaker 00

  Yes, and it's difficult. It isn't just a case of, you know, don't look at DeviantArt or my website. Like what if your picture is on the BBC or CNN with a label? It will pick that up. You know, so it's a lot harder. This is why like we trained our own open clip model. We have the new clip G landing this week. That's even better on zero, I think eighty percent. Because we need to know what data was on the generative and the guidance side so that we could start offering opt out and opt in and these other things. So it's not as easy as some people kind of know.

• 42:39Speaker 02

  Yeah. And then I guess one other area that people often talk about safety is more around defense applications and the ethics of using some of these models in the context of defense or offense from a national perspective. What's your view on that?

• 42:52Speaker 00

  I think the bad guys, you know, for the end quotes, have access to these models already and thousands and thousands of A one hundreds. I think you have to start building defense, but it's a very difficult one. Like we were going to do a $200,000 deep fake detector prize, but then it was pointed out quite reasonably that if you create a prize for a detector, then creates, well, a bouncing effect, where you have a generator and a detector and they bounce off each other and you just get better and better and better. So now we're trying to rethink that. Maybe we'll offer a prize for the best suggestion of how to kind of do this. Similar to, you know, chat G P T is detectable, but not really. So I think the defense implications of this, it's largely around kind of misinformation, disinformation. This is an area that I have advised multiple governments on with my work on counter extremism and others. It's a very difficult one to unpick, but I think one

• 43:36Speaker 00

  of the key things here is having attribution based mechanisms and other things for curation because our networks are curated. And so this is where we've teamed up with like Adobe on content authenticity.org and others. I think that metadata element is probably the winning one here, but we have to standardize as quickly as possible around trusted sources. I think people already don't believe what they see though, which is a good thing and a bad thing. We want to have those trusted coordinators around this. Beyond that, and some of the more severe kind of things around drones and slaughter bots and things like that, I don't know how to stop that, unfortunately. And I think that's a very complicated thing, but we need an international compact on that because again, this technology is incredibly dangerous when used in those areas. And I don't think there's been enough discussion at the highest levels on this, given the pace of adoption right now.

• 44:22Sarah Guo?

  I think that's all we have time for today. So one last important question for you. What controversial prediction? You seem like an optimist, but good or bad about AI do you have over the next five years?

• 44:32Speaker 00

  I think that small models will outperform large models massively, like the hive model aspect, and you will see chat G P T level models running on the edge on smartphones in five years, which will be crazy.

• 44:44Speaker 02

  Great. Thanks so much for joining us. Amazing conversation, Israel.

• 44:47Speaker 00

  It's my pleasure.

• 44:50Sarah Guo?

  Thank you for listening to this week's episode of No Priors.

• 44:53Speaker 02

  Follow No Priors for new guests each week and let us know online what you think and who in AI you want to hear from.

• 44:58Sarah Guo?

  You can keep in touch with me and Conviction by following at Sarah Normas.

• 45:02Speaker 02

  You can follow me on Twitter at @AladGill. Thanks for listening.

• 45:08Sarah Guo?

  No Priors is produced in partnership with Podpeople. Special thanks to our team: Synthel Galdea and Pranav Ruddy, and the production team at Podpeople: Alex Vicmanis, Matt Saab, Amy Michado, Ashton Carter, Danielle Roth, Carter Wogan, and Billy Libby. Also, our parents, our children, the Academy, GovGPT, and our future AGI overlords.