Model Quality, Fine Tuning & Meta Sponsoring Open Source Ecosystem

What Does it Take to Improve by 10x or 100x? This week is another host-only episode. Sarah and Elad talk about the path to better model quality, the potential for fine tuning to different use cases, retrieval systems (RAG), feedback systems (RLHF, RLAIF) and Meta’s sponsorship of the open source model ecosystem. Plus Sarah and Elad ask if we’re finally at the beginning of a new set of consumer applications and social networks. Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil Show Notes: 0:03:00 - AI Models and Open AI Advances 0:08:59 - Addressing Hallucinations in AI Models 0:13:22 - Open Source Models in Consumer Engagement 0:16:23 - New Trends in Social Content Creation 0:21:53 - Balancing Ambition With Realistic Customer Expectations

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Sarah Guo?

  Hi, NoPriors listeners. Time for a host-only episode. This week, Alad and I talk about the path to better model quality from here, the potential of fine-tuning, RLHF, RLAIF, RAG and retrieval systems generally, meta sponsorship of the open source model ecosystem, and finally, the beginning of a new set of consumer applications and social networks. Thanks for tuning in. So, one thing everybody is thinking about is what it takes to get to 10x or 100x better AI systems. Like, I think it'd be useful to just sort of enumerate the elements to step function better. Alad, what do you think?

• 0:42Speaker 01

  Yeah, you know, it's interesting because there's a few different aspects of that that people always talk about. There's scalability of data sets and compute and parameters and all these things. But the reality is, I think, a lot of people believe that in order to 10x or even 100x use cases and usages for AI, outside of that, there's things that could just be done on existing models today. So you don't need to wait for GPT-7 or whatever. You could start with GPT-4 or GPT-3.5 and add these things. And I think they are kind of bucketed into five or six areas. Number one is multimodality. So that means being able to use text or voice or images or video as both input and output. So you should be able to talk to a model, type to it, upload an image and ask about the image, and then it could output anything from code to a short video for you. Second is long context windows.

• 1:25Speaker 01

  So basically, when you prompt a model, you basically are feeding it data or commands or other things, and everybody realizes that you need longer and longer and longer context windows. So magic, for example, is doing that for code. You know, you should be able to dump an entire code repo into a coding model instead of having to do it piecemeal.

• 1:43Speaker 01

  Third, which we're going to talk about today, is model customization. So that's things like fine-tuning, something known as RAG. There's data cleaning, there's labeling, there's a bunch of stuff that just makes models work better for you. Fourth is some form of memory, so the AI actually remembers what it's doing. Fifth is some form of recursion, so looping back and reusing models. And then sixth, which is related, is potentially a bunch of small models that are very specialized, being orchestrated by a central model or sort of AI router that says, "Well, for this specific task or use case, I'm going to route the prompt or the data or the output into this other model that's doing this other thing." Which is basically how the human brain works, right? Process visual information through your visual cortex, but then you use other parts of your brain to make decisions, right?

• 2:28Speaker 01

  And so it's very similar to what evolutions were decided was an optimal approach. But I think it's really interesting because I think many people in the field know that these five or six things are absolutely coming and they can dramatically improve the performance on existing systems. Again, 10x, 100x better for certain things. And so it's more just a matter of when, right? It's not really an if anymore. A bunch of people are working on different aspects of this. And, you know, I think it's all coming really fast. And so what, you know, there's sort of two things that came out in the last week or two that are really relevant to this. It'd be great to get your thoughts on. One is OpenAI announcing that they're not going to allow people to fine tune models. And then second is Google, where they looked at human generated feedback versus AI generated feedback for models and sort of fine tuning models that way.

• 3:11Speaker 01

  So, and if you want to tell people a bit more about what happened with OpenAI and why that's important.

• 3:16Sarah Guo?

  Yeah, so fine tuning as a capability has been offered by OpenAI for several years, right? But they've made like a, a specific investment in allowing people to do that with more sophisticated models, in particular, like three, five, and also making it possible for more enterprise use cases, right? And if you think about sort of like why that matters at all, as you said, like, you know, you have a bunch of these labs who are working on general capability and working on the sort of direction of scaling laws, like transformers predictably improve with scaled data and compute. But I think what's really interesting is like the way every, the way these models end up being used in many business or even consumer application contexts against a specific task, right?

• 4:01Sarah Guo?

  And so we've talked a lot about like where research effort is being put or compute is being spent in the industry right now. And there's a really, I think there's a really interesting question of, we don't even know how good models can be at certain scale, right? At 70 or 30 or 100 billion parameters or more, but not a GPT four scale, based on really high quality data and curation of that data, because it hasn't, it hasn't been explored. And so. I think we should talk about some of the different ways you get these models to actually operate against a specific task with either fine tuning with RHF against, you know,

• 4:42Sarah Guo?

  the reward for your task or with with RAG, as you said, in terms of retrieving from a data set that you've specified, right? And there's reasons you would do all three of these. But I think it's actually a pretty big step for OpenAI to enable this because I think there was, at certain points in the in the research world, there's been a narrative that like fine tuning doesn't really matter, right? The general model matters. And I'd be curious if you think that's a change in research point of view, or just a commercial decision in terms of labs wanting to make money or that being more important than ever?

• 5:18Speaker 01

  Yeah, I think everybody realized that fine tuning works really well when ChatGPT came out because what ChatGPT is, is they took this model GPT 3.5, which existed at the time, and it wasn't seeing as much usage, at least from, you know, people just going in and querying it unless they were really good at prompts. And they basically hired a bunch of people and the people ranked the output of the model and they effectively fine tuned the model against that feedback from the people who are assessing, is this the answer that I wanted based on the prompt that I put in, right? And so fine tuning really just means you create a lot of feedback, usually at least today, through people responding to output and saying, is it good or bad? And it created a dramatic step function in

• 5:59Speaker 01

  the utility of GPT 3.5 for end consumers or end users or students or lawyers or all sorts of different types of people. And it really helped. It was kind of the starting gun for this whole AI revolution right now because everybody suddenly realized how powerful these models were. And the model underlying it fundamentally hadn't really changed that much. What they'd done is they'd fine tuned it with reinforcement learning through human feedback or RHF. And so I think that created this viewpoint that these these types of fine tunings, or, you know, we can talk about RAG in a minute, let me get your thoughts on that, can fundamentally change the user affinity for a product.

• 6:36Speaker 01

  And so you could imagine in an enterprise, you say, well, I really want to fine tune this model so that it reflects medical data that I have this proprietary that could help make a better doctor assistant, or I want to fine tune it against this, you know, set of HR responses that are unique to my company. So that if I have a, an employee who really wants to get answered a question, they can get a really good answer back. And so it really gets into those sorts of things where you can dramatically improve the output of a model against something that you specialize. Do you want to talk about how RAG ties into that? Because I think that's a really key component of it, too.

• 7:07Sarah Guo?

  I think the sort of basic premise with RAG that everybody should understand is you want to retrieve against a specific corpus, right? And so you're still going to reason, you might have a generation or an answer based on that corpus. But if you pick a set of documents, it could be legal cases, it could be internal company documents, it could be medical information, as you said, right? So you still want the reasoning capabilities of the model, right? A diagnosis requires reasoning, but you want it to come from a specific set of data versus like, let's say, all of the pre-training data of, you know, random information on the internet about whether or not you have this disease, right?

• 7:48Sarah Guo?

  And every piece of forum conversation about this disease that has ever happened. So, you know, I think of the, the core driver as like trustworthiness, right? Citation, control of information source. And, and so now you have this architecture where people are using, think of it as like traditional information retrieval techniques and search in combination with these models. I think the other sort of driver besides trustworthiness on these RAG approaches is two things. One is cost and the other is like freshness, right? So every time you retrain a model or even fine tune a model, like there is compute involved,

• 8:30Sarah Guo?

  see the idea that, you know, being able to incorporate new information without retraining and just using the reasoning capabilities of the models, I think is very attractive to people. And very, that's also related to the freshness point of view, which is like, you actually want the most recent medical research or the cases from this past year. I think that's, that's sort of a set of the drivers behind people being excited to take this approach and use it against their private data sets.

• 8:56Speaker 01

  Yeah. And that actually helps a lot with hallucinations, right? And so I think it's important to sort of explicitly point that out because one of the knocks on the current set of AI technologies as well. May hallucinate or say, you know, say things that aren't necessarily true or cite a legal case that doesn't exist. And by using RAG, you can actually help say, okay, I'm only going to use things that that I know exist, or I'm going to filter for things that are going to be answers that fit well with, you know, the the current set of knowledge that people have relative to these sets of issues. So to your point on trustworthiness, I think it's really important to call out hallucinations explicitly, since that's something people keep bringing up, this sort of naysayers, oh, my gosh, what if it hallucinates and some terrible misinterpretation happens, and therefore, we need to regulate this thing.

• 9:38Speaker 01

  Right. So it's kind of interesting. You know, I guess, related to that, there's this reinforcement learning through human feedback versus AI feedback. And Google just came out with a really interesting paper on that, where, you know, they showed that you can have an AI similarly provide feedback to whether the AI itself is generating good output. And for certain use cases, that works as well as people. And so suddenly, instead of having to hire an army of people to go and help fine tune these models, you can actually have an AI help fine tune this model. And I think the the early signs that that was going to be true was actually MedPalm 2, where Google showed that they trained a model specifically on medical data, and the output from the model tended to be more

• 10:20Speaker 01

  correct than human physician experts. And so for certain use cases, we are already seeing AI provide more accurate answers than specialists, experts, right? And in RL AI F, you're trying to sort of generalize that and say, what are all the different ways that instead of using expensive people to do this, we can use really cheap AI models to provide that same feedback and sort of train things. And so there's all these techniques and technologies that are coming now as part of this sort of list of six, six big innovations that are part of the future AI 10x or 100x roadmap that are starting to fall into place. So I think it's a very exciting time. And I think, you know, in the next year, we'll keep seeing stuff like that. So there's a few other announcements that have come out related to this in terms of using different data sets or different

• 11:04Speaker 01

  models, but coming from social networks. So for example, Twitter, or I guess now we should call it X, said it will train ML models off of Twitter data, and that may have really interesting consumer applications or outcomes. And then Meta is really now emerging as a primary sponsor for open source models. Llama and Llama 2 have really taken off in sort of the developer and enterprise ecosystem around LLMs. So it'd be great to hear what you think in terms of why are they doing this? You know, why are they becoming the primary sponsor for open source AI? And how do you think they're going to apply it within their own company?

• 11:40Sarah Guo?

  I really draw a analogy from the current sponsorship of Meta and Zuck of, you know, Llama and the open source model ecosystem to like MySQL, right? So for those of us who remember like what happened with these open source database companies, MySQL ended up being originally made by this guy Monte Weidenius and some Swedish company became part of Sun, become part of Oracle. And in the early days, like MySQL would crash and corrupt data and there were some early internet scale companies like Facebook who wanted to use it, wanted to not be beholden to commercial database vendors, made it scale, made it more robust and contributed back.

• 12:25Sarah Guo?

  Right? And I think like it's a reasonable analogy in terms of like some core technology to your company where you don't want to have a vendor, you don't see it as part of your core business model, but you want there to be open source options. Right? And so I have a lot of admiration for what Meta is doing. And I think like, I think it is very likely to be a big mover in the ecosystem because if they sponsor some baseline of models that are big enough to be valuable, high quality enough to be valuable with Facebook AI research, and then enough people find these models useful and strategic and they create a developer ecosystem, it's hard for me to picture them

• 13:06Sarah Guo?

  not being sustained as an important ecosystem and alternative to these, you know, research labs that in many ways compete with Facebook or Meta in different ways and are very expensive to maintain.

• 13:19Speaker 01

  But if you look at the history of open source, is that really true? So say for example, you look at Linux, right? And Linux in part was very much sponsored by IBM throughout the late 90s to the tune of in some, in some years a billion dollars a year. And so even these external ecosystems tend to get quite expensive, you know? And the reason that IBM sponsored Linux was to provide a real offset to Microsoft, right? They basically said Microsoft is dominant. On the desktop, they're really getting aggressive on sort of the server and infrastructure world. And so therefore, let's fund this offset for open source. How do you think that analog applies to Meta or does it? Or do you just think it's a different reason in terms of why they're pursuing it?

• 13:56Sarah Guo?

  Um, well, I think they're pursuing it because they want to use it and they don't want to be trapped. Right?

• 14:02Speaker 01

  Oh, sure, but they don't have to open source it, right? They could just continue to develop it like they have been. And so why open source it?

• 14:08Sarah Guo?

  One piece of it is like wanting to offset the development costs and the compute costs at some point. Right? And like, that's sort of one of the core premises of open source. They've also done like other really related things like the Open Compute Project. But, you know, if you think about why that analogy does or doesn't apply, right? Like, one is, does Meta want to make money off of this in some sort of like B to B way? If they keep open sourcing it, the answer is no, right? They want to use it in their core consumer businesses. And then two, like for, for this to work, I think one of the ways the analogy breaks down is very much like the need for centralized training today, right? It's a complicating factor, like, can you really coordinate that with the politics and slow decision making of open source communities? I don't know. I think that's challenging.

• 14:53Sarah Guo?

  There, um, there are interesting folks working on at least these sort of like technical coordination of, of this as well. Right? Like foundry and together. But if you, just to like make explicit, like, why might they care? My guess is like the ability to use these models, it applies in sort of the more traditional ways. Like, we can use them to make the data center like more energy efficient. We can, and there's been publishing about this, we can use these models to improve, um, like ad serving. Right? Uh, like lots of things that matter to the core meta business. But it's also just one of the most interesting things to happen in consumer in a long time. Right? You have things like character inflection, mid journey, Pika, experiments like can of soup. Like, these things, they have caught the attention.

• 15:39Sarah Guo?

  Of consumers in a way that few things have over the last few years. And so I think it's known that there are Instagram chatbots being tested, right? And so if this is a path to consumer engagement and then therefore ads and it's going to be a really important element, I think they just want to have access to it without being beholden to a sponsor. What's your view?

• 16:00Speaker 01

  Yeah, I mean, I think it's amazing that Meta has decided to make this move. And I think it's really beneficial to the ecosystem overall. So, you know, at this point, I think Lama 2 is really emerging as the model that a lot of people are rallying around. And obviously, that may change over time. But for now, I think it's one of the primary models people are using on the open source side and that people view as quite high quality. So I think it's super impressive. I think more broadly in social on the eye, it's kind of striking that the last large social network in some sense was TikTok, which was launched seven years ago now. So it's been a while since we've seen a major shift. And part of that is because large scale social products have already been established. And so now you need to sort of pry users away from existing products, which is much harder than just filling time otherwise. I remember talking once with Jack, the founder of WikiHow,

• 16:44Speaker 01

  which was like a how to, you know, community driven website. And he said that the main way that they lost people who were contributing to WikiHow was they went to social gaming. They were just playing games instead, right? So it was sort of this time and attention shift 10 years ago. And when he mentioned this to me, right? And so number one is you have to displace other people. Number two, you know, a lot of the innovation in social kind of stagnated a little bit for startups, right? It became a lot more, let's do Twitter, but more woke or more right wing. Or let's do early Facebook again as a mobile app versus, hey, we're going to reinvent the modality, or we're going to reinvent the use case or the communication channel, whatever it may be. And it feels like generative AI

• 17:29Speaker 01

  is the first thing in many years to sort of create that new window or opening. And I think the big social networks like Meta and Twitter and others may actually be the biggest beneficiaries of this new wave. But there also should be room for startups. And there's some new things, you know, can of soup was in the recent YC batch and they're doing kind of interesting things. And I think it's almost like asking, what's the gen AI native?

• 17:50Speaker 01

  Modality and use case. And typically, when you'd look at social products, you used to have this two by two, or some people had like a two by three of, you know, is it broadcast versus mutual follows in terms of network structure? What's the modality? Is it images? Is it video, et cetera? And then what's the length and persistence of it? Is it long form? Is it ephemeral, et cetera? So for example, Snapchat started off as, you know, short form broadcast and one on one that was ephemeral, right? And so you could kind of map out the whole social world against those dimensions. And now there's this new interesting thing of, you know, new forms of content creation, potentially upending one or two of those quadrants. So it seems like a very exciting time overall.

• 18:30Sarah Guo?

  Yeah, yeah, I had a, you know, long time obsession with Tootsie and TikTok and some of the Chinese social companies that really started as like AI native content aggregators, right? If you think about what they did, they really figured out this like cold start problem in terms of they like Tootsie originally, they aggregated news content from other places and then bootstrapped your preferences. They didn't require explicit user input to say like, I am interested in these topics. They analyzed your social profile for your interests. They collected like location and demo and analyzed articles for like quality and topics. So they had these like rich per user models of engagement based on interaction data.

• 19:14Sarah Guo?

  And then you have this magical experience of like a better content feed that then drove the iteration around better labeling. And I think exactly as you said, if those companies figured out like the cold start on relevance, maybe the opportunity, I think one of the potential opportunities in this generation of social is like cold start on the content itself, right? Like you've seen other amazing companies like.

• 19:41Sarah Guo?

  Like the Instagrams of the world, right? They they create tools for content creation for like magically compelling assets that are much easier and then like turn it into a social network. And so generation feels like a a really compelling answer in terms of like how to have a content feed that is both like really engaging for you and then giving people creation superpowers.

• 20:04Speaker 01

  Yeah, and I think Midjourney and Pika are two great examples of that to the point earlier. And then character is sort of a form of that if you decide to create your own character or sort of interact with something that's more customized there. So it does seem like there are these really interesting shifts that are happening. And then the question is, is it more for creation and sharing or does it become a new social product or a new communication product? In other words, is it Giphy or is it, you know, Facebook, right? And Blenza was a good example of Giphy, right? It was used to basically create content that you share on other social networks. And the question is, what are going to be the big consumer apps that sort of emerge on top of that? And again, it may just be meta again, right? But I think it's a super interesting question and probably the most exciting time in social for a very long time.

• 20:49Speaker 01

  And it's kind of this oddly, almost ignored area from a entrepreneurship and founder perspective right now. Everybody's rushing at the enterprise stuff and the infrastructure and, you know, that whole stack. And it's almost like the generation of people who were going to start social products all did them five years ago and did the, you know, let's do Twitter again. And the generation that's really focused now kind of grew up where SaaS was sort of opportunity or SaaS and DevTools were the opportunities that everybody was mining again. So it'll be interesting to see whether or not that shifts back in any meaningful way. The one other thing that I think is kind of interesting, just related to entrepreneurship and AI right now, and I was talking to a founder about this, where they were trying to do something really hard.

• 21:31Speaker 01

  And by really hard, I mean, addressing a really hard market by using Gen AI. And early in markets, like when a new technology shifts and disrupts a whole market, you actually want to just do the easy stuff. Right? Why do the hard stuff? There's so much low hanging fruit. Why don't you just go after the stuff that's super easy? And my, my sort of advice to founders generically on this stuff is like, don't do the hard stuff right now. Or if it's hard, do something that's technically hard that enables a giant breakthrough in terms of use case. But don't actually do the hard market because there's so many easy markets right now. You should just, you should just go for the easy stuff. And if you're grinding and grinding and grinding and not getting customer attention, don't spend more time on it. It's just not worth it right now. Now, five years from now, when the use of these technologies are a bit more saturated, that's when you have to go do the hard stuff. Right?

• 22:15Speaker 01

  But you know, it's kind of interesting to, to think about, you know, prior technology waves and when should you do the easy versus hard?

• 22:22Sarah Guo?

  Yeah, I was actually just talking to some of the founders that are in our accelerator right now that come from like really great technical and research backgrounds. And they were reaching for a problem broadly in the engineering and code generation space that was very ambitious. Right? And I could see kind of a, a solve it all type problem. And it's not that it's not valuable. It's just that there is so much you could do that is, as you point out, easier and valuable today and like requires pushing the bounds of research. But you have far higher likelihood of having something that's useful to give to customers this year with far less risk.

• 23:02Sarah Guo?

  And I don't mean to constrain people's ambitions, but the ability to give yourself multiple at bats with the wind at your back in terms of the entire field progressing versus trying to get out in front of everyone else with a, a multi-year research goal when there's like, it's just gold hanging out everywhere. You know, my, my orientation is, is I think similar here.

• 23:24Speaker 01

  Yeah, it's no GPU before product market fit. I think that's the takeaway.

• 23:28Sarah Guo?

  A lot slogan of the year. Okay, awesome. Uh, fun to hang out and talk about the news of the week.

• 23:35Sarah Guo?

  Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week and sign up for emails or find transcripts for every episode at no-priors.com.