E20: The Great Implementation with Raza Habib, CEO of Humanloop

Raza Habib is on the front lines of LLM implementation as CEO of Humanloop, a Y Combinator backed startup that helps companies of all kinds, small and large bridge the gap from API access to successful LLM deployment. Nathan sat down with Raza to hear what he has learned in the process of helping so many companies on their LLM implementation journeys – and he did not disappoint – as you'll hear, he shared a bunch of concrete examples of customer use cases, practical challenges that people face, and the strategies they use to overcome them. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics TIMESTAMPS: (00:00) Preview of episode (07:00) Humanloop’s mission and surprising customer base (12:10) Why a language model is so advantageous (15:00) Customization and understanding model performance (18:14) Sponsor: Omneky (18:35) Users graduating from playground experimentation to develop robust implementations (23:25) Raza’s two mental models for using AI (27:50) How customers are actually integrating API (31:50) Challenge of evaluation and feedback (43:33) Prediction: more fine tuning in the immediate future (47:00) RLHF myth-busting (57:12) Robust agents need feedback loops (1:02:27) AI productization challenges (01:07:32) Adopt or die moment for incumbents (01:11:19) Fast iteration cycles (1:15:45) Rapid fire questions for The Cognitive Revolution guests TWITTER: @CogRev_Podcast @RazRazcle (Raza) @labenz (Nathan) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries. From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected]. More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  RHF is like sex in high school, right? Everyone's talking about it, but almost no one's actually doing it.

• 0:05Nathan Labenz?

  I'm not a connoisseur of too many things in life, but one that I might claim connoisseurship of is AI analogies.

• 0:11Speaker 00

  I'm very optimistic about the rate of progress. So I kept making predictions. I thought, oh, that will take this many years. And again and again, I've kind of been beaten down to the point that I've kind of learned that this progress seems to be happening a lot faster than most might expect. I already have a prosthesis. It's my phone and I'm glued to it all the time. So it's only, it's only one step further. But I'd want to be confident that I'm not, you know, opening up literally my brain to some like advertiser model. Spending time in an interactive environment, I think, gives you a much better intuition for the capabilities of the models than just looking at benchmarks or, you know, numbers on test sets. Obviously, those quantitative measures are valuable and important,

• 0:55Speaker 00

  but you gain something qualitative and different through interactive, almost play with the models that I think is hard to gain if you haven't just spent some time with them.

• 1:05Nathan Labenz?

  Yeah, preach. Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas and together, we'll build a picture of how AI technology will transform work, life and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg.

• 1:29Speaker 02

  Before we dive into The Cognitive Revolution, I want to tell you about my new interview show, Upstream. Upstream is where I go deeper with some of the world's most interesting thinkers to map the constellation of ideas that matter. On the first season of Upstream, you'll hear from Marc Andreessen, David Sacks, Balaji, Ezra Klein, Joe Lonsdale and more. Make sure to subscribe and check out the first episode with A16Z's Marc Andreessen. The link is in the description.

• 1:58Nathan Labenz?

  Hi friends. Our guest today is Reza Havid. Reza has a PhD in machine learning from University College London, during which time he also worked at Google AI as a research intern. But at present, he's on the front lines of LLM implementation as the CEO of Human Loop, a Y combinator-backed startup that helps companies of all kinds, small and large, across a wide range of industries, bridge the gap from API access to successful LLM deployment.

• 2:29Nathan Labenz?

  Like many things, but even more so, LLMs are easy to learn but hard to master. It may take just a few minutes to get a starter prompt working reasonably well, but for companies looking to build LLMs into their products or to use them to automate internal process, most of the work remains in the form of capturing and studying data, collecting user feedback to identify failure modes, monitoring performance on an ongoing basis, and running experiments to quantify and compare LLM performance. In my experience, it's more extreme than 80/20, maybe more like 90/10 or even 95/5. Human Loop exists to make this process easier.

• 3:14Nathan Labenz?

  Now, full disclosure, I am a Human Loop customer, but there was no sponsorship or other consideration attached to this interview. I simply wanted to hear what Rosa has learned in the process of helping so many different companies on their LLM implementation journeys. And he did not disappoint. As you'll hear, he shared a bunch of concrete examples of customer use cases, practical challenges that people are facing, and the strategies that they're using to overcome them. If you're a regular listener to the show, first, thank you. I continue to be amazed by all the great feedback we receive, and I'm always honored when someone tells me that they heard about the show from a friend. But second, over the last few shows with Riley Goodside, the world's first staff prompt

• 3:59Nathan Labenz?

  engineer, and also Alex Albert, the creator of JailbreakChat.com, you've now got a pretty good overview of hands-on LLM use and of prompting in particular. So with that in mind, if you'd like to test your own prompting skill or nurture your inner red teamer, you might be interested to participate in the upcoming Hack a Prompt competition. This is a beginner-friendly competition that challenges users to trick the AI into saying specific things.

• 4:30Nathan Labenz?

  I'm a big believer in the importance of such crowdsourced testing. And for me, it's actually quite fun to mess around with AI models in this way. So I encourage you to check it out, either by searching hackaprompt2023 or using the link which we'll have in the show notes. This competition is sponsored by a who's who of AI companies, including Human Loop. And there are some $40,000 worth of cash and AI credit prizes available. The competition begins on May 5th and will run for three weeks. After which, we'll have the organizer, Sander Schulhoff, who is also the creator of learnprompting.org on the show as a guest to talk about the results.

• 5:12Nathan Labenz?

  Now, without further ado, I hope you enjoy this conversation with Razza Habib. Razza Habib, welcome to the Cognitive Revolution.

• 5:21Speaker 00

  It is a pleasure to be here. Thank you, Nathan.

• 5:23Nathan Labenz?

  Yeah, super excited about this. I think this is, as we're just talking, kind of a bridge from our prompting week to our medical week coming up. You are the founder and CEO of Human Loop, a Y Combinator company that is building tools to help people maximize the value that they get from language models. Just looking at your website, find the prompts users love and fine tune custom models for higher performance at lower cost. So I really am excited to just dive in with you and look at like, what are people doing today? How's it going? What are the struggles? What are the tools that you're building to help them overcome those struggles? And obviously, we can get into where we're going in the near future as well.

• 6:08Nathan Labenz?

  But for starters, just tell us a little bit about like your customer base and your business.

• 6:13Speaker 00

  Yeah, absolutely. So I think you're exactly right. Human Loop's there to try and help developers and founders build products with large language models to bridge that gap between here's access to this really smart API to a model and actually turning that into a useful, usable product. And it turns out that there's many steps along that path. It's not as straightforward as it might seem at first. So our customer base tends to be developers, founders.

• 6:39Speaker 00

  At sort of startups, increasingly actually founders at larger startups. We keep getting messages from the CTO of reasonably large companies being like, don't tell anyone. But me and two others have been working on this side project for the last three months. And we think it might change everything. Can you help? But predominantly, it's people who are technical, who are looking to take the API and build a useful product with it. Those are, that's our customer base and the use cases are really broad. We've got people doing coding assistance and code generation. We've got marketing copy generation. We have slightly more rogue use cases like disclaimer Nathan is one of our customers. And he uses it as part of a workflow automation tool.

• 7:22Speaker 00

  So there's a wide range of applications that we see. But a core thread of workflows and problems that people face, which typically are at the early stage, a lot of experimentation and prototyping. Large language models are stochastic, they require instructions for finding the right prompts, figuring out how do I mold my problem into a format the models can understand? Evaluation is another sort of big key problem area people have. So the tasks that people are trying to do right now are much more subjective than you might have done in the past. And so trying to understand if I generate an email or if I'm doing marketing copy or reading a job description, what does good look like?

• 8:05Speaker 00

  And understanding how to monitor that at scale is a challenge. And then customization and improvement. The models make things up. They don't know about company private data. So how do I get the model to understand my context, my company, be customized to my users? And those are kind of the three big problem areas that we see people having and that we help them solve.

• 8:24Nathan Labenz?

  Molding the problem to a format that the AI can solve. I think that's such a huge concept that people who have kind of tried chat GPT are not quite reckoning with yet in terms of the depth and really the degree to which problems can be fit to a format that AIs can solve them. The other thing that kind of resonated with me there is it sounds like a lot of your customers are building for themselves. Like, how would you break it down right now between? Customers of yours that are building applications that they're offering commercially to like, you know, third-party users versus folks, you know, you kind of mentioned the like CTO, and it is fascinating that like at the executive level,

• 9:08Nathan Labenz?

  these tools are so accessible that like where, you know, a lot of these leaders probably haven't coded all that much for a while, they can still dive in.

• 9:17Speaker 00

  Yeah, it's interesting. I don't want to know names, but it's, you know, it's companies you would have heard of. And it's usually the CTO plus like two or three really strong engineers. And they're trying to figure out what the limits are. I mean, we know some examples, right? So like HubSpot, Darmesh was like actually, you know, in the room with his engineers trying to figure out how to make this stuff work. And there's many examples like that. I guess it's a spectrum. So the larger the company, the more likely it is they're building for themselves, right? And so big companies come to us and they want to do two things. They want to build applications or they want to add LLM or AI features to their products. And they also want to automate internal processes, right? Those are the two areas. And then smaller companies tend to be building products for third parties. So if it's a startup, it's almost certainly using the LLMs as part of their product.

• 10:00Speaker 00

  If it's a large enterprise, you see a bit of both.

• 10:03Nathan Labenz?

  Yeah, interesting. Those two modes, thinking a lot about just kind of the mode of human AI interaction or the various modes. And I'm kind of working this framework, which I keep adding dimensions to. But I think the first dimension that I think the most about is like real-time co-pilot style, chat GPT style interaction where you are kind of doing your thing and the AI is helping you. And then on the other extreme of that axis, you have kind of highly structured workflows that, you know, have kind of designed architecture of routing information in such a way

• 10:46Nathan Labenz?

  that you can take advantage of, you know, AI capabilities as like one step, you know, or multiple steps within automated workflows.

• 10:53Speaker 00

  And maybe it's worth giving a concrete example of each of those just to help people hang their hats on. So on the one end, you might have something like GitHub Copilot that's sitting over your shoulder. It's watching the code as you write and it's generating suggestions for what you might have. It's quite open-ended. On the other end, you have something like CopyAI, where you put in a set of bullet points and you hit a button and it generates you marketing copy. And you have very little. You know, parameters for control as the end user or ChatGPT maybe lives on the end closer to Copilot as well. Or something very open ended that you can interact with and and get very different outcomes.

• 11:29Nathan Labenz?

  So it's interesting that you have both. And if I understood correctly, it's kind of the startups tend to be building this some sort of a more opinionated view on essentially a language model service. And they're using your tools to try to figure out, you know, how to make that perform well for kind of the uninitiated, right? The the person that's not super AI savvy. And then is it a spectrum? Or do you see it as more of a binary where the other side is like, okay, I want to, you know, potentially automate first line responses to email tickets, you know, in my CS system.

• 12:12Speaker 00

  So it's a good question. I think it's all about the user experience. And I think it's about thinking about different types of UX that work for AI or that work for L L Ms. I think that if you look at the breakout successes and I, you know, GitHub Copilot, ChatGPT being by far and away the biggest, but there are others, right? There's CopyAI, there's Writer, there's there's a whole bunch of other things. But if you look at those two, Copilot and ChatGPT, very different novel UXs, but I think they're successful because they've cracked some kind of UX insight as well as a modeling insight. So in the case of ChatGPT, I think the reason it's so much more accessible and so much more appealing to people than the original OpenAI models that were accessible through the playground

• 12:55Speaker 00

  is that the interface is fault tolerant by design. If you go to interact with something in chat and it doesn't get it exactly right the first time, you don't immediately give up. Right? The you're in a conversation with someone, you don't expect, your expectation is not for it to be perfect first time. Whereas the original UI that I think OpenAI exposed in the playground, which was this text box that you go and you put text and you hit enter. I think a lot of people bounced off that without giving it the time to discover its boundaries. And so they would come and they would like put something in and it didn't work first time and they go, oh, this is crap. It doesn't work. And so I think like the model was a lot better, right? Like, let's not pretend that GPT three point five or four wasn't significantly better. But I do think that part of what makes those chat experiences work so well and be so popular for people

• 13:40Speaker 00

  is they're very fault tolerant and they give you a chance to correct the model and have a few tries to get what you want. There's just a spectrum of different UX experimentation going on right now. So in some cases, you know, you don't want someone to have to think about the fact that they're interacting with an AI model. You they have a job to be done and you want to give them the shortest path to achieving that job to be done. So it makes sense to put a lot of rails on it. In other cases, you need, you know, more complexity, more generality. You're trying to answer a complicated question or solve a research problem or whatever it might be. Chat makes a lot more sense. In the GitHub Copilot case, like latency was super important. And so having a chat type interface is it requires context switching. It requires some way of, you know, knowing the code you're looking at.

• 14:25Speaker 00

  And it takes a long time to get a response from these larger models. Whereas having a model that sits over your shoulder, can see your code, knows your context and just occasionally makes a suggestion. Turns out to be really useful. And so I guess I wouldn't like my mental model isn't like a binary spectrum from like everything's on rails to everything's free. It's more we're exploring this space of quite novel user experiences. We're at the very early stages of doing this for AI models. And we're starting to discover some of the things that work and don't work.

• 14:53Nathan Labenz?

  Do you see the human loop toolset as inherently

• 14:59Nathan Labenz?

  more geared toward the narrow, you know, kind of guided use cases? It seems to me like that's probably what more of your customers are focused on as opposed to the, you know, much more kind of open ended. You know, free form experience.

• 15:17Speaker 00

  I think there's I think there's some there's some truth to that for sure. It's people who are trying to figure out how do I build a particular feature or product and do that in a way that's robust and kind of has reliable results. I guess there's like two parts to it, right? There's the like prototyping playground type environment that I think lends itself a lot to customization. But there's also these tools around understanding like how well is my model working? What is it actually doing? Is it performing well? Understanding the inputs and outputs and the data that flows through, which is, I think, like infrastructure that you're going to need irrespective of the generality of the problem. And in some cases, the more complex or more kind of free rein you give the model, the more important is to understand how well it's working and where it's breaking and where it isn't.

• 15:59Nathan Labenz?

  Let's maybe take a minute and just kind of run down the product and what it is. Because I think that's also a great way to think about all the challenges that people are encountering. So I, as you mentioned, I'm a customer of Human Loop in my capacity as an advisor at Athena, which is in the executive assistant services space. And we're trying to figure out, you know, and it's a target-rich environment. Like, what are all the things we can use? You know, especially now at GPT-4 quality, there are so many applications. And interestingly, one of the things we're kind of noticing is like, a lot of them don't even take that much prompt engineering to work.

• 16:40Nathan Labenz?

  Especially if we, you know, have a literal human in the loop, which for now, you know, we always do. When I was like, okay, I need something like Human Loop. I went out shopping and I was like, okay, I want a neutral playground. Because I, you know, as much as I do think OpenAI is awesome, I don't want to be entirely, you know, locked into them. I definitely want to be able to, you know, flip over and try Claude and stuff. I also really want to keep the history of my usage, which was something that OpenAI, you know, is really moving away from with their like, you know, we don't keep any of your history. It's a 30-day and delete, you know, kind of policy. It's kind of the opposite of what I was looking for because we have, you know, 1,000 executive assistants working in all these different contexts. We want to be able to see and know, you know, what are they doing?

• 17:25Nathan Labenz?

  And how is it working? And can we coach them? Can we detect patterns, you know, whatever? There's so much for us to learn about just our own, you know, operations there. So I was really looking for history, you know, looking for a way to kind of graduate from a playground, like interactive exploration type experience to something that I could then build into workflow. So I wanted like the flip from the playground to the API mode, which you guys offer. And then I'd even gotten into, you know, all the advanced features yet in terms of like running experiments and collecting feedback. So, you know, it was those kind of first anchor things that led me to you. Comment on those kind of stuff, on those kind of features as much as you'd like. And then I really want to get into, you know, kind of what is the frontier for me?

• 18:09Nathan Labenz?

  Which is like this feedback, experiments and tools as well, which you have. Available too.

• 18:14Speaker 00

  We saw this like repeated pattern of basically people struggling with the complexity of managing a lot of different versions of prompts and trying things out and it being highly experimental. And there being also like knowledge being built up through that experimentation, right? When people do prompt engineering and they interact with the models, they start to implicitly build up intuitions about what does and doesn't work. And then that knowledge was being split between the OpenAI Playground and Excel Spreadsheets and in GitHub Code. And there wasn't like an easy way to manage the prompt experimentation and that experience. And so that's part of, I think, what you've been, you know, using very heavily, which is we have something that looks a little bit like the OpenAI Playground. There's a chat version and a completion version, but it's multi-provider.

• 18:58Speaker 00

  And it also has a few other kind of key differences, which is it allows you to map the same model across multiple inputs simultaneously. And this is like a key thing when you're trying to develop something that's going to work, not just for the one instance you're looking at right now, but you're trying to see, okay, is this going to work in general and robustly? Okay, maybe you don't look at a huge test set all at once, but at least having 10 or 20 use cases that you can say, okay, I'm going to prototype a sales generating email or I'm going to prototype something and I'm going to chest it out at all these different inputs and make sure it at least works for those. And so we see this sort of interactive environment where people will be product prompt, like creating a prompt template and then trying it out on a lot of different things at the same time and seeing how well does that work?

• 19:40Speaker 00

  And they typically then graduate from there, as you said, to deploying that as an API endpoint and using it in production or as part of a product. And at that point, I think that the core challenges are how do I go from quite good performance to robust and good enough that I trust this in production? And even, you know, upstream of that, how do I even know if it's working? Because these things are so subjective. It's really difficult to to understand those bits. And so those are, I think, you know, that's when human really comes into its own. I think that the early prototyping parts are best in class and people use them, use them. I think, you know, they add a ton of value. But in the journey of going from idea to like robust deployed application, being able

• 20:25Speaker 00

  to understand how well things are working and to take actions to improve them, that's the that's the really critical component.

• 20:30Nathan Labenz?

  Could you run down a handful of actual tasks? Like, you know, I'd love to just hear a parade of kind of across different types of businesses, different sectors. What are the sort of object level?

• 20:42Speaker 00

  So early on, the majority of people, I think, when GPT-3 came out, were doing writing assistance of some form, right? This was the first, you know, obviously, this is the first thing people think of. So you have like marketing writing assistance, sales writing assistance, fiction, et cetera, right? So companies like Pseud rewrite, Copy AI, Jasper, et cetera. Different varieties of, I want to overcome this blank page problem and I want to write more quickly and better. And there's a whole range of use cases there. Then you have people kind of doing various forms of like knowledge work automation. So you have a lot of different legal use cases, people doing contract review, summarization, question answering, extracting things from documents. That's like another very large bucket that we see a lot of usage in. Then various forms of like process automation.

• 21:27Speaker 00

  So we mentioned like sales email writing, right? But within sales, there's just a lot of mundane tasks that people do. I have a meeting, I have the transcript, I need a summary, I need to populate this in my CRM, et cetera. So there's like a lot of people, I think, building in that space. We see a lot of use cases for, in the medical domain, typically around just removing some of the grunt work. So doctors have to do like a significant amount of paperwork around insurance claims and filling out forms in the right way and summaries and things like that. So we've seen quite a few companies building in that space. Code writing is another one, right? So GitHub Copilot is just one of many different ways you might try and build code assistance. And so we've seen a few varieties of those, whether that's people doing natural language to SQL so that business analysts can query their own databases.

• 22:12Speaker 00

  Or it might be people building an entirely new IDE from the ground up. One of our companies is a company called Cursor and they're trying to reimagine like what would an IDE look like if I started with an LLM on day one? And that's a, that's another use case. And I could, I could go on, there are so many applications being built now. There's a real Cambrian explosion.

• 22:31Nathan Labenz?

  Yeah, that's amazing. How do you even think about kind of characterizing all of those tasks in terms of like?

• 22:40Nathan Labenz?

  The impact that they make on people. Do you think of them in terms of like minutes saved? You know, doing it manually or, you know, as a human versus kind of the LLM obviously takes whatever its latency is. And then you have some, you know, review overhead. What's your framework for that?

• 23:00Speaker 00

  That's a great question. I don't know if I have a framework, but I can tell you kind of just off the cuff what comes to mind, which is it feels to me that there's sort of like the V naught version of using AI or LLMs, which is like, I'm going to use this to do an existing task a little bit faster or to automate something that someone was already doing. And this is the version that I think teachers worry about. This is the sort of cheats on exam version, right? Like lets me do what I was doing before, but a bit better or a bit faster. And I think there's the interesting version, which is like, I now have an assistant that lets me do things I couldn't do before, or to push what I was able to do a little bit further. And here I think of like some of the more advanced coding tools as being really interesting.

• 23:39Speaker 00

  So I've seen quite a few examples now of people using languages they're not familiar with because the coding assistant is good enough that they can move across code base. So I've seen, I've been speaking to friends who maybe would be backend engineers, but have found themselves able to contribute a bit more elsewhere. Or, you know, I've got a tool myself that I kind of built for personal use where when I'm taking notes, it tries to critique and extend what I'm writing. So it's not like actually taking a set of bullet points and filling it out, but it's doing something more like what a colleague or friend might do of being like, have you thought about this? And I feel like that actually allows me to do things I wouldn't have been able to do on my own. So there's a bit of a spectrum. I'm personally more excited about the tools that give people more capabilities than they would have had previously. But I think there's a lot of value in both, right?

• 24:24Speaker 00

  There are large cost centers. One of our customers is doing a huge amount of customer service automation. They've got several hundred thousand customers. And so customer success is a big cost sink for them. Being able to accelerate those people and reduce headcount there is obviously valuable. On the other hand, you have customers who are augmenting lawyers where it's less about reducing headcount and more about letting the same person do, you know, do the work better and leverage the tools in that way.

• 24:51Nathan Labenz?

  Is it a particular tool that you use for the critique of writing?

• 24:55Speaker 00

  I built my own.

• 24:57Nathan Labenz?

  If it's something I could try, I'd be interested in trying it if it's a. Too bespoke, then maybe it's hard to share. But that's, that sounds really, I have not had a lot of success with the writing tools on the market today. But I mostly just, you know, if anything, I'll ask for a little bit of kind of help clarifying, simplifying from G P T four directly. But that does sound like an interesting UI or UX maybe is a better level at which to think about that.

• 25:29Speaker 00

  Yeah, it's, it's interesting. I think it depends what you're looking for. And I think a lot of the early success of certainly writing assistants in kind of the marketing space are around getting to, either around brainstorming, come out with more ideas, or they're about volume of kind of not necessarily like super high quality content, right? You're not trying to write something.

• 25:47Nathan Labenz?

  Yeah, you're throwing it all into an optimizer anyway, so.

• 25:50Speaker 00

  Yeah, you're not trying to win a Pulitzer Prize. You're trying to produce like a large volume of like medium quality content. Whereas when I'm writing notes for myself, I'm, I, you know, I'm trying to produce a small amount of stuff that I'm going to find useful. So I guess the, the use case is probably slightly different.

• 26:05Nathan Labenz?

  Yeah, okay, that's really interesting. And then the, what you didn't kind of speak to as much there, which I imagine must be a significant part of the business, is this kind of scaling things that were previously unscalable. So this is, you know, as we're looking through again, like the, you know, Athena client base and looking for examples of people that have done great things with A I tools already. One that we see is like the ability, you know, almost all of our clients are like recruiting in some way, shape or form. So what we've seen there is like the ability to scale recruiting workflows that you would previously delegate to a person.

• 26:45Nathan Labenz?

  But now you can kind of systematize, probably honestly do better like profile evaluations even than, you know, the human was able to deliver in many cases, but certainly at, you know, way more scale. Like the, the ability to kind of bring some of these functions where people maybe used to use a firm or whatever, you know, in-house and do it for like a one, two percent of the cost is, is kind of insane. So I guess the question there is like, do you see a lot of that kind of activity? And then also really interested in like, how do people wire that up? Like, are people using Zapier in your experience? Or are they like, you know, stitching stuff together themselves out of Google Sheets? Like, where are the actual A P I calls coming from?

• 27:28Speaker 00

  So the majority of our customers are integrating the APIs into some form of front-end product. So they have some UI that they're building where they're going to be making a call to Human Loop to call the model. And as part of that, we're going to log the data that flows through as well, so that they understand how well things are working. We do have some people who are doing more low-code, no-code automation. Typically, we've seen people use Bubble quite a lot for this, actually, and then some use of Zapier as well. But the majority of our customers are not on that kind of no-code, low-code end of the spectrum. They're closer to, I'm integrating this into a product, and I'm going to be using Human Loop as the backend, the way to understand how well

• 28:08Speaker 00

  things are working, the way to build my data sets to fine-tune and improve stuff. But the user experience and the product is being built by them.

• 28:16Nathan Labenz?

  Do you think that's a reflection of the general market of, you know, LLM usage? Or is that just, you know, maybe more of a reflection of like, you're in the YC network, and so you're sort of wired into product-building companies?

• 28:31Speaker 00

  So, you know, in some senses, it's impossible for me to know, right? Because I've only seen the sample of the market that I've seen. Well, actually, no, maybe that's not true, because we do get, like, we get a huge amount of inbound, right? And we don't, you know, we narrowly focus on builders and founders and engineers. We don't build for everyone. There definitely is a large volume of people looking to do no-code, low-code automation. And, you know, I think that Human Loop's probably not been built for them. So some of them end up using it anyway, because they love how, they love the playground, and they love that kind of experience. But we see less of that, because it's not what we've optimized for.

• 29:10Nathan Labenz?

  Let's start with maybe evaluation. You've gone through this process of kind of experimenting. You're in the playground. Okay, you've got something that's like worked for your, you know, however many different examples you could kind of think up on the spot. By the way, I find that also pretty effective as like a Teaching paradigm for like intro, you know, to what the hell's going on here in the first place, you know, let me run this. And I've done things like, I'm going to put Kobe Bryant, Santa Claus, and, you know, Joe Biden in like the three, and then we'll see how, you know, just a simple prompt like varies with those variables and that that definitely like helps people grok, you know, what a language model is.

• 29:50Speaker 00

  I think for companies adopting LLMs for the first time, or even academics coming to this from other parts of machine learning, spending time in an interactive environment, I think gives you a much better intuition for the capabilities of the models than just looking at benchmarks or, you know, numbers on test sets. Obviously, those quantitative measures are valuable and important, but you gain something qualitative and different through interactive, almost play with the models that I think is hard to gain if you haven't just spent some time with them. And that's partly why I think these playground environments, these REPL style environments are very important.

• 30:27Nathan Labenz?

  Yeah, preach on that point. I think that is so, so important. I mean, good God, I see so much misunderstanding that could be remedied with just a little play.

• 30:37Speaker 00

  Yeah, I mean, like you see, you know, a silly example of this is Noam Chomsky wrote an op ed in the New York Times, basically like outlining all the reasons why LLMs would fail and giving a bunch of things that he thought LLMs would never be able to do, which they can already do. You know, all of the examples he gave, you can just shove into the playground and run it and it works. Like literally the concrete examples, not even versions of them, but specifically the ones he gave. And you just think like, how hard would it have been to just try?

• 31:07Nathan Labenz?

  Yeah, I know. It's baffling. I find that so true. Okay. Well, maybe leave that aside. So we're using human loop, we're doing the playground, we're actually, you know, making progress, getting somewhere. And now you've got this challenge and implementation of evaluation, which is connected to feedback. So how do you think about that? One, for context, for users, I find this particular stat kind of insane. In the GPT-4 technical report, they report a seventy thirty preference ratio for GPT-4 over three point five, which, to me, given the qualitative difference in capability, and like the tenth to the ninetieth percentile on the bar exam and all that, you know, stuff,

• 31:51Nathan Labenz?

  that seems like a pretty low ratio. So that suggests to me that like, evaluation is hard, and it's probably something that a lot of people are kind of diluting themselves and, you know, seeing patterns that aren't there. I mean, this sounds just like a total mess right now.

• 32:07Speaker 00

  Yeah, it's certainly tricky. It's especially tricky for the tasks that are more subjective in nature, where actually, okay, maybe you can, you can sort of assess the quality, but in some sense, the correct answer is what your user says the correct answer is. Right? If you are a writing assistant or something like that, then that's much more subjective. There are, you know, more concrete examples. If you're writing code, then obviously it needs to run and, you know, you're generating an SQL query. Maybe it's easier to have a, have an objective metric of success. But even for question answering or summarization, like a lot of the tasks that people are doing, pure objective evaluation is quite hard. And so within human loop, there's, you know, the option both for quantitative evaluations. You can define metrics, et cetera, but we also give you the tools needed

• 32:51Speaker 00

  to capture human feedback, both internally, but most importantly, from your end users. And so the way that works is we have a feedback API. And so every time you sort of log to human loop or you use our generate call, we return you a unique ID for that session. And then you can capture different types of user feedback from people interacting with your application to get a sense of how well things are working. And typically, we see people get success with three types of feedback, sort of explicit votes. So things like you might see in chat GPT, thumbs up, thumbs down. Those are useful, but you get a small proportion. The vast majority of people don't touch those buttons. And you're getting a kind of more extreme skew. Implicit signals of feedback. So people look at the actions people take within their applications.

• 33:36Speaker 00

  If I generated a sales email, did they send it? Did they copy the marketing copy and use it somewhere else? Did they hit the regenerate button? Right? If the person's sitting there generating again and again, it's probably a sign that things didn't work the first time. So those signals get logged as well as like senses of how well things are working. And the final one is sort of textual corrections. So if you're generating text that the person can edit and they edit it before using it, then capturing that can be a really useful signal for figuring out what went wrong or how you can improve it later, whether that's inspecting manually or just fine tuning a model on corrected results. People can define their own feedback types, but those three categories of kind of explicit actions and corrections, kind of votes, actions, corrections are the ones we see people getting

• 34:19Speaker 00

  the most success with sort of in practice. And then we give you an aggregated view of like, how is that varying over time? And that allows you to kind of get a sense when you make changes. You know, we saw this for a lot of our customers when they made the switch from GPT-3 text Da Vinci two to four or three point five when it first came out. They could just see their, you know, their user satisfaction charts just jump up almost overnight. And I think the vast majority of people who didn't have tooling like this didn't really know. Like we changed the model, did it get better? Did it get worse? Seems better, but like, how do you know if it's actually working better for your customers? And having that feedback both gives you this like high level view of how well things are working, but then allows people to take more granular interventions.

• 35:01Speaker 00

  So something else that we see people doing is they will filter for the failure cases and then manually inspect some handful of those to understand, okay, what went wrong? Was it the data lookup that I did? I grabbed the wrong context information. Is the prompt sort of wrong in some way? Can I tweak that? Is the model just like not capable at this task? And they will do this loop where they'll kind of filter, for examples that didn't work that well, reopen them in an interactive environment like the playground, form some hypothesis for what went wrong, take an intervention and redeploy. And because they have all the monitoring and the metrics in place, they're able to see what the impact of that was. Or human loop also allows you to run a direct comparison experiment. So for some subset of users, you know, be running an A B test or a multivariate test.

• 35:46Speaker 00

  We are seeing, okay, did this change result in a, an increased performance or not? And so we see people run that loop quite often, even before they're fine tuning models. Like they do it when they fine tune as well, and we can maybe chat about that. But this process of like, understand how well things are working, find some of the edge cases, find the things that are going wrong, fix them, repeat is a very common workflow we see.

• 36:07Nathan Labenz?

  Yeah, and the iteration time on that can be, you know, down to minutes in theory, right? In, in some cases, because you're literally just. Giving a slightly incremental instruction or whatever. I have a ton of different follow up questions that are coming to mind that I want to ask. On the nature of feedback, was there, would you include something like generating multiple outputs and then kind of asking the user which they prefer as like an implicit? Or is that just something that you don't see that much? Because I know Anthropic does that in their loop and it seems easy to be like, I like this one more than that one.

• 36:45Speaker 00

  So we see that when people are doing internal evaluation. That's very common. In fact, I don't think, so Anthropic does this like binary selection. OpenAI does ranking of the generated outputs. We have other customers who are building custom models and they'll do preference as well. But that's kind of a sucky user experience, right? And so when people are collecting in production data, they tend not to want to generate multiple options and get the user to pick one. Some companies have done this or experimented with it. I know that like AI Dungeon was doing this for a little while, where when you generated, they would generate a few options and ask you to pick.

• 37:19Nathan Labenz?

  Choose your own adventure. It's the purest form.

• 37:22Speaker 00

  But in a choose your own adventure context, I think it's okay. But for most applications, it's not really what the user wants. And so people are more reticent to do that kind of sort of preference gathering in production. But for internal evaluations, we see it a lot.

• 37:37Nathan Labenz?

  That's fascinating. I kind of don't, I don't know. To me, it seems like an okay experience. This is something that we're considering at Waymark where we're making video content for users. And it's a cool experience that we have right now where you basically find your business online, give like a very sort of Dolly like, you know, short prompt on what you want to see a video about. And then we return a fully formed composition for you. But I've been kind of thinking like, well, what's better than one fully formed composition, you know, maybe two or even potentially more than that. You know, if you watch Mad Men, right, they like have multiple pitches come through.

• 38:18Nathan Labenz?

  So if there's some way in which, you know, it can kind of echo, you know, what it feels like to be a big company. What do you think is, what do you think is bad about that?

• 38:27Speaker 00

  I think it's very context dependent, right? So I think in that case, it's maybe not so bad. And I think it'll be fine. So two things come to mind. One is context dependent. If you're doing something like code generation, right? Or text completion or in GitHub Copilot. Like, I don't necessarily want to choose between a bunch of different pieces of code. Like, I want a completion that's kind of good. Maybe I can tab between a few. But generally speaking, I want it to work first time. But the other thing is just like the, the goal that the developer has in mind. So I think that like with the preference data, they're planning to fine tune the models with RLHF on this preference data. And so like, then I think it makes sense to have it in that format. Whereas where a lot of, what a lot of our customers are doing is like, they're trying to do monitoring or evaluation.

• 39:09Speaker 00

  They want to know like, for the parameters that I have at the moment and the setup I have, like, is this working well? Is it causing problems? Are my customers, you know, getting success? When it's going wrong, like, why? And there, I think like a single, you know, evaluation is fine because you're evaluating the same model. So it's a little bit of what you're trying to do and a little bit of the context. I think in some UXs, it'll be fine and some it won't.

• 39:31Nathan Labenz?

  Yeah, I also had a similar kind of thought on like, I wonder, maybe just have a take on this. Why do you think people don't make a more explicit trade between like feedback and access? It seems like that, that is almost like the AI successor to like the ad driven model, right? Is like the free, at least for a while, like the free version is based on you have to like, help improve the product. But I don't see much of that. Do you have a sense for why that wouldn't work or why people don't?

• 40:05Nathan Labenz?

  Don't make that trade with the public more often?

• 40:08Speaker 00

  The honest answer is no, I don't have a, I haven't thought about it that much. So I don't have a strong view there. My suspicion is that people are quite happy to give this feedback anyway, and a lot of it is implicit. And so maybe, maybe you don't need to make the trade because you're getting a lot of that anyway.

• 40:28Nathan Labenz?

  From what I've heard, it seems like the rates are usually fairly low. Like you had said, you know, the kind of thumbs up, thumbs down, people don't hit too much.

• 40:34Speaker 00

  Yeah, the rates for explicit sources of feedback are definitely quite low.

• 40:37Nathan Labenz?

  So if you're, if you're clever enough to structure it in, in a smart way, then you can kind of work around this problem certainly. But yeah, I don't know. There seems to, I feel like there's some sort of inconsistency here where like the received wisdom, which seems to check out to me is like,

• 40:52Nathan Labenz?

  you know, the fast, like the best RLHF cycle wins in probably a lot of domains. And yet people are not pushing as hard as they could to

• 41:04Nathan Labenz?

  build that like data engine, you know, and really getting it to turn over as fast.

• 41:08Speaker 00

  I think a lot of this probably comes to stage as well. So I think like, you know, I saw in the, in the notes, one of the questions you were going to ask me about is about fine tuning and whether, you know, a lot of it's happening. And the, you know, so are people fine tuning a lot? And the answer is like, in general, not that many. But the people who are fine tuning tend to be the ones who are further along or slightly more advanced in their journey around building LM products. And I think a lot of this has to do with sort of like, what is the low hanging fruit to improve things? And early on, you can probably get quite a lot of juice out of prompt engineering, which has a very fast feedback loop. And so like, if you're spending all of your time making those kinds of changes, then maybe this feedback data isn't yet

• 41:48Speaker 00

  that valuable to you. And when you get to the point where you're now thinking about actually fine tuning custom models and going a little bit further ahead. Then I think, you know, then it starts to make a big difference. And I think the majority of people are still in that first camp. And and I think the reality is it'll stay a bit of both going forwards. But I suspect it's a maturity question, right? So I think about companies, you know, one of our, there's a Y C, you know, Y C company called Find. Perplexity is another example, but I know the Find guys quite well. And they're building L M based search for developers. So kind of think like the best version of Google married with Stack Overflow. You put a query in and it actually generates you answers with code snippets, et cetera. They make heavy use of fine tuning and feedback.

• 42:32Speaker 00

  And they're, they've been able to get substantial model improvements as a result of this. So they're, you know, probably have the best and best in the world model now for this task because of that cycle they've been running. But they also have like significant machine learning expertise. At the time that they were doing this, all of this stuff was really hard, right? Like human loop didn't exist yet. I don't think there was that many other tools around fine tuning. And so I think for a lot of people, fine tuning is a little bit novel, a little bit scary. They're concerned about, you know, what happens when the next model comes out. But also they just have so much ground, like so much juice still to get out of prompt engineering that they haven't got there yet. So I think maybe what you're suggesting will come true in the future. I think like my prediction is, and I, and I think this is a fairly contrarian take.

• 43:17Speaker 00

  There will be a lot more fine tuning in the near future.

• 43:20Nathan Labenz?

  Yeah, that's really interesting. And the, the other big thing that's just kind of changing, you know, in this moment right now is you've both called this kind of the stable diffusion moment for language models. And, you know, kind of until you had that, if you just looked at like what, you know, you could buy straight away with, you know, now G P T four, even before that, you know, three point five versus what you could fine tune, given, you know, the available base models or like, you know, OpenAI's fine tuning is still just the original Da Vinci. You know, it's the best thing that they're offering to the public. And that comes also at like.

• 44:00Nathan Labenz?

  Now, good god, like 100 times, you know, more expensive than than turbo. So that like relative capability of the off the shelf versus like what was available to fine tune. Seems like it's probably also kind of part of what's held the fine tuning back. But that seems like it's flipping literally right now, right? Because we're seeing the stability and a bunch of other models, you know, that are pretty like the llama, you know, class of model. Even if not llama exactly, but like densely trained, you know, pretty inference efficient, but strong. You know, open source RLHF model or libraries coming online.

• 44:43Speaker 00

  Yeah, I mean, my assumption is we're we're at the very early days of this, right? That so like one of the questions we get asked most is around privacy and private deployments. And people being able to have their own models. And, you know, there's a there's a big fraction of the world that will happily use a third party API. But there's also a big fraction who who really don't want their data leaving their servers. And for whom like a custom fine tune is the only option. And so like, there's definitely a strong like demand for this. And increasingly, there are good open source models coming out. I think the first batch of open source models, in all honesty, just wasn't that good, right? So like OPT and Bloom were unfortunately like significantly behind the equivalent closed models. But that gap is closing, right?

• 45:28Speaker 00

  Lama is not a permissible license, so you can't use it for commercial purposes. But it is a model that is out there that is performant, right? It's very competitive with the best closed models. And I imagine it's just the start. I think like large language models are somewhat different to the image models. They're harder to train. They require a lot more scale. The models are bigger. I don't think it's going to be as easy as it was for stable diffusion, where you can, you know, have a four gigabyte model. Or it's, you know, very quickly people had running on phones and laptops, et cetera. The smaller versions of large language models are not as good. They are harder to train. I think there's going to be, it's not going to be as fast or, you know.

• 46:10Speaker 00

  But I also don't think that for GPT three level models, we have to wait that long for there to be a lot of open source alternatives. Or a lot of competition in the marketplace. I think if you want the biggest, baddest, best LLM, you're probably still going to be going to OpenAI or Anthropic for a while. But you don't necessarily need that for a lot of use cases, especially if you fine tune. And the main, you know, one of the main reasons we see people fine tuning is, you know, cost and performance. So they're looking for smaller models that can do equally well or better on the task that they care about.

• 46:47Nathan Labenz?

  So what role do you envision yourselves playing at human loop in that future, potentially, you know, coming soon wave of fine tuning activity? Like, do you run the fine tuning processes? I don't expect you're going to say you would like host models, but like, I'm going to have all these kind of like the full, I'm going to have the full stack of problems if what I've got is like a downloaded, you know, instance of stability LM or stable LM and a dream, right? So like, what parts of that do you feel like are natural for you to help with?

• 47:24Speaker 00

  At least early on, I imagine that we will partner with people to help with this. So what we do a really good job of is we have all of that infrastructure in place that captures the data that you would need to make sensible decisions about fine tuning and improvement. We have that evaluation data, the feedback. So it's very natural. And we already have fine tuning integrations with the large language model providers that have these APIs, right? If you want to fine tune GPT three and do that well, human loop makes that a very painless process. And so I imagine like partnering with others who have similar APIs and increasingly doing that initially, because I think to do that well is probably its own entire company to start with. And maybe over time, we do more and more of that. But at least, you know, in the early days, in the same way that we

• 48:08Speaker 00

  don't really want to own the model right now. I think that's a, you know, a big and separate problem. I don't think we probably want to own the fine tuning process initially, but that might change over time.

• 48:19Nathan Labenz?

  Tell me a little bit about where you think RLHF is today in terms of like,

• 48:28Nathan Labenz?

  Who should use it? You know, at what scale? Does it start to make sense? Like, what are? I think there's a lot of, you know, expectation that like, everything is going to be smooth with RLHF. And I think you have an interesting kind of, you know, maybe not so fast take on the difficulties of implementation there.

• 48:46Speaker 00

  The like, non-PC version of this is like, I don't know, RLHF is like sex in high school, right? Everyone's talking about it, but almost no one's actually doing it. And we get asked about it a lot. And I think like, it is true that RLHF leads to much more performant models, right? The gap between the RLHF GPT-3 models, if you look at the sort of reports or the anthropic ones and the ones that are not, is large. So there is a big performance gain to be had through RLHF. Some of that is just making the models like easier to use, right? So it's not necessarily even about increasing the capabilities of the base model, but it's about making them more aligned with what the user wants out of them. That said, for a lot of the tasks that people come to us and practice,

• 49:28Speaker 00

  where they're looking to do some specific thing and they don't need the full generality of the model and they have a data set, supervised fine-tuning is more than sufficient for them to get good performance. And it's a lot easier. And so in general, you know, I'm, I'm pro RLHF as a concept and where, you know, it makes sense to do it if you, if, if you have a big enough problem. But my advice is always to like, try to find the simplest solution to what you're trying to do first. And only if that doesn't work, go for more complex or harder things. And so if people are thinking about, you know, basically in order of complexity, I think you have prompt engineering, which is like very straightforward and very accessible and very fast, has some limitations to how far you can take it.

• 50:10Speaker 00

  You're never going to get a 10x latency improvement through prompt engineering, right? Supervised fine-tuning is a, is a little bit harder. It's more involved, but you can get gains that you couldn't get through prompt engineering because you can get big, you know, cost or latency improvements that are just never going to come through prompt engineering alone. And RLHF can give you significant performance improvements on top. Pick the, you know, the appropriate tool for the job, like how, how complex do you actually need this to be?

• 50:36Nathan Labenz?

  What is it that makes the RLHF so much more? Challenging in your experience?

• 50:44Speaker 00

  It's a multi-stage process, right? You're first training this reward model. You've got to get all of this annotated data. You've got to make sure you do that process correctly. You're then training a reward model on that. So you have another model to train. And then after that, you're doing that, you know, you're doing an RL step. So there's just more places for things to potentially go wrong. There's more complexity to handle. If you look at the kind of appendices of the InstructGPT paper, there's a lot of nuances and subtleties that you have to get right to make this work robustly. Whereas supervised fine-tuning is pretty straightforward, relatively speaking, right? You just need a data set, not even necessarily that larger data set for some of these large models. I was speaking to a founder last week who's been auto-generating tests. He hand-wrote 50 examples and fine-tuned Da Vinci II on that.

• 51:29Speaker 00

  And now he has a pretty good model. And 50 examples is not that many for him to be able to get, you know, a pretty reliable fine tune. With more examples, he could probably do it with a smaller model. To do the equivalent with RLHF would have been really hard for him and probably just not the appropriate tool.

• 51:44Nathan Labenz?

  What kind of, how does that actually cash out in terms of failure modes? Is it, is it that people like can't make the process work because it's too intricate? And there's just too many parameters to wrangle? I mean, I've heard that kind of account in the past. But then you also see these like mode collapse type reports. And, you know, there's this sense of like, you have, there's, you kind of have like a lot of dark matter in terms of other domains of, you know, possible, possibly much worse model performance as a result of the RLHF. Like, what do you see there in terms of how it actually goes wrong for people?

• 52:25Speaker 00

  I mean, it's more the, more of, at least from our perspective, it's more the former than the latter. It's just like very hard to actually get it to work reliably. Like, do you even get the model to, to learn what you want to do at all? Or, or to do it robustly? Or, you know, small changes in the reward model can have quite big impacts. Reinforcement learning in general is just a lot more finicky than supervised learning. Like RL is harder. And that was, that was my experience, even as a researcher. Like, every time we got rid of an RL component, everything seemed to like, just be a little bit more robust or a little bit more reliable. So it's more that, in terms of things like mode collapse. So like the RLHF models, I think, are in some senses less capable than the base model conceptually, or in some ways, but they're much harder, they're much easier to get to work, right? Like, I think a lot of what you're doing is figuring out how

• 53:10Speaker 00

  do I get the capabilities of this model to be easy to access. People like Riley, who I think you had on the podcast recently, you know, are wizards at taking the base model and having spent a long time playing with them can figure out the right way to prompt them to get the output they want. The nice thing about the RLHF models is you don't have to do that as much, right? I think you mentioned this earlier. You can more reasonably expect an instruction in natural language to get you the outcome you care about. You are sometimes paying a little capability cost. Like, certainly the people who tell us to use the models for more creative tasks, like natural, you know, like fiction writing assistance or things like that, they use the base models where they can. They prefer them to the RLHF models because they get a greater variety of output.

• 53:54Nathan Labenz?

  My kind of working theory on this agent moment, you know, which you said at the top, is like, obviously, all the rage right now, and yet, mostly, they don't work. My sense is that the RL paradigm is like a pretty good fit there, and that there's just going to be a ton of like, you know, what didn't work, and then like, very concretely, what did work in terms of like, you know, code as policy style, API calls generated, or what have you. So I guess, two-part question there, like, how quickly, do you think that that's right? Do you think like, that paradigm will kind of lock these little, you know,

• 54:33Nathan Labenz?

  code generator or agent models into like, form over the not-too-distant future? And then, how does human loop think about playing in that space? You have like, the tools offering, but it's been a minute since we talked about that.

• 54:47Speaker 00

  So it's maybe worth just mentioning, like, what tools are in human loop? You know, one of the problems that a lot of the companies that we work with face is figuring out how to get private company information into their models. And so... Tools in human loop are basically APIs that take text as input and text as output. And they're used to augment the models with extra information. So for example, you know, we have a tool that is that accesses the Google API or the Serp API. So if you do a factual query, it can like look up from Google or you might connect Wikipedia or something and can pull in information from there and include that in the context of the model to help you get a more factual response. And you know, Google and Wikipedia are public sources of information, but increasingly people are connecting private sources.

• 55:32Speaker 00

  So they'll connect a database or they'll collect a an FAQ or something like that and give the model access to that at inference time. So when someone comes and asks a question, the model can pull information or we pull information into the model's context and allow the model to use that. So tools are essentially a way to add extra information to the model, which is slightly different, I think, from some of the other ways that sort of tools have been used up till now where, you know, people have the GPT three plugins, for example, allow the models to take actions. Human Loop doesn't let you do that yet today because we haven't found it yet to be reliable enough. So we're very excited about it in the future, but we see like less of that up till this point.

• 56:13Speaker 00

  In terms of like, will RL just make the agents like fix the agent problem? My suspicion is no. I basically think that the issue with agents is that you start with a core module that has like some reasonably high probability of error. I don't know, eighty five or ninety percent accurate. And then you chain these things together. And so the longer the chain gets, right, like naught point eight five or naught point nine to the power N, as N gets large, that thing tends very quickly to zero. And so your your probability of it working decreases rapidly with the number of steps in your chain. And you're essentially running open loop control. You have a system that's like getting no feedback as it goes. And so if it makes errors, the errors compound.

• 56:54Speaker 00

  So I think that probably the solution will actually be some form of feedback mechanism, not necessarily human feedback, but some way for the models to like heal their agents or heal their chain. So if it goes wrong, there is some other mechanism, another model maybe, that is able to look at that and go, no, that bit's wrong. Try again, or here's what you might have done, because I think you need some way to course correct. Without course correction, I just don't see how agents become reliable.

• 57:20Nathan Labenz?

  Yeah, I mean, I think those kind of work together, though. Like, the multi-agent system is definitely... When people talk about agents, a lot of times you dig into these things. It is kind of a... It might be a single-language model, but it is kind of powering some sort of multi-agent configuration or ensemble or something.

• 57:38Speaker 00

  And I think you need that. I think you need that for it to have a chance of working, to make it robust. Some people may be getting these working. We certainly see a lot of people chatting about it on Twitter and posting very cool demos. From the production side of seeing people actually put these things into production and speaking to people who are trying to do that, we just hear again and again and again that, like, they try and they don't quite succeed yet.

• 57:59Nathan Labenz?

  Well, what do you think I'm missing about this? Because what I kind of imagine happening with this feedback loop is, basically, like, what's the most automatable sort of feedback? Is like whether the code executed successfully or not, right? And then, you know, on the far end is like,

• 58:18Nathan Labenz?

  "Do you prefer this sonnet or that sonnet?" Is like, you know, probably the hardest. Interestingly, like, art, visual art, like, gets very, you know, comparatively close because you can make that judgment often in, you know, a fraction of a second. But, you know, nothing works at the speed of like the code worked or not. So it seems like there's this kind of very natural, pretty substantial dataset that's going to kind of just accumulate in the logs almost... Maybe not super usably automatically, but like with the help of human loop, like, that stuff shows up, right? There's only like so many way... There's only so many APIs. There's only so many ways to call the APIs that are right. Obviously, you do have like the world is shifting and kind of... This is never like a done problem because people change their APIs.

• 59:03Nathan Labenz?

  Although I think it's going to become on the API providers to like, you know, come up with a way to make sure the language models continue to work before you know it.

• 59:10Speaker 00

  No, I don't disagree, right? So, as I said, I've been talking about agents and we've been talking about them in human loop for like well over six months now. We are very excited about it conceptually. I assume that it'll be, you know, a key part of this in the future. I guess I'm just giving you the perspective, I think, given how hyped they are at the moment, I'm just giving you the perspective we're hearing from our customers who are trying to build with them. And the most common perspective we hear is we've tried really hard. It works 80% of the time, which is not good enough for our production use case.

• 59:47Nathan Labenz?

  Yeah, that's definitely consistent, I'd say, with what I've seen as well. We've had two CEOs of agent platform product companies on the show. And notably, neither one is quite scaling just yet. One is not even really quite released and the other has kind of a demo, but isn't, you know, is definitely not like hitting that inflection point in usage just yet, I don't think. Yeah, it does feel like it's going to be another wave that's kind of on us before we know it. Do you think we'll get out of, like, if you had to guess, end of 2023, are these problems largely solved and we have kind of agents like doing our online bidding?

• 1:00:30Speaker 00

  I'm very optimistic about the rate of progress. And part of that comes from kind of having course corrected a bit. So, you know, I've been in the field of machine learning and I did a PhD in it for now, I guess, like six or seven years. And I've consistently been too pessimistic in how quickly I think things would get done. So I kept making predictions. I thought, oh, that will take this many years. And again and again, I've kind of been beaten down to the point that I've kind of learned that this progress seems to be happening a lot faster than most might expect. So that's not a super reasoned answer, I suppose, but my intuition is that, yes, these things will get solved very quickly.

• 1:01:10Nathan Labenz?

  Yeah, I've honestly kind of had the other experience where I've been an entrepreneur for pretty much my entire career and I've always underestimated how long different changes will take. And I may be, I may soon be proven to have done that again in terms of like how quickly AI implementation can happen. But AI, just raw capability progress is the one thing that has found the other side of that for me. Like, I've always been like, oh, by, you know, this time, like, there's no way you won't have like this awesome experience. And then people just kind of, you know, live with the status quo a lot, lot longer than I anticipate. But the This has been the one exception.

• 1:01:50Speaker 00

  There's the point where we get the capabilities and then the point when that gets into products. And the thing that, you know, part of what made us go and build human loop is that gap felt too big to us. That actually, we could see that the model capabilities were far exceeding what was actually being productized or built for, for, you know, a long period. And I think that turns out to be because productizing these things is harder than just having a model API. But yeah, so I think there will be a gap between like when these things get solved in research and like when people figure out, how do I make this useful and actually productize it?

• 1:02:22Nathan Labenz?

  This kind of anticipates one of our typical closing questions, but I've asked a ton of, you know, extremely smart and very like plugged in people now, what applications they use that they would recommend to the audience? And you can answer that one, you probably have a good answer, but honestly, surprisingly, most people haven't named that many things. They've largely been like, well, I use chat GPT. And then we certainly hear like a lot of co-pilot, you know, and then we hear, that's about it, you know, there's like a little bit more, certainly some people are into the art. Occasionally somebody would be like, there's a spreadsheet plugin that's like, you know, killer for me or whatever. The names of applications that people name, like it's honestly very few.

• 1:03:06Nathan Labenz?

  Are these AI first products like just too far behind the incumbents to to overcome them before the incumbents can like layer on the AI? Is it just that like the amazing UX and UIs, you know, of our future like haven't really been discovered or invented yet? How do you see that playing out over a couple of year timeframe?

• 1:03:28Speaker 00

  So I just think it's very early, like I think you're right that most people are not using that many AI products. You know, if I think about my day to day usage, I use like various forms of coding assistant quite frequently. I use the human playground a lot. For me, it's mostly replaced chat GPT usage, but it's like probably similar to, you know, what people are using chat GPT for. I have like various forms of like summarization assistant attached to all of my sales calls that are like creating like transcript notes for me that I use pretty frequently, like I'm probably using. That like four times a day or something, right? So in terms of volume of usage, I'm using like new search engines. So I use find, I use when when perplexity had bird Q L.

• 1:04:08Speaker 00

  I was using that all the time there Twitter search because it was insanely better than a terrible Twitter search experience. Unfortunately, it's not available anymore. So I'm increasingly seeing them, but I also just think that like the current wave of applications only really started getting built. Like four or five months ago. So, you know, to be looking around and saying like, where are all of them seems a bit premature to me like we haven't. We haven't had this for that long. But I guess there's like a second question, which is like, is this more a sustaining innovation that will help larger companies? Or is this more disruptive where like startups end up winning? And I thought about this a fair bit, and I think the answer is almost certainly both.

• 1:04:47Speaker 00

  I think in established categories where you have, you know, a big player that's winning, like integrating these features very quickly. I think happens very naturally. If you're one of the large kind of like contract review providers or your HubSpot or Salesforce and you have a CRM or something like this. Then adding, you know, automated extraction of notes and adding summarization and all these kinds of features. I don't think is that difficult and obviously enhances the product and they have the distribution channels. And they're going to get more user feedback. You know, they probably win there. But I also think that there will be entirely new categories created or new styles of product that weren't possible before.

• 1:05:28Speaker 00

  You know, the people who ask like, what does the I D look like if we build it from the ground up today? Or someone who's building an educational assistant, you know, the next, what is the next version of like a language learning app look like? I'm sure all of the, the big successes right now, you know, are rethinking their products. And maybe it'll work as an add on, but maybe something new built from the ground up will replace that. And then there's just the like UX experiences that people haven't thought of yet because this stuff is so new. You know, I think we're still in that stage of kind of like plays on the television, right? Like we've taken stuff from the old paradigm and slotted it in. I don't think we've fully thought about like, what does this let us do that we really just couldn't do before? Because it's so early.

• 1:06:12Nathan Labenz?

  Yeah, that's a good analogy. I'm a collector of A I analogies that I think I'm not a connoisseur of too many things in life. But one that I might claim connoisseurship of is A I analogies. And I like that one. I haven't heard it before. But I like, what I like about it is, you know, the big difference between the play on the TV and the modern TV show is just a ton of cuts, you know, all these different things kind of integrated together with a whole tool chain into a final product. And that is really where, you know, we're still very much figuring it out. So I think that's a very good image. I'm always very suspicious of like, what is sneaking into this analogy that's actually going to mislead me?

• 1:06:57Nathan Labenz?

  So I'll have to report back if I come up with any, any concerns, but I do like that image a lot.

• 1:07:03Speaker 00

  What I, what I would say is that like, I think the incumbents that don't adopt this technology will fall behind. Right? So I do think for a lot of companies, it is a, it is a sort of adopt or die type moment. If you are a legal tech company and you don't start adding these things, if you are a CRM company, you don't add these things. Someone will, and I will, will give a much better product experience as a result. I just don't think it's that hard for them to add them. And the large enterprises seem to have been remarkably fast to adopt relative to previous technology waves. Right? I mean, this whole thing has been led in some ways by Microsoft. Like who has been faster to add L M features than Microsoft? Nobody.

• 1:07:40Nathan Labenz?

  Replit, they would point out is right up there as well.

• 1:07:44Speaker 00

  Replit is definitely up there. Replit is definitely up there, but you know, smaller product suite to be adding it to. And I just guess like you expect Replit to do it, right? Like Replit adding these things is, is great, but I expect fast moving startups to do it.

• 1:07:58Nathan Labenz?

  Yeah. Your point is, your point is very well taken. I've, I've replit on the brain today because I've got a big announcement and I'm excited to see

• 1:08:05Speaker 00

  Oh, fantastic.

• 1:08:06Nathan Labenz?

  What they're adding to an already exciting product, but you're totally right. That Microsoft could lead in this space is, you know, would previously have been unthinkable.

• 1:08:16Speaker 00

  And it's certainly like something we're starting to see at Human Loop as well, increasingly larger companies approaching us because they want to implement L and features. They've started to try in-house. They're seeing the same pain points that others, you know, had. How do I prototype? How do I handle prompt management? How do I evaluate? How do I make this better over time? How do I fine tune? And they're starting to look for solutions, but that's fairly recent. I would say like more over the last couple of months than previously.

• 1:08:42Nathan Labenz?

  Do you see consultants? What do you think is the role of, you know, OpenAI has this partnership with Bain. Do you think that's going to work? Like there, there are all these big companies that presumably can do like process automation at a minimum. And maybe don't have that like, you know, plugged in CTO that a HubSpot and, you know, some other, you know, more tech-forward companies have the benefit of employing. So are like Bane consultants coming to you and bringing like Fortune, you know, 1,000 projects to Human Loop?

• 1:09:15Speaker 00

  I can neither confirm nor deny. I guess like OpenAI doesn't want to do the like sales part of this, right? I think they're sincere in their objective being to build AGI. And so I think they are, you know, encouraging an ecosystem to develop on the tooling side. They're working with partners on the sales side. I think so that they can maintain their focus on what they see as the thing that they do best, which is pushing the boundaries of AI. So I think insofar as they're working with people like Bane, you know, Bane is going to be better enterprise sales than OpenAI is, almost certainly. And I think they're sort of choosing to partner rather than try to do everything, which to me seems like a very smart strategy.

• 1:09:55Nathan Labenz?

  Yeah, especially when you have the thing the whole world is clamoring for, a a channel partner, yeah, can really add a ton of value.

• 1:10:06Speaker 00

  You know, what we can offer from in terms of perspective at Human Loop is I suspect we have seen more people try, succeed, and fail at getting applications into production than I suspect almost anyone other than maybe OpenAI. And that gives us some insight, hopefully, into like what applications are actually being built. And we discussed that a little bit. And also the challenges people have around reliability, factualness, evaluation, experimentation, that I think almost everyone will face when they do come to actually try to productize this. And that's what Human Loop is there for. It's able, it's, you know, we're building the tools that bridge that gap between I have access to the GP4 API, I have access to an LLM,

• 1:10:51Speaker 00

  and I actually need to make this robust and understand how it works in production. And yeah, I think it gives us a unique perspective. And some of it really is just having the right workflow to iterate very quickly. In fact, I actually, probably that's the thing that I would say is the most important, because a lot of this is Requires trial and error. It's difficult to kind of get it right first time. Having a really fast iteration cycle to be able to try something, understand how well that worked and go again, whether that's fine tuning, whether that's chaining the prompts, whether that's rethinking the UX a little bit. I think that really makes a big difference about whether or not you get to success. And I guess the people who don't get there tend not to set up robust evaluation systems. So they don't know whether as they're making changes,

• 1:11:36Speaker 00

  things are actually getting better or worse. And if you're kind of changing a lot of stuff flying in the dark, you're not actually going to make progress. And so, yeah, I think like really robust evaluation and fast iteration cycles have been the things that, if I was to have to like extract principles that I think make this stuff work well.

• 1:11:53Nathan Labenz?

  Love to kind of hear just how you think about guiding people up the sort of curve of scale and like sophistication of evaluation. Cause those things I think are very much like, you know, ideally should be working in tandem for people, but maybe not always are.

• 1:12:08Speaker 00

  Yeah. So I think like the typical journey we see is like, people start in something like the playground with a handful of test cases. And the question they're really just trying to answer for themselves is like, is this even feasible? Like, can I get the model to do this task? And they don't really know yet. And so they spend some time iterating with that. Usually they're convincing themselves yes or no on that question. Like, can I, you know, is this something that's within the capabilities of the model? Maybe experimenting with a bit of prompt engineering. Typically at that point, they'll go and actually wire this up into some kind of product. They might have like some kind of internal evaluations. We typically see people also will hook this up to something like a Streamlit app or just a very simple UI to collect a little bit more volume of internal evaluation.

• 1:12:53Speaker 00

  And there they're looking at maybe like hundreds to thousands of examples, just trying to get some quantitative metrics on performance. Some of the startups skip that step entirely. So this is like, you know, that step is more common amongst the larger companies. The startups typically, once they've kind of got satisfaction that it works like eighty, ninety percent of the time, will typically just deploy it. And then we'll use the in-production data as a way to understand how well things are working and then iterate very fast. And by the time you have like a few hundred interactions per, you know, per day, uh, which is not that many, because people are often touching the models in multiple places, you start to have enough signal to start actually driving, driving decisions.

• 1:13:34Speaker 00

  So you don't need masses and masses of data, but, but you do need some usage. You do need some people coming through the app and, and most have a lot more than that pretty quickly.

• 1:13:45Nathan Labenz?

  I already did the kind of version of products you'd recommend. If you want to throw any else out there.

• 1:13:51Speaker 00

  Mem's a good example here, right? So I think like rethinking what a note taking app looks like if you try to put L L M's everywhere, self-organizing system for notes where you just dump everything in and it tries to figure out where things belong. I don't think people like think of that as an L L M product, but it's obviously like very A I driven, uh, in the background. I think that's a cool one. I'm a big fan of the new L L M search engines. I think Find is really cool. I use Cursor all the time, uh, which is like that I D product I was telling you about. So I'm a, I'm a big fan of that. They've got an, you know, an integration to V S code. The, the ones we all already know about, like GitHub Copilot, um, everyone in our team uses that pretty frequently. As I say, I do end up using the human playground a lot.

• 1:14:35Speaker 00

  I now prefer it to the OpenAI versions or the, or Chat G P T. And so that's where I live. And also I think it's important for us to use the products ourselves to improve them. So that's another one I'm using pretty frequently. I have a note taker that like goes to all my Zoom meetings. I get transcripts and summaries from that. So I use Fathom for that, but I think like, you know, there are many others, Firefly and others. I don't know if they also have those A I summaries built in. What else do we use pretty frequently? I mean, we also use just the raw like GPT 3.5 and Claude models in our Slack quite a lot. So we have a Slack channel and also individual access to Slack with those models built in.

• 1:15:15Speaker 00

  And just having them there, like where you're working most of the time, means we end up using them more than I think if we had to go to the playground or go to ChatGPT.

• 1:15:24Nathan Labenz?

  Okay, quick hypothetical situation. Let's imagine that in a not too distant future, one million people already have the Neuralink implant. If you get it, just like everybody else, you get now the advantage of thought to device direct communication. So you can like record your thoughts as text or use a, you know, a UI. Would that be enough for you to consider getting an implant yourself?

• 1:15:53Speaker 00

  Has it been safe for those first million people?

• 1:15:55Nathan Labenz?

  Yeah, let's say, you know, we're in like a COVID vaccine safety where you've certainly got, you know, some noise around it. But the general data suggests that it's overwhelmingly safe.

• 1:16:07Speaker 00

  It's a difficult one to answer because I feel like whether or not I say yes depends so much on the political context around the technology. And I feel similarly with AI, right? We're building this very powerful, potentially hugely positively transformational technology. But in order for it to be positive, we need to make sure that everyone gets a say in how it's used and it's democratically controlled and et cetera, et cetera. And I think I would feel similarly about something like Neuralink. Like if the right safeguards were in place, then I would seriously consider it. I already have a prosthesis. It's my phone and I'm glued to it all the time. So it's only, it's only one step further.

• 1:16:46Speaker 00

  But I'd want to be confident that I'm not, you know, opening up literally my brain to some like advertiser model driven, you know, concentrated, you know, like, do I want Facebook or Google or whatever a large tech company to like have direct access to my brain? Probably not, but if all the right safeguards were in place and we had the right political structures in place, I'd be very excited about the technology.

• 1:17:10Nathan Labenz?

  Brilliant answer. I think the distinction there between medical safety and, you know, social, you know, contextual safety is a very sharp one. That's also a perfect transition to a typical final question, which is just zooming out, you know, as far as you can. We've spent all this time really on kind of the current margin of implementation and, you know, the struggles and the triumphs and the tools around that. But zooming way out, what are your biggest hopes for and fears for what AI is going to mean to society over the next handful of years?

• 1:17:45Speaker 00

  So on the hope side, like, I really hope that it augments people to be able to achieve stuff that they couldn't do before. So I really hope that we accelerate the progress of science, right? The, you know, the volume of literature being produced is way beyond what anybody can, can read in full. There's probably lots of connections to be made. If we can make progress faster in medical research, in, you know, other parts of scientific research, we've got diffusion more quickly, whatever it might be. And that would be a hugely positive and exciting win. And we've seen early signs of this from, you know, DeepMind AlphaFold and things like that, where it seems plausible that we can accelerate scientific progress with AI. I'd also be really excited about increasing access to education.

• 1:18:26Speaker 00

  So, you know, whether this be even just in the rich world, giving more people access to more personalized educational experiences, which is, I suspect, what will happen first. But also if we can reduce the cost of education far enough and figure out ways to give more people access to it. That's a really exciting vision to me, right? Like, personal tutoring is one of the educational interventions that seems to have the largest evidence behind it, but it's very expensive to scale with humans. Like, what does an AI version of that look like? And that could be very exciting. I think freeing people from work they don't want to do is also something I'd be very excited about. And I think a lot of us spend time doing, like, just drudge work as part of our day to day. I'm less excited about, you know...

• 1:19:09Speaker 00

  The potential for misuse. I think it could concentrate political power, right? I do think there are like challenges. You mentioned earlier using AIs for like screening CVs. And, you know, it always raises a red flag in my mind when you give autonomous systems these decision making capabilities that have big effects on people's lives. Like, can we make sure that we don't bake in political or social biases? And then like scale them and systematize them worse than they even are now. So I, I think there are risks, but I also think there's potential huge upsides. I'm very excited about a world in which just software is generally smarter and works a bit better and is easier to produce. Right, like one thing I'm just excited about is like lowering how hard it is to build software products.

• 1:19:52Speaker 00

  I think there's lots of places where we could have good software, but the markets are small or it's, you know, it's not interesting for a company to be built around it, but it could improve some people's lives. And the cost of producing software is going down dramatically. So, yeah, many, many like big benefits, education, healthcare, science improvement, definitely risks as well, even before we get to anything that resembles AGI. And so excited, but cautious.

• 1:20:20Nathan Labenz?

  Razabib, thank you for being part of the cognitive revolution.

• 1:20:24Speaker 00

  Thank you for having me.