E19: The AI Agent Revolution with Matt Welsh of Fixie.ai

Nathan Labenz sits down with Matt Welsh, the CEO and co-founder of Fixie. AI which just raised $17M in seed funding. The technology world's attention has turned en masse to AI agents over the last month, but Matt, a former Harvard professor and ex-Google, has been focused on AI agents for longer over at Fixie. The platform is currently in developer preview mode, where developers can build agents and end users can try them. Agents hold the promise of handling not only the core cognitive work, but many of the messy implementation details as well – the searching & finding, sorting through & scanning, signing in & navigating, reading documentation & using APIs – all the manual ad hoc routing of information for which we act today as human plumbing. -- We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics TIMESTAMPS: (00:00) Preview of the episode (02:08) Why AI Agents are so topical right now, and Matt's prescience with Fixie (06:40) How Matt started Fixie (10:55) How AI Agents "figure out the plumbing" (15:57) How Matt envisions work changing in the near term + concrete examples (22:40) Building your own agent (26:00) New threat model and guardrails of this new paradigm (34:48) Synthetic memories in autonomous agents (42:39) Examples of incredible use cases (48:00) Single sign on idea (51:00) Form factor for AI-assisted living (54:32) Matt's hopes and fears for AI TWITTER: @CogRev_Podcast @mdwelsh (Matt) @labenz (Nathan) @eriktorenberg (Erik) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries. From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. https://www.omneky.com/ More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com Also, check out the debut of co-host Erik's new long-form interview podcast Upstream, whose guests include David Sacks. Ezra Klein, Balaji Srinivasan, and Marc Andreessen. Subscribe: Apple Podcasts: bit.ly/3z8TldL Spotify: bit.ly/40letd0 Music Credit: OpenAI's Jukebox

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  And if you think of like a 2D plane, one axis is what you just described. It's that idea of is it being used in line with a human versus being basically fire and forget. You give it a task, you give it a job to do and it goes off autonomously and executes. I think the other axis here has to do with interpreting and synthesizing natural language on one side and automating process on the other. Let's address the elephant in the room, which is the, you know, there's a very real concern amongst many people that AI based automation ends up putting people out of work. I think that this is something that we're going to have to wrestle with as a field, that things that we've been doing by hand are

• 0:43Speaker 00

  going to become obsolete. You know, you don't have to look very far to see the the black and white photograph of a room full of accountants hunched over their ledgers and their, you know, mechanical calculators doing things that a single spreadsheet could do today.

• 0:59Nathan Labenz?

  Hello and welcome to the cognitive revolution where we interview visionary researchers, entrepreneurs and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas and together we'll build a picture of how AI technology will transform work, life and society in the coming years. I'm Nathan Labenz, joined by my co-host Erik Torenberg.

• 1:22Speaker 02

  Before we dive into the cognitive revolution, I want to tell you about my new interview show, Upstream. Upstream is where I go deeper with some of the world's most interesting thinkers to map the constellation of ideas that matter. On the first season of Upstream, you'll hear from Mark Andreessen, David Sacks, Balaji, Ezra Klein, Joe Lonsdale and more. Make sure to subscribe and check out the first episode with A16Z's Mark Andreessen. The link is in the description.

• 1:50Nathan Labenz?

  Hi, everyone. Our guest today is Matt Welch. He is the founder and CEO of Fixie.ai, the automation platform for large language models that invites you to build natural language agents that connect to your data, talk to APIs and solve complex problems.

• 2:08Nathan Labenz?

  While the technology world's attention has turned en masse to AI agents over just the last month, Matt has been thinking about AI for much longer and building AI agents specifically for just over a year. Citing as inspiration Google's early demonstration of LLM tool usage in its January 2022 Lambda paper, Matt, who is formerly a Harvard computer science professor and a 10-year principal engineer at Google, has assembled an all-star team with the goal of driving process automation for enterprise customers with AI agents. You can see how they're going about this online at fixi.ai. The platform is currently in developer preview mode, which means that developers can build

• 2:51Nathan Labenz?

  whatever agents they want, and end-users can try any and all of the agents built by the community. The chat-style interface will by now be quite familiar, and you can literally just tag agents like you'd tag a teammate in Slack. Each agent is designed to do a different job for you. While these agents in general are not super sophisticated just yet, and there are all sorts of authentication, access, and security issues still to be worked out, if I had to bet right now, I'd say that agents will become one of the main ways that AI impacts and ultimately shapes the world over the next few years. It's already clear that AI can do valuable cognitive work, but as long as human users

• 3:35Nathan Labenz?

  have to figure out how to route information to and from AI, process automation in many contexts will be bottlenecked by implementation. Agents hold the promise of handling not only the core cognitive work, but many of the messy implementation details as well: the searching and finding, the sorting through and scanning, the signing in and navigating of websites, the reading of documentation and using of APIs. In short, all the ad hoc routing of information for which we today act as human plumbing.

• 4:11Nathan Labenz?

  I see no reason that this won't ultimately work. Both reinforcement learning and all sorts of supporting tools and frameworks seem very well-suited to overcoming the challenges at hand. But while I'm quite sure that the agents are coming, likely with very visible progress by the end of this year, I really don't know how this plays out. There are just so many ways that the world will inevitably change in response to the introduction of agents, and agents will interact not only with humans and existing software systems, but also with each other. The dynamics here seem extremely hard to predict. So with all that in mind, I was really fascinated by this conversation.

• 4:54Nathan Labenz?

  Matt had extremely insightful thoughts on a 3D conceptual model for understanding different modes of human-AI interaction, as well as the challenges of security and memory for AI agents. What sort of systems and infrastructure agents will really need to go mainstream? Whether embeddings might create a privacy-preserving communication medium for agents? Why Matt believes in Apple's vision for inference on the edge? What sort of ambient interface we're likely to use to interact with AI agents? And much more. Matt, as you might imagine, is extremely busy. He had a client call that limited our time together to just an hour. But if you love this episode as much as I did, we'd love to see your review on Apple

• 5:38Nathan Labenz?

  Podcasts, and maybe we'll be able to convince Matt to come back in six months or so to check in on AI agents' progress. Now, I hope you enjoy this conversation with Matt Welsh of Fixi.ai. Matt Welsh, welcome to the cognitive revolution. Thanks for having me. Really excited to have you here. The energy is at a peak right now, a fever pitch around this concept of AI agents, and you as the founder and CEO of Fixi AI are right in the center of that. Your company just announced a $17 million seed round, which has a little bit of a, like, I don't know, 2021 sort of fundraising vibe to it.

• 6:23Nathan Labenz?

  And then the world is kind of hitting this moment of awareness, curiosity, fascination, with obviously a lot of debate around AI agents as well. So let's start off by just getting your perspective, like, what has it been like to be you over the last couple of months?

• 6:40Speaker 00

  Yeah, it's been bananas. The world is moving so fast here. And the way I kind of think about it is, think of a large language models like ChatGPT as this alien technology that just landed in our backyards. And everyone is now picking it apart and trying to figure out what they can do with it. And they're finding that it's really remarkable in terms of what's possible. And so ChatGPT, you know, you mentioned the funding. The way I think about it is ChatGPT kind of chummed the water, as it will, as it were,

• 7:19Speaker 00

  to, you know, get so much interest from investors looking for opportunities in this space. And we just happened to be in the right place at the right time, I think, because we had been working in this space for a while before ChatGPT came out. And then all of a sudden, it just, you know, burst onto the scene and everybody's talking about it.

• 7:39Nathan Labenz?

  What were the models like when you started? I'm always kind of fascinated by this foresight that certain people have seemed to demonstrate. I think it's, you know, really important and only getting the, the alpha on foresight is only getting higher and higher as the future comes at us faster and faster. So, you know, going back even just a year, we're kind of in like early InstructGPT. Was it like InstructGPT that you were seeing and saying like, I can build a platform on this?

• 8:11Speaker 00

  Yeah, not even. I think, you know, we were in the kind of early days of GPT-3. And at that time, GPT-3 was not really fully a thing that people were building companies around. There were a few, Jasper.ai being one of them. But it wasn't yet being used as a mechanism to drive automations or, you know, access to custom data, access to external tools. That, that idea was kind of starting and open source projects like Langchain started building out some cool demos around that. We were mainly inspired by the original Lambda paper from Google that showed that if

• 8:54Speaker 00

  you gave a language model some, the right kind of prompts and taught it how to call out to an external tool to fetch external data that you could use that to drive interesting processes. And there just weren't very many papers about that topic at the time. So that one was relatively new and we started building around that using GPT-3. I think really what happened here was, you know, ChatGPT, yes, there were certainly model improvements between GPT-3 and GPT-3.5, but it was popularizing that idea that changed everything, right? It went from that kind of clunky playground environment in, you know, behind OpenAI's,

• 9:40Speaker 00

  you know, developer website to a more consumer-focused user experience. And I think that speaks volumes of the, the power of just having that, you know, end-user experience dialed in just right, that you could have done most of the things with ChatGPT with GPT-3's playground, but just nowhere near as elegantly and, and usefully.

• 10:07Nathan Labenz?

  I'm really trying to figure out where I, where I understand agents to be in this kind of, you know, fast-evolving mix of, of usage patterns. So let me try on this theory for you. I don't want to hear, you know, how you would correct me. I've been kind of talking about AI usage as like co-pilot mode on the one hand, and then what I have been calling delegation mode on the other hand, where co-pilot mode is like you are doing stuff, you are driving and you are, you know, pursuing tasks, you know, pursuing goals. And then you've got this kind of helper that's there on the side. You can kind of like ask to like, you know, draft something for you or answer a question or do a little analysis, but it's

• 10:52Nathan Labenz?

  kind of, you know, at your like immediate beck and call. Whereas I had been thinking of this delegation as like, More, I think maybe I should have a more process automation kind of framing for it because it's kind of has been like, there's this AI that can do these cognitive tasks that previously required humans to do. And now if we can just build the piping, then we can like start to automate and scale some of those tasks in ways that were previously impossible. And you can imagine a million of those. And now agents have come onto the scene and I kind of see it almost as like a bridge where it's like, you can sort of

• 11:31Nathan Labenz?

  stay in like your own human pilot mode, but you have like access to this delegation mode where you can say like, I want to kind of not just ask you questions, but carve off distinct things and send them to you. And you need to be able to figure out the plumbing, right? I guess that's kind of the key flip is like, I'm not going to sit there and architect the system. I want you to figure out what buttons to push.

• 11:54Speaker 00

  I think that's right. So yeah, my view is that there's, there's two dimensions. And, and if you think of like a 2D plane, one axis is what you just described. It's that idea of is it being used in line with a human versus being basically fire and forget. You give it a task, you give a job to do and it goes off autonomously and executes. I think the other axis here has to do with interpreting and synthesizing natural language on one side and automating process on the other. And so, you know, the early generation

• 12:35Speaker 00

  of AI applications were largely about taking in language, manipulating it, spitting out language. So Jasper being a good example where you gave it a description of your product or the blog post or whatever marketing copy you wanted, Jasper took that in and then synthesized something for you. On the automation end of the spectrum, you start getting into things where you're using the language model kind of like a computer that you program in English. And that that computer is capable of doing extremely rich and sophisticated kinds of symbolic manipulation, not just on natural language provided by humans, but also on

• 13:15Speaker 00

  things like, you know, like a JSON object, or contents of a spreadsheet, or an XML document, or something else. And so if you think about that axis, I think you could pick almost any point in that space and identify potential use cases and kind of existing systems that are built up around that. So Jasper being very hard on the natural language processing side, and more in the kind of copilot quadrant that you're describing. With Fixi, where we're trying to go is probably down in the opposite end of that spectrum, which is more about autonomy and driving process automatically, and also doing so

• 13:59Speaker 00

  in a way that doesn't need to be a live interaction with a human. It can be, it doesn't have to be. So that's generally the way I've been thinking about it, but I think that's a good framing.

• 14:09Nathan Labenz?

  Yeah, I love it. Added dimension is always a great way to level up. So that's maybe one of the best questions and answers we've had on the show, honestly. I think that's really, makes a ton of sense to me, and I'm going to continue to test some ideas against that 2D framework. You know, let's get into maybe a few of those points. I am advising a company right now called Athena, which is in the executive assistant services business. Their model is that they hire individuals in the Philippines. They have, I believe, a thousand active EAs, working one-on-one with executive clients, mostly in the US, but also elsewhere.

• 14:54Nathan Labenz?

  Clients very widely, you know, it's, they have like, a lot of training and culture and kind of stuff built up, but then, you know, each client is its own like, unique adventure. So it's extremely kind of diverse, obviously a lot of commonalities, but a ton of long tail. So our goal is to become, and this is kind of my mission statement, right? Help us become the EA company that is the best in the world at adopting AI tools. Please do my job for me and pick out some of these points in space perhaps, and help us kind of like get more tangible about how you envision work changing to delegate more of these things to the process agents that you're creating.

• 15:39Speaker 00

  Yeah, I think there's a bunch of places in the space where this makes a lot of sense. And I mean, first of all, let's address the elephant in the room, which is the, you know, there's a very real concern amongst many people that AI-based automation ends up putting people out of work. I think that this is something that we're going to have to wrestle with as a field, as an industry.

• 16:00Speaker 00

  In most cases, I don't think it's true. In some cases, it may well be true that things that we've been doing by hand are going to become obsolete. You know, you don't have to look very far to see the black and white photograph of a room full of accountants hunched over their ledgers and their mechanical calculators doing things that a single spreadsheet could do today. Right? But anyway, coming back to the point about automating things that EA could do, I have an EA through a company called Double, and I can absolutely see lots of opportunities for automating things. I think one side of it is, of course, generating appropriate written content.

• 16:45Speaker 00

  Like, that might be an email reply, or it might be a request to schedule something, or it might be, you know, a newsletter that gets sent out to a group of individuals. This is not always something that people have great training in being excellent written communicators. And especially if you get outside of, you know, countries where people are native English speakers, they're not necessarily going to have the same level of skill there. And this is a place where I think AI can help tremendously, right? On the other side, more on the process automation.

• 17:21Speaker 00

  There's so many things that I know I do very mechanically and repeatedly that I believe can and should be automated. And the only thing that we're kind of lacking today. I mean, AI models could automate it, but the AI models are lacking the appendages into your systems to make this easy to do. So to give a very concrete example, you know, as a CEO, probably 70% of the email that I get is people trying to sell me things that I don't need right now. And that might be, you know, consulting services or, uh, you know, recruiting services or, you know, some kind of, uh, new bank account or whatever it is.

• 18:02Speaker 00

  A lot of my time has been spent drafting or sending out, you know, polite replies to these people. And you could ignore the email completely. The problem is then you get five, 10, 20 emails from them and their automation flow. And they keep bugging you and following up. And so it's easier, I've found, just to spend a minute or so to respond to them. But the problem is that you can't tell whether or not an email is in that category until I kind of sit down and read it. And it feels to me like that's another place where just categorizing email, inbound email or other types of, uh, you know, communication should be an automatable process. Now today, my EA does that, right?

• 18:46Speaker 00

  He reads my email and puts it into different folders and often replies to those things. But again, this is a very laborious process and it feels like something that we should be able to use AI to automate.

• 19:01Nathan Labenz?

  So that, that is a very top of mind example for us as well.

• 19:07Nathan Labenz?

  We, at Athena, they call this pre-drafting and they've, you know, created significant like training, all this kind of stuff around it. And I think some, you know, the clients have to be pretty good at like, kind of systematizing for it to really lock in. And I, you know, personally, I actually started as a client. And so, you know, there's, that's the way I got introduced to the company. The email drafting never quite got there for me. Like there were definitely things that I was able to delegate that were like. You know, routine calendaring and that kind of thing. But then there are some things where it's like, yeah, I just, nothing really wrong with what you're saying, but it's not quite how I would want to respond. And a bit of a hard time getting over that, that gap.

• 19:52Nathan Labenz?

  And then I think, you know, AI is also going to have a bit of a hard time getting over that gap, but it's maybe a different sort of hard time. So you kind of mentioned the appendages and this, this gets to like, you know, the deep dive that I kind of want to go on with you around the different appendages that the systems, you know, are, are in need of and, and kind of what they're going to look like. You know, I'm thinking access, you know, is obviously just one real important one. It doesn't seem to me that we're going to get to the point where like few shot, you know, here's two emails that this person wrote in the past, you know, now you can kind of cover everything. You're going to need to, you know, be able to go a little deeper that than that. Access, you could also kind of tie to like memory, although that's kind of distinct too,

• 20:36Nathan Labenz?

  because your, you know, memory may include like your interactions with the system as well as kind of, you know, the email database, you know, that existed before the system. But those both seem, you know, pretty hard and kind of unsolved problems right now. So maybe let's just start with those, like, how do you guys think about security? To me, it seems just like an absolute hornet's nest of pain, but it is one of your core selling propositions. So let's start there, you know, how, I, I went on the, on the fixie platform, by the way, and already, you know, interesting bit of user feedback. I was comfortable giving the at fixie slash G calendar access to my calendar and playing with that. But then I saw the at busy executive hackathon user who had the Gmail assistant.

• 21:20Speaker 00

  Right.

• 21:21Nathan Labenz?

  And I was not going to authorize that one. There's also an interesting intersection with like your marketplace concept, but I'm talking too much. Tell me, you know, security, how are you guys thinking about that?

• 21:30Speaker 00

  Yeah, so what you will see in Fixie today is we've opened this up as a developer preview. And we've kind of intentionally said, hey, this is a free for all. Let's just let people build things, bring them in, show them off, integrate them with other people's agents and tools and workflows and see what happens. And we know that this is not really the right model. This is not the model that makes the most sense for our enterprise customers, but it does reveal a lot of interesting potential use cases. And so in a lot of ways, the throwing caution to the wind and letting people build has been the first priority. Going into an enterprise situation, though, you're certainly not going to want to just

• 22:14Speaker 00

  pick up a natural language agent that some rando on the internet, you know, put into your, put into the system, right? You're going to have to make some decision about whether this is a good thing for me to integrate, whether I trust it and so forth. And that, by the way, we're used to making those decisions. We make those decisions all day long. We're using, you know, Riverside to record this podcast. You know, I use Zoom, I use Google Meet, I use Google Calendar. There's all kinds of trust relationships there that one has to have with their data when they're doing those integrations. You know, I've plugged countless things into Slack that understand, you know, my Google Drive contents and my calendar and so forth.

• 22:54Speaker 00

  So I feel reasonably confident that there's a pattern for people to develop an intuition around where they want to place their trust. Most enterprise customers, I think, are not going to be wanting to draw too much on these third-party integrations. They're largely going to be looking at building their own integrations where they say, okay, we, this is a natural language interface to our data, our tools, our APIs. We built it in-house. We know where the data is flowing. We know what's happening with it and we can build it into our own workflows. And so that's where we've been focusing most of the work is on that developer experience. Coming back to the security question, though.

• 23:35Speaker 00

  It's very clear that people don't yet have a good sense of how to think about where is the data flowing when I'm using an AI-powered application, right? So let's imagine I build an agent that slurps up my internal company Notion website that has all of the information about everything that's going on in the company, you know, proprietary stuff and confidential stuff and so forth. Now I can take that agent and ask questions of it and I can say, you know, what is the latest on this project? And it can go and pull out the relevant context from the Notion pages and answer that question for me. Well, what's going on here, right?

• 24:19Speaker 00

  Well, first of all, the query from the user has to route out to a language model. Where is that language model? Which language model? Who's got access to it, right? Secondly, the, you know, the, the, the query goes into, you know, some kind of vector database or something that's pulling out the relevant snippets. Again, who runs that? Where is that live? Where does the data live? You know, how can I delete what's in there, right? Third, what happens to the history of the query conversation? Where does that live? The point being that there's a lot of pieces involved and many of those pieces today are third-party things like we primarily use the OpenAI language models, although you can use other things like Anthropic's model and, and so forth.

• 25:03Speaker 00

  And, you know, if you're using an external vector database like Pinecone, then, you know, that's another part of the, you know, the, the software supply chain. So we're going to have to solve this problem in a really,

• 25:18Speaker 00

  you know, explicit way for enterprises that care so much about where the data is flowing and also have, of course, regulatory compliance requirements around that. The good news is though, I think that there's a good pattern for doing that. Plenty of companies have built SaaS products designed for enterprises that address that kind of thing. And we just need to figure out how to do that as well. The biggest gap today for us is the existence of language models that are really good that we can host in our infrastructure and kind of put behind that firewall, if you will. Calling out to OpenAI is not ideal for a lot of enterprise use cases, although there is the OpenAI service from Azure that addresses many of them.

• 26:02Speaker 00

  So, you know, my expectation, though, is open source models are coming fast and furious. We've seen even just this week, I think, two more exciting open source projects. There's the red pajama thing and then Stability AI just announced one today. So my expectation is that we're going to see an onslaught of these that give us that that freedom to go and deploy our own models and not have to rely on third-party providers.

• 26:30Nathan Labenz?

  I want to think a little bit about like new threat models as well because there's the kind of data flowing from, you know, your server to OpenAI and like their Terms of Service and, you know, whether you trust them to not leak the data somehow and, you know, not get hacked on their side. All that stuff, yeah, I think mostly fairly conventional should be, you know, I mean, we're a mess, so it's like, not like it's going that well today. I would say nobody, nobody I know is like, you know, what's in a great state is like web security, but yes, we get by. But there is this kind of new paradigm as well that's like, is the model itself hackable

• 27:14Nathan Labenz?

  or like subject to, you know, misbehavior? And then what is it going to encounter in the wild as I send it on its way? You know, we've seen these like super like foreshadowing, you know, kind of examples where somebody puts a hidden message into the HTML other website and then, you know, that actually does have an effect on search that can be like quite a left turn. And, you know, people are very savvy and very clever with this kind of thing, right? So never, you know, the internet remains undefeated on that kind of, you know, what will, what can be exploited, like, you know, somebody will find and.

• 27:49Nathan Labenz?

  Exploit. So I I'm really interested in just how you think about all that kind of stuff in general too. And then specifically like on the language model, OpenAI's versus yours. I've had this theory that like with 3.5 turbo getting extremely cheap and probably pretty good at like a lot of these kind of core tasks. And also, you know, the turbo reflects the speed that people might just kind of start to see that as a standard and be willing to pay for the overall package of like convenience, speed, quality, safety, like they've got their whole red teaming infrastructure that is going to be, you know, you can't possibly match that on like, you know, whatever stability just put out. And I think they actually also do a pretty good job. I'm in their Discord and follow that.

• 28:35Nathan Labenz?

  So I, you know, I think we're fortunate that stability has put real thought into it as well. But like hard for you to know exactly what, you know, the SLA, so to speak, you know, of safety for a stability, you know, open source releases. So how do you think about all those kind of unknown unknowns? And then like, whose problem do you want those unknown unknowns to be?

• 28:56Speaker 00

  I think that's a very good question, and it's something that it's such early days. I don't think that the industry understands this very well at all. Right? You know, it wasn't that long ago then that, you know, prompt injection attacks were all the rage and people were, you know, digging into how does chat GPT work? And what are the prompts that it, you know, that it uses? And then, you know, undermining those and getting chat GPT to adopt, you know, kind of evil personae based on, you know, what somebody wants to get the thing to behave in a crazy way. These are certainly going to be challenges, right? This is, this is not at all a solved problem.

• 29:35Speaker 00

  I have two takes on this. I think the first take is there's so much value to be brought today, even in situations where confidence in the absolute safety of the model is low. And many of those are coming out in situations where there's going to be a human in the loop anyway. And so the amount of damage, so to speak, that an AI model that's misbehaving can perform is fairly limited. So in the case of, you know, drafting an email, well, again, if it's a draft. And the person who wants to send the email is going to be reading it first. And there's only so much the the draft could have gotten wrong, right? That that there's a there's a natural um speed bump there, if you will.

• 30:19Speaker 00

  Um, so I think that there's a ton of value in in in use cases where there's already a kind of built in check on that. But going forward, as we want to embed language models deeper and deeper into our tech stack, we're certainly going to need uh to have appropriate safety mechanisms in place. And this is a really active area of research. Um, it's not obvious to me exactly how we're going to achieve that goal. And some of it certainly model fine tuning and red teaming it and things like that. But I also have a feeling that we just need to put other software mechanisms in place around and on top of these models to prevent them from taking actions that we don't expect.

• 31:03Speaker 00

  Uh, there's an exciting project that I was just um checking out a few days ago called Guardrails. And uh, this is from uh Shreya Rajpal, who has worked at um Apple. And um, this is a project that is uh effectively allows you to write a spec that um captures the semantic requirements of the response of a language model in a certain situation. So to give an example, if I'm building an agent that is meant to respond to questions about the weather, I can write a guardrail spec that says any response from this use of the model needs to involve the weather in some way. It can't just be about any arbitrary thing. And you can place other requirements on it.

• 31:47Speaker 00

  For example, the temperatures that it responds with have to be within the range of, you know, what one would expect from, you know, this area of the world. Or, you know, it can't, you know, report crazy kinds of numbers in terms of, you know, rainfall amounts that make no sense. Without those kinds of things in place now, when the model responds in a certain way, guardrails can then interpret whether or not the model seems to be operating within its spec, so to speak. And of course, there's problems with this. This is not a panacea, right? There's ways of breaking that as well. Like it have subtle ways of breaking the model so that it always reports things in Fahrenheit instead of Celsius or whatever the, you know,

• 32:31Speaker 00

  whatever the thing is. But it feels like a very good step in the right direction to building out software infrastructure to do this. The other interesting thing there is, I believe that there's a big opportunity to develop tailored models that are explicitly designed only for the sake of validating responses from other models. In other words, I don't need a full generative model that can, you know, spout a children's story or, you know, something about Shakespearean literature in order to do this. I can have a much smaller, cheaper, faster, and also heavily, heavily, heavily task tuned model

• 33:13Speaker 00

  that's job is to basically take in a set of requirements and give me a, you know, basically a score says, does this response match the set of requirements, right? This was a weather agent. Did it answer about the weather? Yes or no? And that seems like something that language models will be very good at and we should be able to do.

• 33:33Nathan Labenz?

  Yeah, okay. I think a lot, that all makes a ton of sense. I kind of see shades of like Eric Drexler's, you know, comprehensive AI services and it all, you also kind of channel a little bit there, Imad with his like million small models paradigm. And the guardrails thing sounds awesome too. I definitely will want to check that out. That to me, I was thinking that kind of sounds like a moderation endpoint. You know, I've even speculated that that could be a new technique that GPT-4 might be applying is some sort of semantically defined loss as opposed to a pure next token loss.

• 34:14Nathan Labenz?

  And kind of a, a semantically, you know, embedded style validation. If you can, oh, Constitutional AI, that was the other thing I was going to say. It's kind of like a moderation endpoint slash Constitutional AI paradigm.

• 34:30Nathan Labenz?

  So the the ability to kind of set something like that in a semantic way and then have this kind of, uh, yeah, guardrails. That sounds brilliant. Cool. Well, again, great, great, great stuff there. You know, I guess maybe next thing is kind of memory. Um, we're seeing like, again, a wide range of sort of approaches where, you know, the, what I see on Fixie today is pretty episodic. You know, maybe there's like a third dimension also of of our previously, you know, 2D space that's like, how long lived is the, you know, is the agent or interaction? Uh, because, you know, a search query can obviously be kind of often answered in just like a quick generation and you're done with the agents on Fixie. I'm having some

• 35:15Nathan Labenz?

  conversation, you know, and going back and forth. And then there's, you know, we're seeing towns of people having like whole days of lives and, you know, reflective synthetic memory. So how are you guys thinking about that? The, that spectrum from, you know, the goldfish to, you know, in theory, this kind of, you know, knows you better than anyone lifelong AI companion?

• 35:37Speaker 00

  Yeah, no, it's a very good question. So in Fixie, there's a notion of what we call a session. And a session is just think of it as the conversational record between some end user and some set of agents. And any number of agents could be involved in a session. So I might use a session to, you know, uh, fetch data from a database and then massage it in some way and then call out to another agent to make a visualization or something like that. So you can think of these sessions as kind of an append only log of the conversational history. Um, and of course, the content, uh, the content in a session itself provides context to the agents when you're using it. So for example, if I said,

• 36:22Speaker 00

  Uh, you know, make a pie chart showing me the top 10 contributors to this GitHub repo by number of commits. So we'll call out to an agent, fetch the data from GitHub, uh, call out to another agent to generate a pie chart. Then if I just say, uh, make a bar graph instead, I don't need to spell out the whole thing. I just say, make it a bar graph. Um, then because the context is there in the session object, that ends up translating into make a bar graph of the top 10 contributors to this repo and so forth. So the way we've been thinking about sessions to date has been that they're largely ephemeral.

• 37:05Speaker 00

  Uh, and they're largely meant to be used for, you know, the kind of few rounds of conversational, uh, you know, uh, uh, interaction between a human and the machine. But there's no reason they need to be limited in that way, right? And so one could imagine having a kind of perma session, if you will, that is constantly going on for you and all interactions that you have with any aspect of the system, right? And so that forms the memory, if you will. Um, and then of course, we need mechanisms to ensure that that memory is then fed into uh, the downstream agents in the right way so that it can be consulted as necessary.

• 37:46Speaker 00

  And that's, that's roughly what things like AutoGPT and Baby AGI are doing. So, uh, the short answer is this is pretty easy to do with Fixie, uh, but Fixie isn't currently, um, focused on that. We do think it's going to be an important use case. One of the biggest things that people are going to be concerned about there is privacy, right? And just like the problem that you have where, you know, you make a mistake and click on that shoe advertisement on CNN one day. And now for the next three weeks, you're seeing nothing but shoe advertisements all over the web, right? You want to avoid a situation where someone, uh, inadvertently poisons their memory with something that really is irrelevant to whatever they're trying to use the system for.

• 38:31Speaker 00

  And that ends up kind of causing it to veer off course or just become less relevant to them over time. And so I think there's going to be, uh, an important need for some kind of semantic clustering of that history. Uh, you know, a segmentation, if you will, so that if I'm interacting with the system in a work setting, okay, I use one kind of, kind of history. If I'm doing it in a personal setting, I use another kind of history. You know, the way I solve that is I have uh two different Chrome profiles, one for work, one for personal, and never the twain shall meet. I never look at personal stuff in my work profile and vice versa, because I want to avoid those two uh, you know, uh, sessions effectively intermingling.

• 39:17Nathan Labenz?

  I guess I'm, I'm interested in whether you think for now, the sort of long-term version is just like, not what your target enterprise customers are looking for, because that could definitely make a ton of sense to me. Uh, you know, you think of like, trying to bring this kind of technology to bear in customer service, and it's like, let's just get a single call, right? You know, if we can do that, we're, we're winning. And, you know, we can, we can, we can build on that, right? Versus like, we don't need long-term companions for our customers.

• 39:47Speaker 00

  That, that's really it. I, I, and most of the use cases that we're seeing don't have a need for permanent history. There is an affordance in Fixie today to, to record things for later consumption. Um, so for example, an agent could say, I want to stash away this piece of state or this text or this information about this particular user, so I can consult it later. And that ends up being used in a lot of places, but it's an explicit action, as opposed to implicit, just, oh, through the nature of our conversation, I remembered this information. The other thing that's important about it is that it has to be segmented by the end user, that you do not want to be in a situation where an agent is, um, you know, answering questions about someone else's order

• 40:33Speaker 00

  in the, in the, in the process of, of handling a, a, a single customer ticket. So I think it's a combination of those, but we do think that there's going to be a need for this kind of permanent history, or maybe semi-permanent history in the world of building agents that are more autonomous. So if you look at the AutoGPT stuff, effectively what it's doing is, you know, using the history as a way of itself figuring out what to do next. And that doesn't need to be like a permanent history, that doesn't need to be full AGI with a permanent memory in order to be effective. But it does provide a great deal more context for them to guide the model's operation.

• 41:17Speaker 00

  And that's something that we want to make sure we do provide in Fixie.

• 41:20Nathan Labenz?

  So going back to my kind of assistant use case,

• 41:25Nathan Labenz?

  in a, in a world where we have like these very episodic, you know, kind of ephemeral session style agents, presumably coming online and like working well first, then I can sort of see the like human assistant plus AI assistant being a very natural paradigm. But it does seem like that longer term memory, you know, starts to mature into form as well. So what do you kind of think is the future of, you know, how, what does a human assistant look like in a year? You know, or two years as these kinds of tools start to come online and like, maybe even just more broadly than that, I mean, I'm, I'm thinking hard about that.

• 42:10Nathan Labenz?

  But I'm also just like, what does the individual, just everyday, you know, computer user's experience start to look like? What do you think it's going to look like to sit in front of a computer two years from now and do stuff?

• 42:21Speaker 00

  Or even six months from now, I think is probably the more relevant question. I mean, you know, we're seeing all kinds of just incredible things that people are starting to build. There's a product called Ember that acts as a kind of a natural language agent that's embedded into my desktop environment on my laptop. There's one called MultiOn that's embedded into my web browser. And as soon as you have these things, and they're not just responding to explicit queries by a user, you know, invoking the hotkey and typing, you know, show me a bar graph of this thing, but they're actually paying attention to everything that's going on. Right, I I often joke that, you know, my laptop here, my little MacBook Air,

• 43:06Speaker 00

  this is the entire company fixy

• 43:10Speaker 00

  in this little tiny package. I mean, everything for the company that I do is right here on this machine. And so that's all the email, all the Zoom meetings, all of the, you know, documents, all of the code, everything is here. So there's no reason that an AI agent can't effectively have access to the full state of everything happening in my entire work life. And probably most of my personal life, too, right? So the question then becomes, what does that autonomy look like, I think? Because what you're speaking about is not just a person asking for help with something and then kind of getting a an answer. It's it's more of a an autonomous or a proactive

• 43:52Speaker 00

  process that's happening based on my, you know, the knowledge of what's going on in my life. And so, you know, we've seen little bits and pieces of that, you know, my favorite example, just in day to day use is things like, you know, using Siri to give me a reminder, you know, I can't remember anything in my old age. So I'm constantly asking my phone to remind me of things that I need to do in a couple of days or in a week or whatever. That's such a trivial one, though, because it feels like in the future the machine should just know that you've got to deal with this thing, or it's got to remind you that you need to take action here. And so the the place where I think it all needs to come together

• 44:35Speaker 00

  is, first of all, a beautiful streamlined and cohesive user experience, just like my point earlier about ChatGPT. Certainly, there were ways of doing ChatGPT before ChatGPT came along, but nobody did it because it was just, you know, too clunky. So the streamlined user experience is essential. One can make the same argument about the web, right? We had hypertext before there was the web. It was called Gopher.

• 45:07Speaker 00

  And it was awful and it was terrible. But it, it, it, you know, conceptually was the same thing. It just wasn't presented in the right ways. Right? So I think we need to have that streamlined user experience. I think the second is there really needs to be a way to connect an AI system to all of the personal information that you're comfortable giving it. That there needs to be a clearinghouse for that. Right now, it's way too fragmented. And you know, what I am looking at in my web browser and what's on my calendar and what's in my email and you know, what's happening in my text messages and what's happening on Twitter are all different places. And they're all, you know, completely separate from each other.

• 45:49Speaker 00

  And for an AI to help use any of that information, it needs to really have the same level of access to all that that I do. And so that kind of comes back to your point about delegation. Obviously, that needs to be done in a way that is, you know, highly transparent. Something that users can really trust, they can control, they can revoke the access anytime they want. And by the way, I think it needs to be personal in, by personal, I mean ideally running on your own device. I'm not sure that people are going to be comfortable with giving access to every single thing that you're doing in your digital life to Google or Facebook.

• 46:32Speaker 00

  Now, people might argue, well, but we already do, right? But we're doing so in a way that we're not necessarily aware of it. And so, you know, I used to be on the Chrome team at Google and you know, Chrome effectively is able to keep tabs on everything you do on the web. Every website you visit, every link you click. And most people aren't aware that that's happening. You can turn it off, of course, but that's just kind of the default behavior. And the reason they're not aware is that they don't tend to surface that information in their own. Day-to-day lives, there's a history tab buried away somewhere in Chrome that you can go see that stuff.

• 47:17Speaker 00

  But because it's not something I'm interacting with, I can kind of just ignore the fact that it exists. So those, those I think are the three main, the three main things that have to come together is the, the user interface, the clearinghouse for your data, but also really the transparency and the control over it. Once somebody builds that, and I don't think that's very hard to build. I think we're going to be off to the races in, in enabling people to leverage AI in a really meaningful way.

• 47:44Nathan Labenz?

  Yeah, the sort of single sign-on or like, you know, login with Facebook, login with Google, but be able to bring your sort of embedded content. I'm, a lot of things right now are leading me to this idea, this is a very kind of immature idea. A critic might even say it's a half-baked idea, but I keep coming back to this idea of like, bring your own embeddings as maybe how a standard kind of gets set. Seems like the sort of abstraction and maybe some like cryptographic layer on top of that as well could allow you to kind of never even share the like raw information,

• 48:28Nathan Labenz?

  but still give like semantic access to applications in ways that could be really useful.

• 48:33Speaker 00

  That's a fascinating idea, and it's not one that I've thought about. It, it has, it bears some similarity to some of the work that's been done in federated learning, where you're able to train models that operate on data from the collective, although it's a different approach, right? I like this idea a lot that it, it, you know, effectively leveraging the embeddings is really almost a one-way function from something that you've done down into this vector space, but you can't really go the other way. You can't necessarily go the other way.

• 49:04Nathan Labenz?

  Although I'd guess you could probably reverse pretty well.

• 49:08Speaker 00

  Today you can, today you can, but the question is, could you effectively, and I believe that there has been some research in the space of effectively, you know, taking the embedding vector that can be generated through a language model and applying a one-way function to it such that you can still perform nearest neighbor retrieval against that, but you can't go backwards from that transformed vector into, you know, the original content to the original context. And that seemed, unless you, you know, grant access to do that, for example. I think it's a fascinating idea. I do, I do think that some of these techniques will be important.

• 49:52Speaker 00

  I think, I don't think the world will be willing to wait for that. In other words, I think time and time again, we've seen that, you know, new technologies take, take off well before the world is ready from a security and a privacy perspective. I mean, look at the web. When the web started, there was no HTTPS, there was no SSL. Right? Everything was being done in the, in clear text for many, many, many years. And it took, and it, and once SSL came along, look how long it took to actually get all these websites to start adopting it. And so I, I think that we're going to end up in a place where people are happily granting the Facebooks and the Googles of the future,

• 50:35Speaker 00

  the access to all of their sensitive data with the belief that they're getting value from it, for sure.

• 50:42Speaker 00

  I did work at Apple for a time, and Apple has, I think, a really interesting perspective on this, which is, all else being equal, they want everything with your personal data to only live on your device. They, they don't want access to your personal data. And yes, there are things like iCloud backups and so forth, but they're, they're not trying to mine the personal data. They're actually trying to develop the power on the end-user device so that all the processing can happen there and all the storage can happen there. And I think that it holds them back, certainly. You know, the ad targeting and other things that one would like to do is not going to be as good, but I think it's ultimately where the world will end up. And I'm glad that they are focused on that.

• 51:28Nathan Labenz?

  Yeah, certainly might end up looking extremely prescient, especially as bundled with the, you know, heavy investments in the chips that, you know, can actually make it, make it go on the edge. That was, so on the question of the form factor, what do you think that might be? And, you know, could it be like the Apple Glass, you know, that we sort of, if the agents are good enough and then, you know, it's natural language interface, you might even be able to just kind of hum to it, or even, you know, it's not even too much of a stretch to be like, probably not first release, but like, just kind of think and have the glass like read, you know, directly from the neurons into what it's supposed to do.

• 52:12Speaker 00

  I think it'll look more like the movie Her, the Spike Jonze movie Her, where, you know, you're wearing an earpiece and you can talk to it. That, that seems more likely to be the ambient interface going forward. At some point, we may find that, you know, augmented reality glasses are a thing. When I was at Google, I was fortunate to be one of the, you know, few people that was able to use Google Glass in the real world, outside of the Google offices and use them in, in a real life setting as a dog footer. And, you know, of course, it never really caught on, but, you know, I think in, in large part because it was not really socially acceptable to be wearing a camera on your face. And

• 52:57Speaker 00

  so that's, that, that if there wasn't a camera, maybe people would say, okay, well, that's a quirky little screen or something. So, you know, it's, it's more around the social acceptability. The nice thing about the AirPods kind of UX is that it can be very kind of hidden. It's not something that's just like, you know, really obvious to other people that I have an AI here whispering in my ear, so to speak.

• 53:25Nathan Labenz?

  So imagining that it's kind of like it obviously can talk back to you and then it can show you stuff on the phone and.

• 53:31Speaker 00

  Exactly. And, and, and, and everybody has a phone. You already have the screen. You already have the touch interface. You already have the voice interface. You already have the, you know, the, the speaker in your ear. And so to me, that, that seems like the thing that will ultimately just really take off is that at any time I'm able to ask this thing or even just get proactive help from it um in the midst of doing something. I do think that there's going to be a need for textual interfaces for things where you're working on documents or on a screen, a larger screen. I, I, I don't think that just the audio interface is the only thing, but if you are thinking about taking it out into the, the real world, so to speak. Um, that's really where I think it's going.

• 54:14Nathan Labenz?

  The big zoom out kind of final question, what are your biggest hopes for and fears for society as this AI wave crashes over us?

• 54:26Speaker 00

  I think the, the big, there's so many risks in this space and it's kind of hard to know exactly. Um, you know, what the, the downsides are going to be. And I think we're just learning now about all of those risks. In terms of the, the positives and the upsides, the thing that I'm extremely optimistic about and hopeful for is that AI will enable anyone on the planet to take full advantage of computing. Today, if you want to take advantage of computing, uh, you need to be a member of an elite priesthood that knows how to write code and write programs and uh, or, or customize or tailor complex

• 55:09Speaker 00

  pieces of software to do what you want. Um, and you know, much has been said about digital literacy and making computers easier to use, but with AI, you end up with this natural language interface and it's not just an interface. I think it goes well beyond that to the reasoning abilities of these models and the, you know, task execution abilities of these models as we've talked about. So if you think about putting that in the hands of everyone on the planet. And no longer will we have a situation where only the very privileged, only the very wealthy, only the people that have had the right level of education will be able to take advantage of this.

• 55:52Speaker 00

  And I think that that's going to lead to kind of a second information revolution, if you will. Uh, so I'm very excited about that. As far as fears go, you know, I think about my kids and I think about what they're going to, what kind of world they're going to be growing up in. And, um, you know, there is this fear that, uh, the old ways of learning, the old ways of, um, gaining skills, the old ways in which you're evaluated in things like school and so forth are, are going to, you know, fall by the wayside. And, and no longer are people going to be evaluated on how effective they are at writing an essay

• 56:33Speaker 00

  or how effective they are at, um, coming up with a clever computer program to solve an assignment. And so then it becomes a matter of, well, what are we evaluating for? What is the educational curriculum of the future? And how do we decide who gets the As and who gets the Bs and who gets the Cs? Um, and then frankly, how much does that matter? Right? Is that many of these structures have been developed, um, in a world where we needed to grade people according to some kind of in what we believe to be an intellectual measure, but usually is not an intellectual measure. It's a measure of some other properties that are usually not that important.

• 57:15Speaker 00

  But with AI in the mix, uh, there is just such a change in terms of how the world will be organized. And, uh, I don't know what that means for the future and it's kind of hard to know what the rules are going to be. And so, um, I'd say cautious optimism there, but also a bit of a fear around, um, what that might end up looking like.

• 57:40Nathan Labenz?

  This has been a fantastic conversation. Matt Welch, thank you for being part of the cognitive revolution.

• 57:45Speaker 00

  Thank you very much.