What Is an AI Agent?

In this episode of AI + a16z, a16z Infra partners Guido Appenzeller, Matt Bornstein, and Yoko Li discuss and debate one of the tech industry's buzziest words right now: AI agents. The trio digs into the topic from a number of angles, including: - Whether a uniform definition of agent actually exists - How to distinguish between agents, LLMs, and functions - How to think about pricing agents - Whether agents can actually replace humans, and - The effects of data siloes on agents that can access the web. - They don't claim to have all the answers, but they raise many questions and insights that should interest anybody building, buying, and even marketing AI agents. 00:00:39 - How do we define “AI agents” 00:04:40 - Degrees of agentic behavior 00:09:34 - Can agents really replace humans? 00:15:32 - Is it agentic behavior or a software function? 00:18:46 -  How should we price AI agents? 00:22:54 -  Identifying true value 00:25:59 -  Where do agent capabilities live? 00:29:40 -  Agents vs. a hostile web 00:34:04 -  What does success look like?

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:03Erik Torenberg?

  Today, we're discussing one of the busiest and most confusing terms in AI right now: Agents. Are they just fancy rappers around LLMs? Full-blown autonomous workers? Or something in between? A16Z Info Partners Guido Appenzeller, Matt Bornstein, and Yoko Lee break down the technical definitions, pricing models, use cases, and why the term "agent" means so many different things to different people. If you're building, buying, or just curious about what agents are and aren't, this episode is for you. Let's get into it. As a reminder, the content here is for informational purposes only. Should not be taken as legal, business, tax, or investment advice, or be used to evaluate

• 0:45Erik Torenberg?

  any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com/disclosures.

• 1:08Guido Appenzeller?

  So I think there's some things which are probably kind of easy to say, which is that, A, there's a good amount of disagreement, what is an agent? We've heard a lot of different definitions of it, both on the technical side as well, I'd say, on the marketing and sales side in some cases, because there's some sales models associated with it. So let's start with the technical side. I think there's sort of continuum here. You know, the simplest thing that I've heard being called an agent is basically just a clever prompt on top of some kind of knowledge base or some kind of context that has this sort of a chat-type interface. So from a user's perspective, this looks like an human agent would look like, right? So for example, I ask it, "Hey, I have a technical problem with my product XYZ."

• 1:50Guido Appenzeller?

  It looks at the knowledge base and comes back with a canned response.

• 1:54Speaker 02

  But there doesn't have to be a knowledge base, right?

• 1:57Guido Appenzeller?

  It doesn't even have to be a knowledge base. I see, got it. So maybe it's just a trained model, it's all the model weights, the knowledge. So it's even simpler. So an agent could just be an LLM, but a chat interface or something like that, by some definition, right? I think on the other end of the spectrum, there are some people who basically say for something to be a real agent, it has to be something fairly close to AGI, right? It needs to persist over long periods of time. It needs to be able to learn. It needs to have a knowledge base. It needs to work independently on problems. If you take then the most extensive definition, is it fair to say that doesn't work yet?

• 2:30Speaker 02

  I think so, it doesn't work yet.

• 2:34Guido Appenzeller?

  Will it ever work?

• 2:36Speaker 02

  That's a philosophical question.

• 2:37Guido Appenzeller?

  All right, fair. Very fair, very fair. So if we take that continuum in between, is there at least a way to chop that up into a couple of categories of maybe degrees of agentic behavior?

• 2:52Speaker 02

  And different types of agent. There's some artsy agent that help artists to come up with new Bezier curves.

• 2:59Guido Appenzeller?

  Yeah, yeah.

• 2:59Speaker 02

  There's coding agent, which we like to talk about as the agent of the day. Yeah, which we use. There's agent that's just a wrapper on top of L L Ms.

• 3:08Guido Appenzeller?

  That's right, yeah.

• 3:09Matt Bornstein?

  I may be the contrarian in this group.

• 3:12Guido Appenzeller?

  All right.

• 3:12Matt Bornstein?

  Look, I kind of think agent is just a word for A I applications, right? Anything that uses A I kind of can be an agent now. Before, before we started this talk, I actually went online just to refresh myself about some of the more interesting A I agent perspectives out there. I found a really cool talk from Karpathy that he gave a couple of years ago about agents, which I can describe a little bit. But the really funny part was on the YouTube recommended videos to watch next. It's like A I agents are going to revolutionize your lifestyle and the rise of super intelligent A I. You know, it's just kind of like marketing. And so I actually do think that's what's going on in a lot of ways. The cleanest definition I've seen of an agent is just something that does complex planning

• 3:56Matt Bornstein?

  and something that interacts with outside systems. The problem with that definition is all L L Ms now do both of those things, right? They have built in planning in many cases and they at least consume information, you know, at least from the internet, maybe from some servers that expose information through M C P or some other protocol. So the line really is very blurry. And you know, what was so interesting about the Karpathy talk is he basically, he related to autonomous vehicles and said, AI agents are a real problem, but it's like a 10-year problem. It's like a decade problem that we need to work on. And I think most of what we're seeing in the market now is like, is not the decade version of this problem. It's like the weekend demo version of this problem. And this is why we sort of generate so much confusion.

• 4:38Matt Bornstein?

  You have this kind of poorly defined nebulous thing that LLMs are kind of consuming themselves over time. And so I don't think anything we have are actually agents is kind of, and agent itself may be a poorly defined and kind of overloaded term. But if someone's willing to do the hard work and define exactly what it's like to kind of be a human, but in digital form and spend 10 years to make it actually work. You know, that's sort of what I'm excited to see.

• 5:00Speaker 04

  Okay. So defining agents is a difficult job. Maybe it's easier to talk about how people use the tools they call agents and what are the differing degrees of agentic behavior?

• 5:09Speaker 02

  I wonder if part of the conversation is redefining agents, because we all know that agent as a term is just not a great term. It means so many things to so many people. If it's interesting to dissect, like, what do we mean? What do different people mean when they say agents? What are different ways we could utilize this process we call agents?

• 5:28Guido Appenzeller?

  So it seems to me this, if we're trying to define agents, or maybe even degrees of agentic behavior, which might be a little easier.

• 5:35Guido Appenzeller?

  There's something like a user interface aspect to it, right? Where something that's a pure copilot, but basically a user goes back and forth with an LM to work on a particular task that's often not called an agent. Is that fair? There's a little bit the copilots versus agents UI models.

• 5:53Speaker 02

  Yeah, I guess like, what are the elements we would think that goes into agentic behavior? Like Matt mentioned, planning could be one. There could be decisions made by the agent. There has to be LLM somewhere, but curious about your take.

• 6:08Guido Appenzeller?

  So I think another definition we heard from Anthropic recently was this idea that an agent is an LLM running in a loop with tool use, right? Which there's two important parts to that. One is this notion that it's not just a single prompt, and not even just a single static sequence of prompts, right? But something where the LLM takes the output of a prompt, feeds it back into itself and based on that makes decisions on what the next prompt and likely also when to abort, like you went to, went to complete a task. I think that for the real agents or the more agentic behaviors, I think that's a reasonably good definition. And I think the other thing-

• 6:44Matt Bornstein?

  But just by that definition, isn't every-

• 6:48Matt Bornstein?

  Chatbot effectively an agent then in this world, right? Like if I go just to chatgpt.com and use their latest reasoning model with web search, right, isn't it using tools and like feeding, you know, feeding its outputs into a new prompt in order to do kind of chain of thought?

• 7:04Guido Appenzeller?

  Chain of thought is a little bit in between. If it's just a single prompt that comes back with a result, then it wouldn't have this notion of planning and doing a more long-term concept and deciding itself when it is complete, right? If you have a chain of thought reasoning where I'm given a more complex task, that's starting to look agentic, I agree.

• 7:20Matt Bornstein?

  I just think it's really tough to define a system based on what someone says to it, right? Because these are by design unstructured inputs. These systems will accept literally anything. And so sure, if you tell it, you know, what's today's weather? I would agree that's not agentic, right? That's just fetching, you know, from an API. If you ask it, define a new philosophy of weather, right? It'll happily go do it, right? So it's like an agent, if you ask it one thing, but not an agent if you ask it another thing. I think that's kind of a lot of the confusion in the market around this. And, you know, if we spoke in the terms that you're talking about Guido of like, hey, this is an LLM in a loop with a tool. Like, that's actually a much more productive way to talk about it, I think.

• 7:58Guido Appenzeller?

  Yeah, yeah. I mean, that said, it seems like we're seeing to some degree a specialization of user interfaces in sort of two directions, right? There's, let's say, a cursor or something like that, which really emphasizes the tight loop between the user, the tight feedback loop between the user and the LLM and the thing I'm working on, right? So I want immediate gratification when I do something, you know, and so response time matters. Then there's sort of more the back end as, you know, source code management system type plugins where it's more about throwing something over the wall by maybe answering a couple of questions. And then you try to maximize the amount of time the agent can work independently. So it seems like, I think you're right that there's no clean system definition split between the two, but there seems to be a little bit of user interface specialization. Is that a fair statement?

• 8:43Speaker 02

  It almost feels like for all the use cases we've described, there's one element that all agents have, which is reasoning and decision. Would you call just a call to LLM to say, translate this text to JSON? That's probably not an agent. But then if you ask LLM to say, hey, decide where, you know, this response goes and routed for me, it feels more like an agent than before. So it almost felt like planning. I'm actually not sure, does the agent need to plan or does it need to decide? Maybe both? I actually feel like it's a multi-step L M chain with a decision tree.

• 9:19Guido Appenzeller?

  A dynamic decision.

• 9:20Speaker 02

  A dynamic decision tree.

• 9:21Guido Appenzeller?

  Yeah, I think that's fair.

• 9:22Matt Bornstein?

  I think we've all just been nerd sniped. I just think, you know, it's like humanities, people love classifying and, you know, they, they draw a kind of like fine distinctions between different types of things, entities, whatever. We're computer scientists, like, you know, you know, not that there's anything wrong with humanities, but we're just not that. So I think we're not well-equipped when it's a bit isn't just zero or one, it's maybe something in between and we just talk about it a lot.

• 9:47Matt Bornstein?

  We like try to like coerce it to one value or the other.

• 9:49Speaker 02

  Yeah.

• 9:50Speaker 04

  Of course, agents are more than pure technology. They're also becoming products, which means they need to be marketed. And how someone positions their product has a major effect on how they price it. What's more, the ultimate value of any given agent, which is still to be determined for the vast majority of them, is to what degree they can actually replace or simply augment human workers.

• 10:13Guido Appenzeller?

  There is an interesting point, which is, I think there is a marketing angle to agents, right? I've heard this narrative from a couple of startups that they're basically saying like, hey, you know, we can price the software that we're building much, much higher because this is an agent. So we can go to a company and say, you're replacing a human worker with this agent. The human worker makes, I don't know, $50,000 a year and therefore this agent you can get for only $30,000 a year. This sounds really compelling from a first glance. And actually, I mean, there's some value to it in the very early days because it essentially, it's very easy to understand comparative pricing for somebody who has to make a buying decision, right? You know, on the flip side, we all know that the cost of a product over time converges towards the marginal cost of production, right? And so today, if I used to use a translator, maybe to translate a page of text, today you use ChatGPT.

• 10:57Guido Appenzeller?

  I do not pay, pay ChatGPT like I paid my translator. I paid a tiny fraction of a cent, right? Which is the, via the API, which is the actual cost. So I sort of wonder, how much of the agent debate is driven by marketing and pricing?

• 11:11Matt Bornstein?

  I just actually think this is a really interesting topic. What fields can you think of that are actually suffering complete replacement from AI or AI agent? And this is a setup, I'll warn you. I have another extreme point of view that I'll say afterward, but can you think of fields where this is actually happening?

• 11:27Speaker 02

  Not completely, but definitely partially because there's a lot of, for example, voice agents I replace.

• 11:34Speaker 02

  Receptionist, I don't know if we should name. Replace people who would, you know, get back to customers. So there's definitely a lot of workloads that have been offloaded from the folks who traditionally did the job. But I don't think they're, you know, a hundred percent replaced. They can't, you know, they can do something else. But we are seeing headcount growth in some areas are slowing. So it's not that existing jobs are being replaced. It's more like they're hiring net new humans slower.

• 12:02Guido Appenzeller?

  I think you're exactly right. I mean, I think in few cases, humans will get replaced by AI. In most cases, you know, two humans will get replaced. One human that is more, by one human that's more productive with AI.

• 12:13Matt Bornstein?

  Or, yeah, or maybe they keep the two employees.

• 12:16Guido Appenzeller?

  Maybe they go to three employees because now they're more productive.

• 12:18Matt Bornstein?

  Yeah, right, right. It's just a really interesting question. And the reason I think it's really relevant to agents is I think part of the ethos and part of the confusion around agents is this idea that we actually will develop human replacements, right? And that this thing we called an agent, which by the way is a name for a person, right? Before we had AI, we had people called agents and we still have all kinds of people called agents. And it just doesn't seem like that's happening, right? Not in the replacement sense, right? You, you mentioned Yoko with agents. We've always had, you know, customer support automation. You know, we've had one eight hundred numbers where you like press one for sales. Plus, you know, that's existed for a long time. This is a much better form of that, obviously. Translation is a great example too, Guido. These systems can perform translation extremely well, but you're probably not going to just stick something to chat GPT

• 13:03Matt Bornstein?

  and then publish it on your website, right? There is actually work that needs to take place. And I think the reason for this is there's just fundamental creative work in most things that humans do, right? I, I think from our kind of perch in Silicon Valley, we can forget that sometimes, that people all over the country and doing all sorts of jobs actually have hard jobs and not just hard in, in the sense of someone's got to do it jobs, but hard in the sense of it does take thinking and human decision making, which I, I just don't know that AI kind of has what we would think of as decision making or intent, right? It's a system that's still, somebody has to push the button, right? It may be running somewhere, it may do a great job of whatever. Someone tells us to give it a prompt and hit go. And to me, that's a lot of the, that's a lot of the confusion around.

• 13:43Matt Bornstein?

  We're all thinking at some point, a human person with intent and creativity and thinking is going to be replaced. I'm just not sure that even is theoretically possible, right? It's almost just like a catch 22 to say an AI system is thinking for itself, right? Because somebody has to have sort of created, you know, this is old sci fi philosophy I'm getting into now. But like, I actually do think it's a part, a big reason for the confusion that, you know, we sort of experience now.

• 14:04Speaker 02

  It's interesting because there's two types of agent we're already talking about. There's one type where the agent is replacing humans, work with humans, do things humans can do. There's the other type of agents as more low level system processes. They work with each other, they hand off tasks to each other. To some extent, agents are like technical details in the system in that way. But we mean both when we talk about agents.

• 14:27Guido Appenzeller?

  In that case, is there actually a difference between an agent and a function?

• 14:31Speaker 02

  I think so. I think agent will be multiple functions with our L M in the middle.

• 14:36Guido Appenzeller?

  If I have a low level agent and I'm giving this low level agent a task and I get back a task result. It looks a little bit like a classic API call.

• 14:43Speaker 02

  But with the L M in the middle to make decisions on what to do for that API call.

• 14:47Guido Appenzeller?

  So, but I understood, but that's sort of how this function works internally.

• 14:51Speaker 02

  Yes.

• 14:51Guido Appenzeller?

  To some degree.

• 14:52Speaker 02

  Yes.

• 14:52Guido Appenzeller?

  Right?

• 14:53Speaker 02

  Yeah.

• 14:53Guido Appenzeller?

  So from the outside, would I care?

• 14:55Speaker 02

  You wouldn't care. It's like, most of the time when we see AI SDRs. When we talk about AI SDR agents, what we mean by that is when the agent can go to the CRM, pull something out and then filter the list, draft an email and send the email. So that feels very process level instead of human level. Yeah. So that's what I meant.

• 15:19Guido Appenzeller?

  If you don't know how this thing works internally, a classic function and an agent become indistinguishable.

• 15:25Speaker 02

  Totally. I absolutely agree. But when you, as a programmer, when you find, write the function, you will define agent, that that's the, that's this.

• 15:35Matt Bornstein?

  Implementation.

• 15:36Speaker 04

  We'll get back to pricing shortly. But first, let's dive a little deeper into this discussion of how interacting with an agent is different than or similar to traditional software based functions.

• 15:47Matt Bornstein?

  So here's one interesting thing to think about on that topic. I totally agree with you, Guido, and I think you sort of agreed too. It's really a function if you kind of just look at it that way. Shareable, reproducible functions have never really been a thing. This has been one of these longtime goals that people in the market have tried to say, oh, I can just write a function and then anybody on earth can use it, right? You know, we have packages, right? That you can download a whole package with various functionality, but literally just one function that you can share. If you kind of squint a little bit, that kind of exists now with AI, right? Because you have these models that's trained by somebody. Somebody else may download it, fine tune it, train a Laura, package it up into some new and interesting way. And then it's actually immediately available for someone else to use on hosting services or Hugging Face or something like that.

• 16:31Matt Bornstein?

  So while it does seem to be just an implementation detail, whether you're using an LLM or not, there is this interesting thing where the model itself takes up so much of that functionality in the function. And it's just a different kind of animal compared to normal code. It's actually more, it's kind of shared by default in a way because nobody's going in and training their own model every time they're writing code. You know, it's obviously heavy, right? It's harder to move around. There are all these different characteristics from normal functions, some of which are actually very desirable. Some are kind of, you know, bad, right? Characteristics you don't want, but many of them are kind of interesting. And I think we'll actually see new infrastructure, new dev tools kind of built around this in the long run.

• 17:06Guido Appenzeller?

  I think it would make sense. I mean, when, if we go back in time, the last time we sort of invented a major new component for building systems, which was probably networking, right? How we thought about calling a function before networking afterwards changed a lot, right?

• 17:20Matt Bornstein?

  Totally, totally.

• 17:21Guido Appenzeller?

  The complexities of APIs and the infrastructure around it is completely different today.

• 17:25Speaker 02

  This is such a good point because now I think about it, I feel like humans are just functions too. Like if you have a thought experiment and then replace LLMs in the program to a human, like how the kind of answers we'll give to the program is not that different from what an LLM would give to the program.

• 17:42Matt Bornstein?

  So if we actually all get hooked up to servers one day and can be called as a function from Lambda, then I will agree that agents have been created. So that's what an agent is.

• 17:52Guido Appenzeller?

  Isn't Mechanical Turk exactly that? Or maybe even your email inbox?

• 17:55Speaker 02

  There's an Amazon Go supermarket a while back in SoMa. I think they were advertising that it's computer vision models behind the scenes identifying what you took from the supermarket. But then people found that they hire a lot of people behind the scenes to actually label the data in real time. So the humans in that case are the functions that today maybe

• 18:19Guido Appenzeller?

  Secret agents.

• 18:20Speaker 02

  Right, replaced by LLMs with

• 18:22Matt Bornstein?

  Well, but this was exactly my point, though, right? There actually is important creative work, even in a grocery store checkout clerk, right? You could naively think, oh, this is an easy job. Actually, it's not an easy job at all, right? And so you can take this work and kind of shift it, right? And you can squeeze it down with automation and stuff, but it never really goes away.

• 18:41Speaker 02

  Oh, yeah, absolutely.

• 18:42Matt Bornstein?

  Yeah.

• 18:43Speaker 04

  All right. So given all of this, how should companies think about pricing their agents? Per seat? Per token? Per task? Hint: It might be too early to truly tell.

• 18:54Guido Appenzeller?

  Usually, if you introduce a brand new product category, right? You often initially put a pricing that prices against the status quo, right? Whatever you replace or augment in some cases. But let's assume we have a direct replacement, right? So that's, I think, where this idea from, oh, this replaces a human, which it doesn't. And but if it would, right, then you could charge, you know, X amount for it. Usually, over time, competition kicks in, right? And you're effectively priced by how much your competitors are charging, right? And it starts off in erosion. Then it depends on many things like how much of a mode do you have? Do you have customer login, right? And so on. Long term, converge against the marginalized marginal cost of production, right?

• 19:35Guido Appenzeller?

  Which, I mean, look, if I look at most agents today, it's probably very low, right? Any agent you can purely model in software with a couple of LLMs calls, you can run at a very low cost. And the cost is decreasing over time.

• 19:48Matt Bornstein?

  And I would sort of argue that's kind of already what's happening. That in practice, most AI applications, and if we, in particular, if we want to call them AI agent applications, you know, they have their sales pitch around, you should pay us X because we're saving you. You know, it's like a classic ROI calculation.

• 20:07Guido Appenzeller?

  Established value.

• 20:07Matt Bornstein?

  Yeah, exactly. Value-based pricing, you know? But in practice, I think most buyers are actually pretty sophisticated about what's going on under the hood. And to your point, they know it's a pretty simple stuff happening. And so it's like, hey, what does it cost you to run all these GPUs? And we'll pay you some premium over that. And I think that's how a lot of vendors are pricing in practice these days.

• 20:26Guido Appenzeller?

  I mean, long time you'd expect pretty healthy margins, just like in SaaS, right? Which software traditionally has very good margins.

• 20:32Speaker 02

  It's so funny because we always advise companies to not price based on the margin, but price based on the value you add, whatever that could be. It could be compared to other vendors on the market, could be compared to just, you know, what it is building in-house. And traditionally for infra, a rule of thumb, not always the case is that if the surface is used by a human, it's a perceived pricing. And if it's a service is used by other machines, it's a usage-based pricing. And I actually don't know where to put agent here.

• 21:04Guido Appenzeller?

  Well, it could be used by either, right?

• 21:05Speaker 02

  It could be used by either.

• 21:08Matt Bornstein?

  Look, I think your analysis is exactly right. And the reality is, most AI companies don't know what value they're generating. Yet, this is so new and so nascent that it's like, hey, we're just going to charge something that we're not going to lose money on. And, you know, in the case of OpenAI, they have how many millions of users? They probably don't have a very strong sense of what they're all using it for. And once they do, right, and you see this more, they're trying to verticalize a bit more and have kind of specific products for specific use cases, code obviously being the big one. You know, then you'll be able to see the pricing kind of catch up, is kind of my hypothesis.

• 21:40Speaker 02

  This reminds me, the OpenAI point you brought up. I was thinking about AI companions, because that's the closest to perceived, perceived human pricing. Like, you can't charge someone every sentence they talk to their companion, although some of the foundation among those.

• 21:55Matt Bornstein?

  Ooh, they're, there are services that will charge you per response. I haven't used them, but they do exist.

• 22:02Speaker 02

  I see. Wow, okay. So usually, it's kind of weird to charge someone, like, by tokens of how much they talk to the companion, rather than like a flat monthly fee.

• 22:12Guido Appenzeller?

  It doesn't feel like a true friend.

• 22:14Speaker 02

  Right, exactly. It's very transactional.

• 22:17Matt Bornstein?

  This is, look, this is all theory, right? People love sitting around and talking, oh, we're going to charge per person, per task, per, you know, world economy that we rescue. You know, it's like, it's all made up, right? I think Guido's thing was exactly right. Let's look at the actual technology underlying what we're calling agents right now. Where are they being deployed and why? And honestly, the pricing, the marketing, the sales tactic, all of this kind of follows from what they're actually selling.

• 22:40Speaker 04

  If I'm selling something that looks like an agent, but I haven't truly figured out the value I'm providing to my users, how do I justify the jump to a higher price point when I do figure out that value?

• 22:50Matt Bornstein?

  You just need to be selling a solution rather than a product, right? This is really well-worn expertise in enterprise go-to-market. Code, you can somewhat see the decoupling of price from the underlying technology now because it really works. There's a very clear ROI to people who use it. And so as a VP of Engineering or a CTO, you can look at this and say, okay, I'm actually saving a lot of money and my guys are getting a lot more productive. I can value, I can do a normal ROI.

• 23:14Speaker 02

  And they're happier?

• 23:15Matt Bornstein?

  Yeah, so you're kind of buying a solution, right? You're buying from a vendor, something that solves a problem for you, which again, Microsoft, Oracle, Salesforce, people have been doing forever. Once we start to see more of that, it's going to be these things that become real products and kind of decouple pricing and look kind of like real businesses, I think.

• 23:28Speaker 02

  I think it's dictated by the high-level application. So I'll give you an example. So I'm a Pokemon Go player. So for those who have played Pokemon Go, once you collect enough Pokemons, you are out of storage in your pocket. So you need to pay extra to buy a new bag, virtual bag, that you can put more Pokemon in. And as an infrastructure investor, I invest in storage businesses. And then when I look at how much I need to pay for like 30 extra Pokemon, it was thousands of types more expensive than what storage is. So it actually reminded me- I'm surprised it's something thousands. It's only- What if it gets 10 to the 15 or so? There's a whole price curve on Pokemon storage, it turns out.

• 24:10Matt Bornstein?

  Because this is one JSON blob, basically.

• 24:12Speaker 02

  It's one JSON blob. I know.

• 24:15Matt Bornstein?

  And they charge you like five dollars.

• 24:17Speaker 02

  And then the Pokémon, normal Pokémon players, they wouldn't think about this, like how much do you storage cost, right? Like a normal Pokémon player would be like, oh, this capability, I would be happily paying thousands more than I were to have a S3 bucket somewhere. So one of it is monopoly. So it's the application layer monopoly that you wouldn't have been able to store the Pokémon anywhere else. And two, it's a use cases. It's for a different audience. They wouldn't be asking these questions. So it'll be thinking about what is the net new value, what's the net new cost I will be willing to, you know, for the bill for if I were to get this value. Is it a fun game? It's a fun game. Take hundred more dollars.

• 24:55Matt Bornstein?

  Yeah, I think that's exactly right. And in implicit is what you're saying is this idea that the product or the solution has to actually work for them, right? For a less technical person who's, you know, the person who's not going to try to provision their own storage bucket to self-host

• 25:08Guido Appenzeller?

  Can it remote us three for Pokémon? Yeah.

• 25:10Speaker 02

  And it's quite defensible differentiated too, because, you know, Pokémon Go is not open source. There's no other replacement of Pokémon Go. There's only one Pokémon Go. So there's only one place where you would be willing to pay so much money for Pokémon storage.

• 25:25Guido Appenzeller?

  Plus very strong brand, plus you have a little bit of network effect because you can play together.

• 25:28Speaker 02

  Yeah. And then we'll see the AI agent version of this. I can't wait to see the AI companion version of this. Playing storage for AI companions wardrobe.

• 25:38Speaker 04

  As the AI market continues to shake out and evolve, where will agent capabilities ultimately live? For example, can they live inside LLMs? Or must they call external tools? And who's ultimately in the best position to influence this?

• 25:52Guido Appenzeller?

  Super interesting question, right? What's the systems perspective of how an agent is built? And I personally think that architecturally, there really is no difference between your typical SaaS software today and agent in terms of how you build it, right? And let me explain why, right? So an agent is that you have sort of an overall loop with an LLM and prompts that feeds into itself plus external tool use. The LLM itself, you probably want to run a separate infrastructure just because it's highly specialized. You need these vast GPU farms. You can't easily run today's large LLMs in a single GPU. So it's a very specialized infrastructure that's externally. So the LLM call is external. The state management, well, today in SaaS applications, we do all the state management externally in databases or something like that.

• 26:36Guido Appenzeller?

  So you probably also want to externalize that, right? And then what remains is fairly lightweight logic, right? Where basically I'm taking context that I retrieve somehow from databases. I assemble that into a prompt. I run the prompt and then I occasionally invoke tools. Maybe I do that with MCP or something like that with an external server. But the core loop is actually pretty lightweight, right? And I can run exzillion agents on a single server. Not exzillion, but many agents on a single server. I don't need a lot of compute performance for that. Does that sound about right?

• 27:05Matt Bornstein?

  Yeah, yeah, I totally agree. The interesting architectural question for me has always been, how do you handle the kind of non-determinism that may come? Many of these successful AI applications that we all use and love really just spit model outputs back out to the user, right? Like a chatbot or image generator. It's like, hey, I called the LLM. Here's what I got, you know? Good luck. When you try to actually incorporate the output from an LLM into the control flow of your program, that is actually a very hard, very unsolved problem that, you know, to your point, there are relatively minor architectural differences today, but this may actually drive more significant changes in the future.

• 27:41Speaker 02

  I actually think the winners will be the specialists, not the foundational models. It's the people who will build on top of the foundational models, who are fine tuning the foundational models. So like a very artistic example of this is that I've been spending the last two weeks just prompting GPT-4O, their image model. It's very good at cartooning, so it's very good at manga. It can spell, so it has the storyline. But then I realized that there's only top two or three styles it's good at. So it's good at Ghibli, it's good at manga, and then there's variations of the style in that realm. So now where art comes in is that the market likes out of distribution art. Everyone doesn't want to see the same things over and over again because that's how they value art, something that's different.

• 28:27Matt Bornstein?

  Ideally, maybe.

• 28:28Speaker 02

  Ideally, maybe.

• 28:30Guido Appenzeller?

  Then some reason define art as out of distribution samples?

• 28:34Matt Bornstein?

  Yeah.

• 28:34Speaker 02

  Yeah, art can be in distribution as pop art, right?

• 28:39Speaker 02

  It could also be out of distribution. That's like when impressionism came up many years ago. Everyone was drawing impressionism. And at the time, the painters before, they were like, what's wrong with your eyes? Why are you drawing blurry images? So styles come and go. But because of that, I think it's a pushing distribution question. How the foundational model will never cover 100% of everything. So it's really up to the humans and specialists of the next wave to come up with the new data, new workflows, new aesthetics to push that distribution.

• 29:11Speaker 04

  Of course, at the end of the day, agents are only as useful as the tools and data to which they have access. So what happens if major web platforms decide they want to keep agents from accessing their data?

• 29:22Guido Appenzeller?

  It seems like one of the hardest things about agents today are data modes. Right? In some cases, just because they're technically difficult and trying to access data and agents trying to access data is just very hard to integrate with that system. In some cases, it's very deliberate, right? My iPhone, the photos are not accessible via any API because it's a walled garden.

• 29:41Matt Bornstein?

  So sort of data silos you're talking about.

• 29:42Guido Appenzeller?

  Data silos, right? So is that something that's holding back agents or is making them more difficult? Or to make it even stronger, consumer companies traditionally often were opposed to offering automated access to their services because they want their user engagement. They want the time to advertise to the user. Will that limit how much we can deploy agents?

• 30:04Speaker 02

  And would that be changed once we have the browser native agents that can browse the web and browse?

• 30:09Guido Appenzeller?

  Great question, yes.

• 30:11Matt Bornstein?

  I think that, I think Yoko is totally right. You know, it's like there are strong incentives for people who own data about, you know, physical entities, you know, people, businesses, et cetera, to keep it to themselves, right? Especially because they may be scared what AI is going to do to them, by the way. So they're kind of clinging tight to what they have. And these problems are rarely solved by defining a new protocol and just saying, hey, if we make it easy for people to give away their core assets, they'll just do it. You know, obviously, you know, that's very unlikely to work, but. Someone eventually will solve this by saying, hey, if your data is publicly visible, we're gonna get it. You know, it's like, by the way, it's not actually your data, it's data about me. So why should you be holding on to it?

• 30:49Speaker 02

  Yeah, actually, I feel like the new advancement in models may just change the data mode, kind of to the point of today, web browsing using an agent doesn't work super well. It's very slow, it's very clunky, you have to try it multiple times for it to do any task. But imagine if we have foundational model capability of giving an agent ability to go to any website, logging as a human, we'll table that one. I don't know how agent identity works yet. Or go, you know, go SSH into a server, like execute certain commands or like spin up a virtual machine for a mobile or access a device, devising of device farm to play Pokemon Go.

• 31:28Speaker 02

  Like, maybe those are the data traditionally only available to humans under that account, now may be available to agents.

• 31:37Guido Appenzeller?

  There's also the opposite that could happen, right? That basically all the consumer sites are starting with more and more complex anti-agent captures, trying to keep out the agents because they only want the humans that have attention to come to those sites. I mean, I recently did use one of these deep research tools, one of the major LLMs. And one of the steps, if you look through it, all the steps I went through was like, you know, trying to see how I can get around a capture mechanism for a site. There was an actual reasoning step, right? Where basically it felt it know what information it wanted and it was blocked from accessing it. So is that, you know, how dystopian is the future gonna be here? It solves it, actually.

• 32:10Matt Bornstein?

  I mean, it's so interesting. So here's a really early machine learning example of this. I don't know if you guys remember when Gmail first implemented ads. It was a big controversy because they basically said, okay, we're not going to read your emails, but our algorithms are going to read your emails and we're gonna suggest ads that you should watch, you know, or click on based on that. We all sort of, I think, just forgot and got used to it. I still think we don't love the idea, but we kind of lived with it. But some of the data providers reacted by removing data from email, right? So Amazon famously now, when you order something, they send you a confirmation email that says, hey, you just ordered something. Click here to find out what you ordered, when it's going to arrive, or any information you might want to know.

• 32:51Matt Bornstein?

  And so that actually did happen in practice in that example, that the major data holders kind of found ways to withhold it. It'll be interesting to see whether that's possible now or not.

• 32:59Speaker 02

  But that same data is scraped on the client's side from the ad networks that install.

• 33:06Matt Bornstein?

  Oh, sure. Yeah, yeah, yeah. Yeah, there's always some other way. Yeah, not maybe exactly the same, but pretty good proxy. Yeah.

• 33:12Guido Appenzeller?

  It may be that it's much harder to tell the difference between an L M and a human than a classic, you know, sort of

• 33:18Matt Bornstein?

  Yeah, yeah.

• 33:19Guido Appenzeller?

  API call mechanism and a human. That may be, that may change the dynamics.

• 33:23Speaker 04

  Finally, Guido, Matt and Yoko answer an obvious question on the longest timeline into which we might have clear visibility. What needs to happen to make agency truly game-changing innovation within the next, say, two years?

• 33:35Guido Appenzeller?

  I think the positive vision is that in two years, we figured out how an agent working on my behalf can use most of the tools that I have access to. I think it's also clear what are all the pieces that are missing for that, right? We have not figured out security, authentication, access control for agents working on my behalf yet, right? We have not figured out how data retention works, you know, we have not figured out the relationship with consumer websites that potentially want to block that agent. But if you had that, it could make many tasks much, much easier, right? Today, if I have data sitting, say, my Google Drive or so, right? How easy I can reason about that data versus other data that's in more fragmented sources, it makes an incredible difference.

• 34:18Guido Appenzeller?

  And so I think that's the bull case, right? Where you have agents that can take all the data that you can access, they can access it on your behalf and perform tasks on your behalf, right? And save you a ton of time. It could make you, depending what you do, like, you know, multiple times as productive as you are today.

• 34:32Speaker 02

  My answer to that is actually different modalities on the foundational model. Today is still very much text-based and that worked really well for coding and text-based tasks. But then for more visual-first tasks, there's just no one-to-one mapping. Even for web browsing, it's like a very clunky experience of take screenshot every couple of seconds and send it back to the foundational model. So I will actually bet on multimodality when it comes to if we train the model with different traces of clicking on buttons on the website, navigating the web, using different devices, drawing, producing vector art. I think there will be that new things that the model could unlock on the agent level.

• 35:11Matt Bornstein?

  You can probably guess my answer. If we don't use the word agent two years from now or five years from now, I think that's a huge win. There's actually a fun paper put out by some folks at Columbia, I think, called AI is Normal Technology. And they sort of make the argument that there's a false dichotomy out there. It's like. AI is either going to bring about utopia or dystopia, meaning everything's going to be amazing because we have AI, or everything's going to be terrible. This is kind of the national discourse. But if you just think of it as normal, right? Like water, or electricity, or the internet, or things like that, I think that's the world we're kind of headed towards. An agent is this kind of way to help us get there. And so that's my goal. I mean, this stuff is just incredibly powerful. We understand how to use it. We understand the use cases, and we're kind of putting it to use for us.

• 35:58Erik Torenberg?

  Thanks for listening to the A16Z Podcast. If you enjoyed the episode, let us know by leaving a review at ratethispodcast.com/A16Z. We've got more great conversations coming your way. See you next time.