The Race to Recursive Self-Improving AI

a16z General Partner Martin Casado sits down with MakerMaker. AI co-founders Dhaval Adjodah and Owen He to unpack recursive self-improvement, or RSI, and separate the increasingly popular term from what it would actually mean for an AI system to improve itself. Dhaval and Owen distinguish true RSI from simply using AI to automate pieces of AI development. Their goal is a closed loop where the system being improved is also doing the improving, across components including models, harnesses, and eventually more of the AI stack. They explain why closing that loop still doesn’t guarantee runaway “takeoff,” and why measuring real progress requires accounting for both capability gains and the resources required to achieve them. They also share what they’re seeing in their own research, including early signs that self-improving systems can increase performance while improving efficiency. And they dig into the practical challenges of proving those gains are real, from variance and reward hacking to the enormous difficulty of distinguishing genuine improvement from noise.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  The fundamental idea here is to build AI that improves itself. And I think we're getting there.

• 0:05Speaker 01

  For the first time, we could swap the optimizer as the optimizer. And this leads to a reinvestment of cognitive improvement into itself.

• 0:15Speaker 00

  You can think of RSI at many different levels of the stack. Prompts, harness, weights, and you can go all the way to basically chips. Our goal here is to optimize all the levels of the stack.

• 0:27Speaker 01

  Our approach to RSI is that we have a multi-agent system working together. That's been recursively improved.

• 0:34Martin Casado?

  I'm so impressed by your belief that like this idea of a model improving itself.

• 0:38Speaker 00

  But it's not a belief.

• 0:39Martin Casado?

  That is a dramatic statement, which is, I think it's the first time I've heard you say it.

• 0:42Speaker 00

  Because we have been so thorough, we have not released things until we're super sure it's correct.

• 0:48Martin Casado?

  Do you envision a system where the amount of energy that goes into it somehow decreases over time relative to the improvement?

• 0:55Speaker 00

  We can't share too much, but one of the very important things that we've tried to do.

• 1:00Erik Torenberg?

  A16Z general partner Martín Casado sits down with MakerMaker.ai co-founders Dhaval Ajuda and Owen Heh to unpack recursive self-improvement, or RSI, and separate the increasingly popular term from what it would actually mean for an AI system to improve itself. Dhaval and Owen distinguish true RSI from simply using AI to automate pieces of AI development. Their goal is a closed loop where the system being improved is also doing the improving across components, including models, harnesses, and eventually more of the AI stack. They explain why closing that loop still doesn't guarantee runaway takeoff and why measuring real progress requires accounting for both capability gains and the resources required to achieve them.

• 1:43Erik Torenberg?

  They also share what they're seeing in their own research, including early signs that self-improving systems can increase performance while improving efficiency. And they dig into the practical challenges of proving those gains are real. From variance and reward hacking to the enormous difficulty of distinguishing genuine improvement from noise.

• 2:04Martin Casado?

  Welcome, both of you. We're going to be talking about everybody's favorite topic, which is RSI, recursive self-improvement. Maybe to start, it'd be great to just introduce yourselves and your backgrounds. So we'll talk a little bit about what you're working on now, and then we'll dig into the content.

• 2:17Speaker 00

  I think of myself as a physicist of some kind. I studied that, then I did PhD in reinforcement learning at MIT Media Lab. I really like building things and I'm very excited to be building a cool startup MakerMaker with Owen and Grace.

• 2:32Speaker 01

  My name is Owen. I was first studying math and then one day I realized that all mathematicians do is outproving theorems. And I was taking a class called Formal Language and Logic. And they were showing how you can build a theorem prover. And I was like, oh, that would be cool. I can just build a machine that proves theorems for me. That's how I got into AI.

• 2:51Speaker 00

  Amazing.

• 2:52Speaker 01

  And then for my PhD, I worked on continual learning and meta-learning. Then worked for a few years at DeepMind as a research scientist. Now I'm working on RSI.

• 3:02Martin Casado?

  I see a lot of pitches come in and a lot of them use the term RSI. And I'll be like, oh, so what do you do? And they'll say, we use AI to create AI. That sounds very interesting. So what do you do? We use AI to create GPU kernels, which that doesn't, to me, kind of match the definition of what I would consider to be RSI. So maybe can we start by you defining what you think or what you would include in RSI definitionally?

• 3:25Speaker 01

  Yeah, I think RSI is definitely an overloaded term today. Many people when they say RSI, they mean different things. So I think the important thing when you are differentiating them is to ask the question, what is the optimizer and what is the optimizer? When you're self-improving, who is improving, right? So there are people who are saying, okay, we're doing RSI. We have human using AI to develop better AI. But that to us is just autocatalysis. In a sense that the improver is still human with tools, even though these tools are getting better and better. But the ideas and the improvement come from human brains.

• 4:01Martin Casado?

  Right. I don't mean to interrupt, but to be clear, most technical epochs have had autocatalytic effects, right? There's no way you can create a computer chip without a computer, right? You could argue that the internet development was sped up by the fact that people that weren't geographically located could coordinate, right? I mean, the steam engine, you could build better steam engines because you could get coal faster, et cetera. So this notion of using the thing to build a thing faster is not new. And so you're saying something different.

• 4:29Speaker 01

  Yes. I think for us in the history, right, our civilization, we have this autocatalysis using computers to build better computers. But the idea for this change, for this improvement, come from human brain. But that human brain was never the improvey, never the optimizer. Now, for the first time, we could swap the optimizer as the optimizer. So the the brain of the agents, which is LM, the model, can be the thing that you're optimizing. And this leads to a compound, this reinvestment of cognitive improvement into itself.

• 5:05Martin Casado?

  But there has to be a reason that you decided to take on this hard problem as opposed to what anybody else would do, which is like to off pieces of the problem.

• 5:11Speaker 00

  Yeah. I mean, before I answer your more important question here, the reason I did this is I think it's a hard problem and it's fun working on the hardest problem. And I think.

• 5:21Martin Casado?

  But do you have an intuition that this is the actual right answer as opposed to like something that's more pragmatic?

• 5:26Speaker 00

  Yeah, I think a pragmatic thing would be just from a control theory perspective. If you have a gap in the system, you can have a closed loop. And autocatalytic has no closed loop, right? So whatever you improve, there's still like a gap. Humans have to come in, you have to pour in other resources to make it work. But if you have a closed loop, you still have to pour in resources. But the marginal gains ideally get either constant or larger over time. And that's the fundamental difference. I think what's really different here can be explained pretty easily. So if you think of capability, what we're trying to do here is we're trying to improve the rate of improvement of capability, right?

• 6:07Speaker 00

  So the first derivative of capability with respect to time is the thing that everybody is trying to improve, right? And now you can put that to be equal driven by two things, the denominator and the denominator. The denominator would be performance, which is what everybody is trying to improve. And then the denominator is some kind of cost, let's say capital, right? Or difficulty. And this one is pretty, it's pretty set because the higher the capability, the higher the difficulty of the next step, right? Now, the numerator is the interesting part. For things like steam engines and whatnot, that was fixed, right? So then you end up with a very simple differential equation where you basically plateau, right?

• 6:52Speaker 00

  Now, if you have a system where the improvey, the things being improved, is also the thing that's doing the improvement, you now have a new way to think about it. And the simplest case would be that the numerator is also correlated with capability, which makes sense, right? Because the better you are, the better you are at also improving. And if you do that, the math can start out and you basically get constant improvement over time. That could be one level of RSI, right? And then there's one more case, which is now that the numerator could be correlated with a higher power of capability. And then you get exponential takeoff.

• 7:31Martin Casado?

  Can we dig in a little bit more between RSI's having closed the loop and actual convergence properties? For some reason, when people see a graph where there's like a circle with an arrow to another circle and it goes back, they look at that closed control system and they say, take off. Right? Which, I mean, if this is a nonlinear control system, it's very difficult to know what the hell happens. By the way, most people don't seem to understand this distinction because we very quickly go to takeoff and boom and all that stuff when we talk about these things.

• 8:03Speaker 00

  Well, as a network guy, you know that it depends on the exponent of your function that's self-referencing. And it's, what we found is that it's almost not useful to speculate on what we think could happen because this is really an empirical problem. And the real way to know what's happening is to test things. And testing things is really, really hard. Like reward hacking is one of many problems that we have to deal with. But the question you're asking is very empirical and for some.

• 8:39Martin Casado?

  Oh, interesting. So but, so I'm a networker. That's how I would have answered. I would have thought math people would have been like, we've got this, we've got this differential equation. And I took out my pencil and I wrote down the answer.

• 8:51Speaker 01

  I mean, it is possible to write down the differential equation, but you still have to empirically measure the variable, the exponent.

• 8:57Martin Casado?

  So there's no analytical solution.

• 8:59Speaker 00

  I mean, honestly, we don't even have a learning theory of deep learning. We don't know fundamentally why it works, right? And then now you're like five, six layers on top of deep learning. And so we don't have a good theory. I think the closest things we have are scaling laws, and these are empirical laws.

• 9:17Speaker 01

  I think one misunderstanding people have about our approach to RSI is that they think the entire system has to be fully automatic and digital. But actually, if you look at our mathematical foundation, we have a multi-agent system, not just one digital agent. And in this multi-agent system, human researchers and engineers are actually special case of these agents. And it's this hybrid system of multiple agents working together that's been recursively improved. And we don't want to leave out any component to not optimizing them. Because if it's possible to optimize the model, why wouldn't we do that? Because you will have more gains if you do include those. Anything that you don't optimize will later become your bottleneck.

• 9:59Martin Casado?

  So one observation that we have made here is there are claims of speed is improving, which is actually the case, right? So if you look at like the model release time that it is reducing. And so on one hand, you can be like, okay, so clearly we're in some sort of a rapidly improving phase. But if you divide by the actual amount of capital raised by these companies, that's no longer the case, right? And so you're like, well, maybe they just raise a whole bunch of money and then like AI allows you to apply that money. So how would you, this is not for yourself, but how would you encourage people that are either evaluating claims about this or even working on them to think about not fooling yourself by throwing more money at the problem?

• 10:43Martin Casado?

  Like you're actually showing that you're making an improvement here that's meaningful with the denominator.

• 10:49Speaker 00

  For us, it's it's pretty straightforward. It's because if you get improvement and they just come at more and more resources being poured in, it's not RSI. You have to be able to back your claim of how much improvement you're getting.

• 11:05Martin Casado?

  But do include like all the man hours and experimentation that went into that? Because I I feel like at some level, like that, like it's like, like they'll talk about like the actual flops that go in and money that go in. But the reality is this is like the twenty third experiment and it's actually a human burning a lot of mental capital to get to that final configuration. Like, do you actually include that in the denominator?

• 11:28Speaker 00

  We do, and I think it's it's good company discipline to do that.

• 11:32Martin Casado?

  Yeah, yeah.

• 11:33Speaker 00

  And we've tried again to very intentionally from the beginning, have a culture of tracking where things go, cost, time, all that kind of stuff. I share funny thing internally, we have this thing, people talk about leaderboards, you know, which is like score. You know what I'm about to say? We have something called a bleeder board, where we keep track of the cost of of experiments of everybody who's running experiments.

• 12:01Martin Casado?

  That's awesome.

• 12:02Speaker 00

  You know, and and everybody, it's the company, everybody any day can look at like who's who's high up on the on the bleeder board.

• 12:11Speaker 01

  I can also give an example from AI. I don't know if you know this NanoGPT benchmark you are running. It's basically asking you to retrain GPT two, which when it just came out, it's a very expensive model to train and takes weeks, I think, on multiple GPUs. And right now, people can train it on single GPU within hours, and that number has been shrinking constantly.

• 12:34Martin Casado?

  Yeah. But that's also totally separate, right? Like, that's just like whatever Moore's law, like.

• 12:40Speaker 00

  No, no, it's the same GPU. It's the same hardware.

• 12:43Speaker 01

  Using the same hardware and then you improve the techniques of pre-training.

• 12:47Martin Casado?

  Oh, interesting.

• 12:48Speaker 00

  What has improved is like algorithmic techniques, data techniques, kernels, all of that kind of stuff.

• 12:54Martin Casado?

  Do you know how much it costs to train GPT two?

• 12:57Speaker 00

  I think it's a few hundred dollars.

• 12:58Martin Casado?

  A hundred dollars, I just, oh, wow, so the same same GPU.

• 13:02Speaker 01

  Same GPU, same data.

• 13:03Martin Casado?

  And this is just algorithmic, and did those come from humans or did those come from?

• 13:07Speaker 01

  Agents, a lot of them are auto research results.

• 13:10Martin Casado?

  But that but that's definitely the denominator. That's that's not the numerator.

• 13:13Speaker 00

  Correct, yeah, correct.

• 13:14Martin Casado?

  What you want to see is like, you can do that with GPT five, which we're not seeing, right?

• 13:18Speaker 00

  Well, I think.

• 13:19Martin Casado?

  The the cost of these things is going up dramatically.

• 13:22Speaker 00

  I think one of the most closely guarded secrets in the industry is exactly this thing, which is how much compute you can get out of this.

• 13:31Martin Casado?

  Yeah.

• 13:32Speaker 00

  Because in some sense, even if all you were improving was your denominator, you could get faster training because now you can get more rollouts.

• 13:40Martin Casado?

  Right, yeah, of course.

• 13:40Speaker 00

  Right?

• 13:41Martin Casado?

  Yeah, yeah.

• 13:41Speaker 00

  And that means now you can sample that, what what you're describing, like the manifold of the space of of language more extensively.

• 13:50Martin Casado?

  Sure.

• 13:50Speaker 00

  And then you can get better trajectories, which means you can get smarter models when you use that to train.

• 13:55Martin Casado?

  For sure, improving the denominator is good.

• 13:57Speaker 00

  Yeah.

• 13:58Martin Casado?

  For sure, improving the numerator is good.

• 14:00Speaker 00

  Yeah.

• 14:00Martin Casado?

  I've yet to see, which is great that you have, you know, glimmers, uh, where you can improve both the numerator and the denominator. I mean, that's a pretty meaningful statement, which is awesome.

• 14:09Speaker 00

  What we found is that a lot of the papers that claim to do RSI, when we tested in our internal infra, we find that they don't work. And a lot of it is because they are cool ideas that humans try to introspect and say, oh, you know, this is what I think should happen.

• 14:27Martin Casado?

  Yeah.

• 14:27Speaker 00

  They're complicated, they sound cool, they get you a good paper. But then when we test it at scale, you know, on benchmarks, on real economic problems, they don't scale. And what we found is that the the simpler thing that closes the loop is the thing that scales. I would say one thing that we do that's probably unique is that we always start with a very grounded mathematical framework that that basically lays out the map of the whole space.

• 14:53Martin Casado?

  Yeah.

• 14:54Speaker 00

  And the reason is because this space is so large that if you don't do that and you don't have this very principled way of doing it, you get lost very quickly in this complexity. And the other reason is that often when you have this really complex system, there's two extremes of how you can build it. You can take an approach where you basically build like two hundred different systems. And you hope that like this, this, this, you know, what is it, the Rube Goldberg machine will kind of work out?

• 15:25Martin Casado?

  Yes, yes.

• 15:26Speaker 00

  Or you take the other way and you're you kind of basically spend some time thinking about these two hundred things. And maybe you realize, you know, they're actually just twelve um abstractions that generalize to these two hundred things. And so what we have tried um to to create a culture of, let's really think from first principles. Let's, you know, look at the math and then we realize what are the assumptions we make to be practical, because you have to do both. And then we see what works. And and when you do that, you then can see that, oh, you know, there's like multiple papers or ideas are actually, you know, subclasses of the same general framework.

• 16:05Martin Casado?

  Yeah.

• 16:06Speaker 00

  And if you do that well, you can scale. And I think um that's that's one thing that I see that's missing. And that's hard to do and and takes a very unusual group of people to to really like make that work.

• 16:21Speaker 01

  I think Richard Feynman said once that there are two kind of sciences, there there's a stamp collections and then there are physics. So without a unified math framework, all these research are sort of just stamp collections. You could, like, a lot of these papers we're reading, they're just ideas inspired by some human behavior. And then they try to mimic that and say, oh, look, we have RSI. But then if you don't put into a unified system, you don't know if the the overall system is actually improving or you're you're just like locally applying an idea that that's getting better. So what we have is that we're putting all these components that you could possibly recursively improve into a unified framework

• 17:02Speaker 01

  and look at it from holistic view, so that improving one part um does not negatively impact the other part, right?

• 17:10Martin Casado?

  I see, I see.

• 17:11Speaker 01

  So so for example, there are people who are doing recursive improvement just on harness and then there are people who are only doing on weights. But weights and harnesses are not isolated component, they have impact on each other.

• 17:22Speaker 00

  You you can think of RSI at many different levels of the stack, right? You can think of um prompts, um harness, weights, and you can go all the way to basically chips, right?

• 17:34Martin Casado?

  Sure.

• 17:35Speaker 00

  And our our goal here is to optimize all the levels of the stack, right?

• 17:44Speaker 00

  Efficient because we care so much about the denominator. And then we started doing weights, weight updates. And what we found is that you get more gains, but you still have to do harness stuff, right? And you can keep going down the stack. And and my guess is that you probably will need to do a lot of the kind of co-optimization of many different parts of the system to make it work.

• 18:09Martin Casado?

  Can you talk a little bit about the problem domain that you're going after just to prove that this works?

• 18:14Speaker 00

  Um, we are trying very hard to pick areas of the AI space that are as close as possible to AI itself. Again, we try to be recursive.

• 18:27Martin Casado?

  Yeah, yeah, yeah.

• 18:28Speaker 00

  And if you think about it, there's there's not that many problems that are super close to AI and gives you improvement on both the numerator and denominator. And one of those is the inference problem, right? Because the more you can spend on, um, sorry, the more you can get improvements and and efficiency out of inference, the the more everything kind of unlocks, right?

• 18:54Martin Casado?

  Yeah.

• 18:54Speaker 00

  And it's not just um for technical reasons, but also the most promise of both the numerator and denominator being improved, right?

• 19:04Martin Casado?

  It kind of breaks the mind, right? So like, just just for everybody that's listening to this, there's two improvements going on here. And one of them is the ability of the model to make inference faster. And the second one is now that the inference is faster, it can do that faster, right? And again, that does not necessarily mean takeoff, that can still, you know, asymptote at some point.

• 19:26Speaker 00

  Yeah.

• 19:27Martin Casado?

  But it's just wild to think of this kind of compounding impact.

• 19:30Speaker 00

  Yeah, yeah, we're startup, we're not a research lab. And one of the reasons we chose it is because everybody in the world right now wants to serve their own model.

• 19:40Martin Casado?

  Yeah.

• 19:40Speaker 00

  And everybody wants to use the latest, for example, open weight model, Kimi K three or whatever, just coming out. And they want to be able to serve it as efficiently as possible, right? So it's it's an economically extremely important problem right now.

• 19:57Martin Casado?

  Just for the sake of discussion, if I if I wanted to make the inference for Kimi K three faster, would I use Kimi K three in a recursive way or would I use a separate model and system to do that for Kimi K three?

• 20:12Speaker 00

  To improve the serving stack of Kimi K three, you mean?

• 20:15Martin Casado?

  Yeah.

• 20:15Speaker 00

  In our, well, our system um is not just using one LLM. You have many different LLMs that are doing many different things.

• 20:26Martin Casado?

  Oh, so you're doing, are you doing the inference set for all of them?

• 20:30Speaker 00

  Um, I mean, we, again, it's it we pick whichever is is the most effective and because it's being used the most time. But the interesting thing is

• 20:40Martin Casado?

  I see.

• 20:40Speaker 00

  the choice of the of which LLM to use for which thing is also not always up to us. We let the model choose.

• 20:47Speaker 01

  To be honest, today, it's not really us who are deciding what to optimize, it's our agent.

• 20:53Martin Casado?

  Interesting.

• 20:53Speaker 01

  Because it's recursively self-improving. And what we're trying to do is not to limit itself to only one component.

• 20:59Martin Casado?

  So it chooses whether to.

• 21:00Speaker 01

  It chooses whether to.

• 21:01Speaker 00

  Yeah, we try to be like as hands-off as possible, let it do its thing.

• 21:05Martin Casado?

  Do you have like a single master prompt? Is it like, improve yourself?

• 21:10Speaker 01

  But even that master prompt has been improving.

• 21:13Martin Casado?

  Really, it improves the master?

• 21:16Speaker 00

  Yeah, and I mean, and it's not one RSI system because you can think of it as there's the thing that that does the improvement. But then you also have to have the thing that knows what to improve. And that's both are dynamic problems and they are very co-co-linked, right?

• 21:35Martin Casado?

  How general do you think this ends up being? Like, right now, you're focused on speeding up inference in a specific domain. Do you think that whatever approaches you're coming up with here generalize? Or do you think there'll be a different one per?

• 21:53Martin Casado?

  Target solution space.

• 21:55Speaker 00

  Mathematically, we have defined it so that it's one system that generalizes to everything. I think the one part of the system.

• 22:03Martin Casado?

  And by everything, you mean any problem domain that you can point AI to?

• 22:07Speaker 00

  Correct, but then there are physical limits, for example. And one big limit of this is the human element. I think the way that a lot of AI companies are thinking is, you know, let us use AI to replace humans. But what we found is that there's so much that if you have good researchers and good engineers that they can bring to the table, that's gonna be very hard to replace by agents. And so that's why, from the very beginning, we've tried to think of our basically entire company as a multi-agent system, where humans are a kind of agent that's cohabitating with other AI agents.

• 22:49Martin Casado?

  Can we at least reduce bounds? So for example, do you envision, do you envision a system where the amount of energy that goes into it somehow decreases over time relative to the improvement?

• 23:01Speaker 00

  We can't share too much, but one of the very important things that we've tried to do in our company is not just work on the numerator, which is what I was describing earlier, which is your performance, but to work on the denominator, which is how much you can get out of the same amount of physics, money, whatever you want to call it.

• 23:21Martin Casado?

  Right, but do you think it's possible to have a system where the numerator continues to increase at the same way with the denominator being fixed or decreasing?

• 23:31Speaker 00

  It's not just possible, I think it's happening.

• 23:34Martin Casado?

  Really?

• 23:35Speaker 00

  Yeah.

• 23:36Martin Casado?

  So it takes less energy to get better results?

• 23:40Speaker 00

  Yeah, over time.

• 23:41Martin Casado?

  Do you think there's a, there's a universe in which, like, that continues without asymptoting? Because if that was the case, then you actually have takeoff.

• 23:51Speaker 00

  I mean.

• 23:51Martin Casado?

  But it just seems to disobey information theory and laws of physics.

• 23:55Speaker 00

  Yeah, yeah, I mean, I think at some point you're gonna reach Planck constant.

• 23:59Martin Casado?

  Yeah.

• 24:00Speaker 00

  You know, Planck length, Planck energy, Planck time, right? So it will asymptote at some point, but I think.

• 24:07Martin Casado?

  Oh, I see. And so you think that asymptote is basically like the computation of the universe?

• 24:10Speaker 00

  At least, yeah, I mean, you know, how many, how many particles are there? 10 to the 80, like, we know, speed of light, you know? We know, we know what it is. You multiply those two and you basically get an upper bound of computation. And, and so given this fixed amount of computation you can do every second, we know we have a limit.

• 24:27Martin Casado?

  I'm so impressed by your, your belief that like this idea of a model improving itself.

• 24:32Speaker 00

  But it's not a belief.

• 24:33Martin Casado?

  No, you got to correct that, that's amazing.

• 24:35Speaker 01

  I mean, also, it's a question of what metric you use to, to view the progress, right? So if you use, I don't know, benchmark accuracy, it's upper bounded by 100%. You're never going to see exponential improvement on that because eventually bottlenecked 100%. So for a long time, we had this question, how do we even know if something has ticked off, right, if we don't have the right metric?

• 24:56Speaker 00

  It's not a belief that this is happening. I think we're already seeing glimmers of that happening.

• 25:01Martin Casado?

  But it is a dramatic statement, which is, I think it's the first time I've heard you say it. So wait, is this like, is this been kind of, you know, a continuous result? Or is it relatively recent that you're starting to see glimmers of this?

• 25:13Speaker 00

  No, we've been seeing like, in, in these results kind of grow and grow and grow over the last few months.

• 25:19Martin Casado?

  That's amazing. How do you have confidence in the results given, you know, like, it's not working yet and you don't know?

• 25:26Speaker 00

  No, it's not a belief, it's empirical measurement. But, but also as a, as a company, there's, there's a very funny thing that, that is happening, which is because we have been so thorough, we have not released things until we're super sure it's correct, right? And every few months, you know, there's like an announcement of something RSI related, we look at it, we're like, we should have released our results or something. And then we look at it, we try to replicate it, and we realize, oh, it's noise, you know, it's cherry picking. And so it's very, very hard to, to know in this, in this sea of, you know, AI, I don't know what you want to call it, like, everybody's trying to do our AI RSI stuff to know what's legit or not.

• 26:10Speaker 00

  The, the technical answer is you have to be really, really good at measuring things. So for example, we have one of our engineers that we, we, we named him the viceroy of variance.

• 26:23Speaker 00

  That's a real thing. Like, his job, his job is to go and like destroy the results we have and find what's wrong. And we even have little like Slack emojis that we have. There's one where like, there's like red laser stuff coming out of his eyes when like, he finds that there's bad variance and stuff. One of the things that we found is that, you know, when, when you're dealing with a new benchmark or a new problem of economic value that you want to apply this to, so far, we have not found a replacement for what we call quality time. Literally, running hundreds of experiments, looking at it, developing an intuition, trying to figure out what is wrong, what is not wrong? What is, what is just variance, what is not?

• 27:05Speaker 00

  What is this weird thing that actually shows that there's, you know, some kind of reward hacking, like two levels down, the agent that built the agent that built the agent did hack something and now you have this weird results?

• 27:17Martin Casado?

  Right.

• 27:17Speaker 00

  It's not easy.

• 27:18Martin Casado?

  So when I have dealt with, you know, nonlinear control systems, there's a bunch of, in networking, there's a bunch of degenerate behavior you have to deal with. It's not, it's not like just convergence and divergence, you have things like oscillations. And this is a very multi-dimensional problem. So do you have to deal with all of that stuff, like potential oscillation?

• 27:42Speaker 00

  And then, and then on top of that, you have noise.

• 27:44Martin Casado?

  Yeah.

• 27:45Speaker 00

  At a lens of fundamentally super stochastic.

• 27:47Martin Casado?

  Yeah.

• 27:47Speaker 00

  So what we find is that specific agent on a specific benchmark, you know, there's levels of noise that sometimes are unpredictable. And some of them are extremely like weird, like you even have some weird noise that comes from like this GPU and this other GPU, they're both, you know, B200, but they have slightly different behaviors, like on the physics, and you can see some of that being replicated at the high level. So that's why it has to be an empirical thing.

• 28:18Speaker 01

  So from that, you could already see that intelligence is not a single scalar. It's more like a multi-dimensional vector. And there will be domains and field where this RSI will give you takeoff. And there will be dimensions where you might plateau or you will have relatively slow improvement. And I think the, the, the defining factor is what is the slowest component in it? So you're an optimization person, you know that there's this AMDOS law where, you know, like the component, the, the speed up you can get by optimizing a component is.

• 28:56Martin Casado?

  You just take it, you just take it to zero, yeah.

• 28:58Speaker 01

  Yeah.

• 28:58Martin Casado?

  The thing else dominates.

• 28:59Speaker 01

  Exactly. So, so the digital part, we could optimize and accelerate, you could get like a thousand times faster or a million times faster. But if it's only 20% of your overall time, you're only going to get 20% speed up, right? So, so for, for domains where you still need to move atoms that interact with physical world that's really slow, you're still going to be bottlenecked by that. So let's say every eval, every evaluation you need to do, let's say drug discovery, you still need to do wet lab experiments and then clinical trials and that take months. Then every new idea have to go through a month or multiple months evaluation. So those fields probably won't be accelerated as fast as, you know, coding or mathematics, where you have very fast and cheap evaluations.

• 29:44Martin Casado?

  What I've, I've yet to see is anybody put a balance on that. Like to say, in order for me to, you know, improve this by X to solve this problem, you know, I need this much net new energy to do it, right? And if you have that, you could actually predict, like, if I'm, like, listen, like, how big of a warehouse do I need to solve, like a drug discovery problem? Or how many different robots I would need to, to solve? So do you think that we're getting any closer to actually having bounds? Because I think this is such a critical question for all of us to answer.

• 30:17Speaker 00

  One of, one of the internal jokes we have in the company is that the, the secret to RSI is somewhere in a garden hose company in Wisconsin. And, and the joke here is that if Einstein hadn't discovered relativity, he would not have these, these laws that tell you how speed and, you know, contraction of time works.

• 30:40Martin Casado?

  Yeah.

• 30:41Speaker 00

  And if some unfortunate engineer had to design the GPS system, they would have had to find a, find like a correction of time. And they probably would have said like, no, no, it's clearly not the fabric of spacetime that's contracting, right? But I don't know why, but this is the formula I've devised empirically.

• 30:59Martin Casado?

  Some, you know, like some fudge factor.

• 31:01Speaker 00

  Yeah, exactly, right? And then, and then people later would have been like, holy shit, this is, this is like spacetime contracting, right? And, and we have this theory that, you know, like RSI right now is, is being thought of, let's, let's solve all these like super hard kind of fashionable problem. But what we think is that the complexity of the world is such that the, the real challenges probably exist in spaces that are not being applied to, right? And so that's the joke, that there is some workflow somewhere in a garden hose company in Wisconsin, that if we solve it, we get, we get, you know, boom.

• 31:43Martin Casado?

  I love it. I just want to quickly summarize. So you're going after RSI, and in your case, this is not auto catalytic effects, this is actual RSI, where the system improves the full system. And for you, it's an actual system, which includes the harness and the weights. If you think about the problem is you've got a numerator and denominator, the numerator is like how good the system gets and the denominator is how much energy goes into it. You believe you're seeing glimmers of the sign where both improve at the same time using this type of approach, is that correct?

• 32:10Speaker 00

  Yeah.

• 32:11Martin Casado?

  When can we expect to hear more from you?

• 32:15Speaker 00

  Very soon.

• 32:16Martin Casado?

  Amazing.

• 32:16Speaker 00

  Yeah.

• 32:17Martin Casado?

  Thanks so much, both of you, for joining us.

• 32:18Speaker 00

  Thank you.

• 32:21Erik Torenberg?

  Thanks for listening to this episode of the A16Z Podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at A16Z.substack.com. Thanks again for listening, and I'll see you in the next episode.

• 32:47Erik Torenberg?

  As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com/disclosures.