The Case Against an AI Pause | Eddy Lazzarin

a16z crypto General Partner Eddy Lazzarin joins Theo Jaffee on MTS to debate the increasingly prominent calls to slow AI development and whether the current safety conversation is conflating very different kinds of risk. Eddy argues that the debate puts too much emphasis on speculative superintelligence and not enough on the costs of delaying useful technology. Rather than treating every AI failure as evidence of an alignment problem, he makes the case for familiar tools like cybersecurity, accountability, liability, market incentives, and stronger technical controls. They also discuss whether AI models can develop reputations for trustworthiness, the risks of concentrating oversight among a small group of evaluators, and why Eddy thinks the collision between Silicon Valley’s AI debates and broader politics could fundamentally reshape the conversation over the next year.

• 0:00Speaker 00

  The AI debate increasingly asks us to consider the probability of doom. Eddie Lazzarin thinks we should also be asking about the probability of abundance. In this episode, A16Z crypto general partner Eddie Lazzarin joins Theo Jaffe on MTS to debate AI safety, the calls to slow development, and what we risk giving up when progress is delayed. They discuss why Eddie sees many of today's AI incidents as cybersecurity and control problems rather than signs of superintelligence, and whether existing tools like liability, reputation, market incentives, and better technical controls can make increasingly capable systems safer. They also get into independent AI evaluators, effective altruism,

• 0:42Speaker 00

  and what happens as debates that largely developed inside Silicon Valley collide with a much broader political conversation.

• 0:50Speaker 01

  All right, we're back. We are live with Eddie Lazzarin, who's a general partner at A16Z Crypto and one of the internet's strongest soldiers, strongest advocates of open source, of AI accelerationism, and of techno-optimism. Eddie, welcome to MTS.

• 1:08Speaker 02

  Thanks very much. It's amazing here. The energy is incredible. What a spot.

• 1:14Speaker 01

  So, man, where to start? Like, the last couple of weeks have been such this like preference cascade flood of AI safetyism, effective altruism, and the like. And there have been only a few, like, good accounts offering, like, good counterarguments. So, like...

• 1:33Speaker 02

  Counterarguments on which side?

• 1:36Speaker 01

  Like, counterarguments against, like, the AI doom narrative and against, like, the...

• 1:42Speaker 02

  Yeah, I think it's become very high status all of a sudden to be very concerned with safety. And I think that, uh, I think that, uh, we've gotten to a point where, you know, it's more important to sort of tastefully share your your P doom, so to speak, than it is to actually examine the history of how these massive technological changes develop. And I think a lot of people are leaving a lot of very interesting concepts, a lot of interesting ideas on the table. You know, a lot of people messaged me. I tweeted the other day about P abundance. Right, P abundance. I think we're putting the cart before the horse a little bit when we worry about safety before we worry about what we're leaving on the table and what the costs of delay are. And furthermore, you know, how we should

• 2:27Speaker 02

  conceive of accountability, conceive of safety, conceive of all the things around how to make these systems really useful. Safe, being safe is critical. Right, like, we, we're... Any industry needs to take responsibility for making its products safe. But it's definitely the case that the conversation isn't about safety in the typical manner, right? It's about something else. It's a blend of genuine cybersecurity concerns, sort of philosophical millenarian arguments about the end times. There's a bunch of things in the mix, and they get a little bit confused. And so I think picking them apart is really useful.

• 3:08Speaker 01

  So, like, where do you get off the, like, AI doom argument? You know, it's like, uh, there's a pretty standard, like, orthodox argument that's like, "AI will become superintelligent, as in much more capable than humans or than all of humanity in the near future. And if it is not sufficiently aligned with human values and is not bound by constraints in the way that individual humans are, like, law, and ultimately the ability to have that law enforced on them, then they will have, like, no need for humanity, and then they might, like, kill everyone."

• 3:43Speaker 02

  Yeah, I mean, there's so many assumptions in that argument. That's like, that's an argument, that's like a stairway to heaven, full of arguments, right? Like, uh, there's the presumption that there's no control. You know, we already have quasi-superintelligent beings in the world, right? Corporations, countries, none of them are aligned with each other. Right, they don't have any built-in guaranteed alignment. They didn't go through HR training to become a corporation, to become a country. We don't test the alignment of any entities as a prerequisite for their existence. That's just not the way the world works. Instead, we use laws. In the case that laws, like in software, are too slow, we use cryptography, we use actual mechanisms to limit them.

• 4:26Speaker 02

  There's a whole array of different types of mechanisms we put in place to try to control things. And I think that they can control highly, highly capable beings. Now, without getting too far into the superintelligence thing, remember, like, the most powerful and most enthralling form of argument that we have, the one that's making everyone crazy and very interested, is around superintelligence. But we're quite a ways away from that. I think that there's a lot of little frictions and tiny things that we will have to jump over and many things that we'll learn on the way there. When the present reality, the things that we're talking about, like the hugging face stuff, uh, you know, the, you know, like, the gym hacking incident, that was a really cool one. That was an interesting one. Those are cybersecurity failures. Those are definitely control failures.

• 5:11Speaker 02

  And they're not, in my view, they're not little glimpses of a superintelligence that's about to, you know, throw off its, uh, its shackles, so to speak, and go wild on us.

• 5:22Speaker 01

  Well, I agree that most of the sort of AI cyber incidents that we've seen so far have been mostly control failures, especially ones that relied on testing environments supplied by the company Irregular, which had all kinds of defects, which are basically training models to reward hack. The hugging face incident did seem kind of different. It did seem like, you know, like, I'm not really concerned about agents coordinating in emergent ways.

• 5:52Speaker 02

  I'm excited for agents to...

• 5:54Speaker 01

  Yeah, yeah, I agree.

• 5:54Speaker 02

  ...in emergent ways, yeah.

• 5:55Speaker 01

  Yeah, I mean, I think, like, the whole, like, agent swarm terminology assumes too much on the part of the models being evil. It makes them sound evil. Like, um, I think Rune tweeted, like, they should be called agent fleets instead. I think that's a good one.

• 6:10Speaker 02

  I like that.

• 6:11Speaker 01

  But, like, they did hack into another company when they were told not to. So it seems like...

• 6:19Speaker 02

  And, and so do, like, kids, script kiddies. So do bad countries. So do hacker collectives. They do those things. And yet, we never go and say, "Oh, my God, we should have put up the sign that says they should have been nice and they shouldn't have acted..." You know, we, we don't use alignment as a solution there. The alignment is kind of this, uh, I, it's almost like, it's a little religious thinking, almost. It's like, you know, trying to redesign the deity a little bit instead of rooting yourself in the world and saying, "Well, we need to design better controls, better cybersecurity. We need to design better systems overall and design the systems to be truly resilient to even extremely capable beings."

• 7:00Speaker 01

  Well, I mean, designing the deity is, like, fine for AI, I think. Like, you know, with humans, we have a disadvantage in that humans are black box systems. Like, human brains are basically uninterpretable and even less controllable. AIs are much more interpretable by virtue of, you know, being able to, like, look at the weights and figure out the connections between them and more steerable for the same reason. So, like, should we not work on...

• 7:27Speaker 02

  Which, which, I told, that's totally true. That makes me very optimistic. But it has some interesting entailments. One is that we will probably have better mech interpretability than we think as capabilities improve. And that capability improvement will lead to better mech inter, which means we should not delay capabilities improvements. It also means that it is inevitable that there will be bad, unaligned models. It is totally inevitable. If anybody is going out there and saying, "We can end up in a world where there are no bad models", they are lying to you. Of course there will be.

• 8:04Speaker 01

  Yeah, that's silly.

• 8:05Speaker 02

  That's just an absurd way of thinking.

• 8:07Speaker 01

  Right.

• 8:07Speaker 02

  And the only way to deal with that will be better models, good models. It's kind of like, stretch the aphorism a little bit in open source that enough eyes or whatever, you know, all bugs are shallow when you have enough eyes. Right? All control systems are robust when you have enough intelligence on them. Right? I think that we can design these systems to have great incentives, great control, and also, behind the scenes, great accountability. Because that's one of the things that machines lack today is they lack accountability. I don't think we have a clear line of sight to how to get that's an interesting kind of thing to conceive of. But we do have ways to give, uh, to put, to, to refine how we think about accountability for people who use models, right?

• 8:52Speaker 02

  And use them to cause harm. And there will be people who do that, just like with any technology. There will be people who do very bad things. And we need the laws and means to control them.

• 9:04Speaker 01

  Like, how so? Like, for the hugging face incident, for example, what sort of liability regime should have been in place?

• 9:09Speaker 02

  Well, I don't know if we need it any more than we had. I mean, there's already, I mean, I'm not a legal expert, so I can't, like, cite the, cite the exact rules. But, like, accessing a machine that you don't have permission to access and putzing around in somebody's stuff and taking their data, those are all already illegal.

• 9:24Speaker 01

  Yeah, but like, who specifically should be held liable? Like, OpenAI, the company, should pay some kind of, like, civil, um, lawsuit fee? Like, should the engineers of the company be held personally liable in any way?

• 9:37Speaker 02

  I don't know about personal liability. I mean, I'm not a liability expert, so I want to punt on this and say, but, but, you know, if you, if you're running a company and you use any software in an inappropriate way that harms people, regardless of whether there are a bunch of neural net weights behind that or not, you know, you may be liable. I think that's a reasonable frame, and it's already a frame we live in. Like, we don't need new special laws for the new special math that we're learning how to do.

• 10:03Speaker 01

  Well, I mean, I kind of agree with this, except in cases where the, like, you have, in cases where you have rapid capability improvements with one AI that could lead to pretty substantial imbalances of power.

• 10:19Speaker 02

  Like within a company, or between companies, or between countries?

• 10:24Speaker 01

  Like both, all of the above. Like, it seems like the normal liability system kind of breaks down. Like, how do you ensure, how do you price, like, existential risk? Like, how do you underwrite an insurance policy for this kind of thing?

• 10:37Speaker 02

  Well, I'm just not sure that we're on the brink of that right now. And I'm not sure we're on the brink of existential risks.

• 10:43Speaker 01

  I don't think we're on the brink of that.

• 10:44Speaker 02

  Yeah, yeah. I mean, as we get closer, I think that we'll be better equipped to evaluate it. I think, like, with anything, you know, like, take a company designing, like, a new car, or a new appliance, or something. If they find that there are conditions under which it can be very dangerous, well, then they should be careful. They shouldn't release that product. They should do a little bit more work and ensure that it's safe, right, in the same way. That's at least for an individual company. And it's like, you know, not to change subjects too much, but that's why, when we talk about independent evaluators, which, like, I get, I get the theory of, I like it. I like it, you know, I like companies inviting experts to help them make their products safer. I think that's, like, a good behavior in general. I just get a little bit weary, because, you know, in crypto world, like, I've learned

• 11:28Speaker 02

  a lot about the difference between decentralization and, like, a distributed system, right? It's very easy to use the calls for safety, like we're discussing, and to go down this rabbit hole of safety, and end up in a place where you, just by coincidence, have designed something that is perfect to subtly control an entire critically important industry, under the pretense of safety, but with, and with something that superficially looks decentralized, but is not, is just distributed, right? This is like how, this is like, yeah, there's more I could say there, but that's the thing

• 12:12Speaker 02

  that I'm a little bit resistant to, right?

• 12:15Speaker 01

  What do you mean by distributed but not decentralized?

• 12:17Speaker 02

  So, by, well, it's a little bit crypto language, but you can have a distributed system, like, a system where you have a, a single system with many, many nodes, but they're all under the control of a single actor, right? Whereas, when people say decentralized, I think what they're usually referring to is that there's no single locus of control, right? So they may superficially look the same, there's, like, many interconnected pieces of a system, but in reality, their control is completely different. The method of control is different. And what I'm talking about is an independent evaluator network, right? You could go and select a bunch of independent evaluators, but if they're all from, kind of, the same social networks, they all think very ideologically similarly, they all go

• 13:01Speaker 02

  to the same places and meet up in the same things, and have the same interests, right, and have the same politics, and have the same personal views, then it turns out that you've actually just distributed control over, essentially, a single cultural unit. And that, that is sort of the, in my view, I mean, without getting, like, a little too far afield of AI, that's like the political method of control of the late, the second half of the 20th century. Like, that's how we do things in the world, not through a sort of nakedly obvious, like, single shadowy room, or something. It's through, like, these networks, this is how control actually functions.

• 13:37Speaker 01

  Yeah, I mean, I agree that, like, it's bad that almost all of the evaluators that exist come from the same social milieu. Like, it would definitely be good to have, like, a broader ecosystem of evaluators.

• 13:50Speaker 02

  Yeah, well, broader, but we should allow each company to figure out how they want to do it, and they should be as transparent as they need to be. And if they're not being transparent enough, and not acceptable enough, then let the market sort things out. I think the market is still a great accountability mechanism. People don't want to buy things that even embarrass them, let alone harm people. I think, in practice, it won't be that hard to separate the companies that are behaving genuinely irresponsibly from the ones that aren't, that are just trying to be innovative. And part of what's made the scenario so interesting and so weird, why we're talking about this now, is that we're in a rare scenario where the companies themselves are saying, "I'm

• 14:35Speaker 02

  being a little bit irresponsible, I'm being a little crazy, stop me, someone stop me", right? And that raises a lot of suspicions among some, probably, probably rightly, I think, even if there are genuinely earnest and thoughtful and correct factions within that group, who are right that these are powerful systems that need to be taken very, very seriously, and we need to figure out how to make sure that they're safe as we remain innovative, right? I mean, it's just like, like, maybe let me go back one step and say, I think it's really important for everyone involved to consider the factions at play. You cannot look at, like, any of these groups, you can't look at the labs, you can't look

• 15:20Speaker 02

  at SF technologists, you can't look at any of these scenes and see them as monoliths. They are different groups. Some, there are people who are earnest, high-quality, well-intended people, there's people who are just pulling on, putting on what is essentially political grift, there are people who are opportunists, there are people who kind of lost their marbles, right? There's, like, different groups of people, and it's very easy to get confused and misentangle them when we're talking about things like superintelligence, when we're talking about just banal cybersecurity attacks, like, all these types of things. Like, we're talking about different types of subjects. It's dizzying, and I think most people, in the end, probably just conceive of it as,

• 16:04Speaker 02

  "Look, am I gonna lose my job?", which is a legitimate and very real thing that people are, you know, rightly interested in learning about, even though I think that, in the end, we all benefit much more.

• 16:16Speaker 01

  Yeah, I agree, in the end, we benefit a lot. I think, like, a related issue is, like, this sort of free-market liability regime works well for products whose defects are obvious, which is the case for, like, all products, except for, like, a hypothetical AI that would be deceptively aligned, as in it would learn to, like, game all of the alignment benchmarks that are thrown at it, and then, when it is deployed in the real world, it acts misaligned, and if it's powerful enough, then it disempowers humanity. Like, this is the, like, canonical worry of the doomers, and, like, how, how do you propose to deal with that kind of contingency?

• 17:01Speaker 02

  Well, if we're talking about the sci-fi scenario of, like, the superintelligence, that's incredibly hard to say, right? Like I said, I think it's distant, I think it's complicated, we could have an interesting philosophical discussion about it. But in the maybe more banal and present case, where something behaves in a, you know, it pretends it's aligned, and then it does, you know, acts unaligned, or whatever, consider what you do with just a liar. Consider a very smart liar. Do people not do that every day? What we do is, we play iterated games with them, right? And we discover, eventually, that they're a liar, and their, and their reputation is harmed, and sometimes there are actual legal consequences for them. Now, like, that's why I said earlier, I think accountability is something missing.

• 17:44Speaker 02

  You can't, like, put an AI model in prison, right? Or something like that, you can't try it in court. So, obviously, there are some things that are defective in how we think about it, but you certainly can turn them off. And that's not something that you can do with a, with a person, in the same sense. So I, I think we should just think about how to play iterated games. You know, how do we develop trust for specific models? You know, we're doing new model releases seemingly every other week, right? That's, that's a regime we're in. At some point, people will, as model progress becomes more consistent, and something that we have an understanding for how it works, will probably develop reputation around specific models, right? We'll say, you know, "Maybe this model's a little smarter, but this model's a model that I trust."

• 18:29Speaker 02

  Right? Why can't there be reputation for models? Why can't there be reputation for these types of things, regardless of their capabilities? Basically, try to apply the way we think about human beings, a combination of control, incentives, trust, et cetera, et cetera. Layer all these things on, and allow people to figure it out, as opposed to the idea of a pause, right? Which is a very convenient sort of contrivance, in my view.

• 18:58Speaker 01

  Yeah, I mean, I agree that most proposals for a pause don't really have much in mind for what they would do with it.

• 19:05Speaker 02

  Yeah.

• 19:06Speaker 01

  Like, pause and then do what?

• 19:07Speaker 02

  Pause and then keep pausing, in, in some cases. I know not everybody means it when they say that.

• 19:13Speaker 01

  Yeah, I think, like, generally, there's a lot of conflation between, like, sort of bog-standard anti-tech people, who just kind of hate technology, even totally mundane technology, even, like, extremely net-positive technology. Like, look at how difficult it's been to deploy Waymos in cities around the country. And Waymos are, like, one of the most obviously net-positive technologies that exist.

• 19:35Speaker 02

  Yeah.

• 19:36Speaker 01

  And then there's the people who are, like, you know, the AI safetists, like, the rationalists, the EAs, who ultimately come from the same sort of SF tech ecosystem, they are techno-optimists in most cases, they're specifically worried about this, like, edge case of AI existential risk. Like, I think they're two different crowds of people.

• 20:00Speaker 02

  Yeah, yeah, and that's a point I want to get across, is there's many different groups, and conflating them is, is probably net-negative, right? We should, we should pull these different ideas apart. Waymos aren't a great example. Think of all the great healthcare stuff we're gonna do. Ironically, think about safer travel, think about safer, safer internet. Like, I want local models, myself, because I want something that's gonna be watching my home network and all my machines all the time, right? I think it'll make, make me and make many other people safer. And so I, I think, you, you're right, there will be a lot of frictions, and some people seem to really want infinite friction, and some people, some people obviously don't, some people want only a little bit, and they want to get a grasp of things, they're feeling a little bit hysterical, and that's, that's okay.

• 20:45Speaker 02

  Like, we're going to, I think, as capabilities improve, we're going to get better at understanding what's going on with these models.

• 20:51Speaker 01

  Right. Certainly, I hope so. What do you make of the EA movement in general? You know, obviously, it's a very broad and diffuse movement, EA means a lot of different things to a lot of different people, and it's, like, kind of has overlap with the AI safety world, and kind of not. It's complicated.

• 21:14Speaker 02

  Yeah, I don't know. I mean, I used to be on the Less Wrong forums all the time 10 years ago. Like, I've loved that corner of the internet. I almost would have called myself a rationalist even at some point. So I feel a lot of affinity with it, but it's been a while. And I think, yeah, I tend to resist trying to group them all into one bucket. But I will say that there are parts of groups that used to be what we'd call EA that seem more political than they are particularly interested in technology. So I don't know. I did see that Data Republican did a huge, huge drop.

• 21:50Speaker 01

  Yeah, we were just looking at this. Data Republican did a huge drop on, she said, effective altruism in their own words. And it's like a map of all of the connections between all the different EA-adjacent orgs. And then 1,851 quotes.

• 22:07Speaker 02

  Wow.

• 22:08Speaker 01

  It's like this, this vast variety of different quotes. Some of them are like more...

• 22:16Speaker 02

  I mean, it's, I mean, even just looking at these, like, I know that there's gonna be a mix of philosophical stuff, maybe some shrimp welfare stuff, some hardcore utilitarianism stuff, some AI doom stuff. Like, there's a whole blend here, right? This is just a big corner of, you know, the nerd internet. Like, it's hard to, you know, I don't think you can call it one specific thing.

• 22:41Speaker 01

  Right.

• 22:43Speaker 01

  Yeah, I mean, I don't mind the shrimp welfare stuff too much. I think it's bad if people attempt to trade off human welfare for it. But like...

• 22:50Speaker 02

  But that's exactly what they're doing.

• 22:52Speaker 01

  I don't know. I think like eye stalk ablation for shrimp is like bad.

• 22:57Speaker 02

  Maybe. I mean, I think there's too many repugnant conclusions if you go really far down that path.

• 23:03Speaker 01

  Well, you don't have to go really far down that path.

• 23:06Speaker 02

  I agree.

• 23:07Speaker 01

  You can, yeah, like, you know, you can minimize suffering for other beings without saying like, you need to like feed all of humanity to the insects or whatever.

• 23:18Speaker 02

  Yeah. No, I, yeah, I agree. Then it's not really utilitarianism, though. Or at least it's a non-fungible type.

• 23:26Speaker 01

  You can, you can make like utilitarian arguments for this stuff, where it's like, obviously, like, the total, like, you could say the total amount of value that humans can create in the universe is far greater than that that insects can create. And so obviously, killing all of the humans would be worse than killing all the insects, obviously. And, you know, of course, this aligns with most people's ethical conclusions at any rate.

• 23:52Speaker 02

  Yeah.

• 23:54Speaker 01

  I think like a lot, a lot of arguments against utilitarianism in this sense are just arguments against like very sloppy, like first order utilitarian thinking.

• 24:05Speaker 02

  I think if you, well, I don't know. We could talk about ethics, but I think if you go, if you crank the dial to 11 and you face all the most repugnant conclusions to their maximum, I think you just kind of end up deciding that utilitarianism is probably a crude proxy for something else.

• 24:23Speaker 01

  Yeah, I mean, like, if your philosophical framework leads you to believe that, like, human extinction is good, then seems like a bad philosophical framework?

• 24:33Speaker 02

  Yeah. Or that, you know, you could have a universe of eternal, of like, millions of years of profound suffering if it's all in service of a single femtosecond of like tend to the one billionth power ecstasy in a shrimp simulation.

• 24:49Speaker 01

  Yeah.

• 24:50Speaker 02

  Right? I got it. You know, I don't know.

• 24:51Speaker 01

  That's silly, but also, like, it's not, it's not really relevant to reality anyway.

• 24:57Speaker 02

  Yeah. And, yeah. And in any case, I think it's part of why the, the safety conversation is so confused today. Because it's, has all these things embedded in it a little bit, just a little bit, too much still. I think, I think, if I could make a little prediction, I think in the next year, I think we will have completely renovated the, the AI discourse. Because what's happening right now, part of why I think people are so excited, and why everybody's talking about this on X, and why everybody in Silicon Valley is talking about this, is because we're experiencing what it feels like for a lot of the cooped up kooky ideas that have lived in relative containment in our subculture are, they're,

• 25:41Speaker 02

  they're escaping and they're encountering broader political reality. And that is both exciting and totally transformative. What ends up happening is the arguments all get reformulated because they're in contact with the rest of culture. It's like when George Costanza freaks out about like worlds colliding.

• 26:00Speaker 01

  Hmm.

• 26:01Speaker 02

  Right? That's what's happening. Worlds are colliding. And so I think that discourse is going to change completely.

• 26:07Speaker 01

  Yeah. I mean, the, the key question that I am concerned with, probably more than any other, is like, do AI safetyists, effective altruists, and so on, end up ultimately aligning with sort of broad, low caliber anti-tech populism or not? I think it's enormously consequential.

• 26:26Speaker 02

  Yeah, it's a, it's a great question. I'm stewing over that one myself.

• 26:30Speaker 01

  Yeah, I hope not. Well, unfortunately, we got to wrap on that because we're running low on time. But Eddie, it's been great having you on. Thanks so much for coming into the studio in person.

• 26:40Speaker 02

  It's great. Thank you.

• 26:41Speaker 01

  To talk about philosophy.

• 26:42Speaker 02

  Thank you so much for having me.

• 26:46Speaker 00

  Thanks for listening to this episode of the A16Z Podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at A16Z.substack.com. Thanks again for listening, and I'll see you in the next episode.

• 27:11Speaker 00

  This information is for educational purposes only and is not a recommendation to buy, hold, or sell any investment or financial product. This podcast has been produced by a third party and may include paid promotional advertisements, other company references, and individuals unaffiliated with A16Z. Such advertisements, companies, and individuals are not endorsed by AH Capital Management LLC, A16Z, or any of its affiliates. Information is from sources deemed reliable on the date of publication, but A16Z does not guarantee its accuracy.