Marc Andreessen & Amjad Masad on “Good Enough” AI, AGI, and the End of Coding

Amjad Masad, founder and CEO of Replit, joins a16z’s Marc Andreessen and Erik Torenberg to discuss the new world of AI agents, the future of programming, and how software itself is beginning to build software. They trace the history of computing to the rise of AI agents that can now plan, reason, and code for hours without breaking, and explore how Replit is making it possible for anyone to create complex applications in natural language. Amjad explains how RL unlocked reasoning for modern models, why verification loops changed everything, whether LLMs are hitting diminishing returns, and if “good enough” AI might actually block progress toward true general intelligence. 00:00 Intro 00:37 Programming in Plain English 03:00 The Vision Behind Replit 05:15 From Machine Code to English Code 07:00 Building Apps with AI Agents 09:30 When the Agent Becomes the Programmer 11:00 Long-Horizon Reasoning and Coherence 13:45 Reinforcement Learning and Problem Solving 17:30 The Verification Loop and Multi-Agent Systems 21:15 Watching AI Work Like a Human Programmer 23:45 From Stochastic Parrots to Real Reasoning 26:00 Why Coding Is Advancing Faster Than Other Fields 30:15 Verifiable Domains: Math, Code, and Physics 33:45 The AGI Debate: Are We on Track? 37:45 Transfer Learning and the Limits of Human Intelligence 41:15 Functional AGI and Automating Labor 45:20 GPT-5, Diminishing Returns, and Lost “Humanity” 53:10 Creativity, Reasoning, and Finding Truth in AI 57:30 The Origins of Replit and Early Coding Days 01:03:00 Hacking His University and Getting Caught 01:08:00 The Redemption and Lessons Learned for the AI Age Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends! Resources: Follow Amjad on X: https://x.com/amasad Follow Marc on X: https://x.com/pmarca Find a16z on X: https://x.com/a16z Find a16z on LinkedIn: https://www.linkedin.com/company/a16z Listen to the a16z Podcast on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX Listen to the a16z Podcast on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711 Follow our host: https://x.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Marc Andreessen?

  We're dealing with magic here that we, I think, probably all would have thought was impossible five years ago, or certainly 10 years ago. This is the most amazing technology ever, and it's moving really fast, and yet we're still really disappointed. Like, it's not moving fast enough, and like, it's maybe right on the verge of falling out. We should both be like, hyper excited, but also on the verge of like, slitting our wrists. Because like, you know, the gravy train is coming to an end.

• 0:17Amjad Masad?

  Right. It is faster, but it's not at computer's speed, right? What we expect computer's speed to be.

• 0:22Marc Andreessen?

  It's sort of like watching a person work.

• 0:24Amjad Masad?

  It's like watching John Carmack on ClickCane.

• 0:28Marc Andreessen?

  The world, okay, the world's, the world's best programmer on a stimulant.

• 0:32Amjad Masad?

  On a stimulant, yeah, that's right.

• 0:34Speaker 02

  Every few decades, programming takes a massive leap forward, and this might be the biggest one yet. In this episode, Marc Andreessen and I are joined by Amjad Masad, CEO and founder of Replit, to talk about how AI agents are changing what it means to code. We discuss the end of syntax, the rise of agents that can think and build software for hours, and how reinforcement learning and verification loops are pushing AI towards something that looks a lot like reasoning. And finally, Amjad shares his story from hacking his university database in Jordan to building one of the most powerful developer tools in the world. Let's get into it.

• 1:12Marc Andreessen?

  So, let's start with, let's assume that I'm a sort of a novice programmer, so maybe I'm a student, or maybe I'm just somebody who took a few coding classes and I've hacked around a little bit, or, I don't know, I do Excel macros, or something like that. But I'm like, not, as well as I'm not like, a master craftsman of coding. And somebody tells me about Replit, and specifically AI and Replit. What's my experience when I launch in with what Replit is today with AI?

• 1:32Amjad Masad?

  Yeah, I think the experience of someone with no coding experience, or some coding experience, is largely the same when you go into Replit. The first thing we try to do is get all the nonsense away from setting up development environment and all of that stuff, and just have you focus on your idea. So, what do you want to build? Do you want to build a product? Do you want to solve a problem? Do you want to do a data visualization? So, the prompt box is really open for you, you can put in anything there. So, let's say you want to build a startup, you have an idea for a startup. I would start with a paragraph-long kind of description of what I want to build. The agents will read that, and will push it out.

• 2:08Marc Andreessen?

  Just type in English. In standard English, you just type it in. I want to sell grapes online. So you just like type in, I want to sell grapes online.

• 2:15Amjad Masad?

  It literally could be that four words or five words. Or it could be if you have a programming language you prefer or stack you prefer, you could do that. But we actually prefer for you not to do that because we're going to pick the best thing for, we're going to classify the best stack for that request. If it's a data app or big Python streaming or whatever, if it's like a web app or like JavaScript and Postgres and things like that. So you just type that.

• 2:38Marc Andreessen?

  Or you can decide, you can say, I want to do it. I know Python or I'm learning Python at school and I want to do it in Python.

• 2:43Amjad Masad?

  That's right. The cool thing about Replit is we've been around for almost 10 years now and we built all this infrastructure. Replit runs any programming language. So if you're comfortable with Python, you can go in and do that for sure.

• 2:53Marc Andreessen?

  Okay. And then just again, I know this is obvious, people have used it, but like I'm dealing in English.

• 2:57Amjad Masad?

  Yes.

• 2:58Marc Andreessen?

  Go ahead.

• 2:59Amjad Masad?

  Yes, you're fully in English. I mean, just a little bit of sort of background here, like when I came here and pitched to you like 10 years ago or whatever, seven years ago. What we were saying is we were exactly describing this future is that everyone would want to build software. And the thing that's kind of getting in people's ways is all the, what Fred Brooks called the accidental complexity of programming, right? They're like essential complexity, which is like, how do I bring my startup to market? And how do I build a business and all of that? Accidental complexity is what package manager do I use? All of that stuff, we've been abstracting away that for so many years. And the last thing we had to abstract away is code.

• 3:41Amjad Masad?

  I had this realization last year, which is I think we built an amazing platform, but the business is not performing. And the reason the business is not performing is that code is the bottleneck. Yes, all the other stuff is important to solve, but syntax is still an issue. Syntax is just an unnatural thing for people. So ultimately, English is the programming language.

• 4:01Marc Andreessen?

  By the way, just to, does it work with other world languages other than English at this point?

• 4:05Amjad Masad?

  Yes, you can, you can write in Japanese and we have a lot of users, especially Japanese. That tends to be very.

• 4:10Marc Andreessen?

  Does it support these days, like does AI support every language? Or is it still, do you still have to do custom work to craft a new language?

• 4:15Amjad Masad?

  Most mainstream languages that has 100 million plus people who speak it, AI is pretty good at it.

• 4:19Marc Andreessen?

  Okay.

• 4:20Amjad Masad?

  Yeah. So I did a bit of historical research recently for some reason. I just want to just understand the moment we're in. And because it's such a special moment, it's important to contextualize it. And I read this quote from Grace Hopper. So Grace Hopper invented the compiler, as you know. At the time, people were programming in machine code. And that's what programmers do. That's what the specialists do.

• 4:43Marc Andreessen?

  Yes.

• 4:43Amjad Masad?

  And she said, specialists will always be as special as they have to learn the underlying machinery of computers. But I want to get to a world where people are programming in English. That's what she said. That's before Kirtathie, right? That's 75 years ago. And that's why I invented the compiler. And in her mind, like C programming is English. But that was just the start of it. You had C and then you go higher level, Python, JavaScript. And I think we're at a moment where it's the next step. Instead of typing syntax, you're actually typing thoughts, which is what we ultimately want.

• 5:15Marc Andreessen?

  And the machine writes the code.

• 5:15Amjad Masad?

  And the machine writes the code.

• 5:16Marc Andreessen?

  Right, right. I remember, you're probably not old enough to remember, but I remember when I was a kid, there were higher level languages by the 70s, like BASIC and so forth, and FORTRAN and C. But you still would run into people who were doing assembly programming, assembly language, which by the way, you still do, like game companies or whatever still do assembly to get on.

• 5:31Amjad Masad?

  And they were hating on the kids that were doing BASIC.

• 5:33Marc Andreessen?

  So the assembly people were hating on the kids doing BASIC, but there were also older coders who hated on the assembly programmers for doing assembly and not, and not, and not, and not, no, no, no, not doing direct machine code.

• 5:41Amjad Masad?

  Right.

• 5:41Marc Andreessen?

  Not doing direct zero and one machine code. So if people don't know, assembly language is sort of this very low level programming language that sort of compiles to actual machine code. It's incomprehensible gibberish to most programmers, even most programmers.

• 5:51Amjad Masad?

  You're writing in Octal or something.

• 5:52Marc Andreessen?

  You're writing like very, very close to the hardware, but even still, it's still a language that compiles to zeros and ones. Whereas the actual real programmers actually wrote in zeros and ones.

• 5:59Amjad Masad?

  Yeah.

• 6:00Marc Andreessen?

  And so there's always this tendency for the pros to be looked down the nose.

• 6:03Amjad Masad?

  Yeah.

• 6:04Marc Andreessen?

  And say the new people are being basically sloppy. They don't understand what's happening. They don't really understand the machine. And then of course, what the higher level abstractions do is they democratize.

• 6:11Amjad Masad?

  The absolute irony is I was part of the JavaScript revolution. I was at Facebook before starting Replid. And we built the modern JavaScript stack. We built ReactJS and all the tooling around it. And we got a lot of hate from the programmers that you should type vanilla JavaScript directly. And I was like, okay, whatever. And yeah, and now that's mainstream. And now those guys that built their careers on the last wave we invented.

• 6:33Marc Andreessen?

  Right.

• 6:34Amjad Masad?

  Are hating on this new wave. It's just, people never change.

• 6:37Marc Andreessen?

  Okay, got it. Okay, so you're typing English. I want to sell grapes online. I want to do this. I want to have a teacher, whatever the business is. Okay, what happens then?

• 6:43Amjad Masad?

  Yeah, and then a Raplet agent will show you what it understood. So it's trying to build a common understanding between you and it. And I think there's a lot of things we can do better there in terms of UI. But for now, I'll show you a list of tasks. I'll tell you, I'm gonna go set up a database because you need to store your data somewhere. We need to set up Shopify or Stripe because we need to accept payments. And then it shows you this list and gives you two options initially. Do you want to start with a design so that we can iterate back and forth to get locked that design down? Or do you want to build a full thing? Hey, if you want to build a full thing, we'll go for 20, 30, 40 minutes. And the agent will tell you, go here, install the app. I'm gonna go set up the database, do the migrations, write the SQL, build the site. I'm gonna also test it.

• 7:27Amjad Masad?

  So this is a recent innovation we did with Agent 3. Is that after it writes the software, spins up a browser, goes around and tests in the browser. And then any issue, it kind of iterates, kind of goes and fixes the code. So I'll spend 20, 30 minutes building that, I'll send you a notification, it'll tell you the app is ready. So you can test it on your phone, go back to your computer. You'll see, maybe you'll find a bug or an issue, you'll describe it to the agent. I'll say, hey, it's not exactly doing what I expected. Or if it's perfect, you're ready to go and that's it. By the way, there's a lot of examples where people just get their idea in 20, 30 minutes, which is amazing. You just hit publish. You hit publish, couple clicks, you'll be up in the cloud. We'll set up a virtual machine in the cloud, the database is deployed.

• 8:11Amjad Masad?

  Everything's done and now you have a production database. So think about the steps needed just two or three years ago in order to get to that step. You have to set up your local development environment, you have to sign up for an AWS account. You have to provision the databases, the virtual machines, you have to create the entire deployment pipeline. All of that is done for you and it just, a kid can do it, a layperson can do it. If you're a programmer and you're curious about what the agent did. The cool thing about Raplet, because we have this history of being an IDE, you can peel the layers. You can open the file tree and you could look at the files. You can open Git, you can push to GitHub, you can connect it to your editor if you want, you can open an Emacs. So the cool thing about Raplet, yes, it is a vibe coding platform that abstracts away all the complexities. But all the layers are there for you to look at.

• 8:56Marc Andreessen?

  That was great, but let's go back to you said it, you say, I've got my idea, you plug it in and it says it gives you this list of things. And then when you describe it, you said, I'm gonna do this, I'm gonna do that. The I there in that case was the agent as opposed to the user.

• 9:06Amjad Masad?

  Yes.

• 9:06Marc Andreessen?

  And so the agent lists the set of things that it's going to do, and then the agent actually does those things.

• 9:11Amjad Masad?

  Agent does those things. Yeah, that's a very important point. When we did this shift, we hadn't realized internally at Raplet how much the actual user stopped being the human user and it's actually the agent programmer.

• 9:23Marc Andreessen?

  Right.

• 9:23Amjad Masad?

  So one really funny thing happened is we had servers in Asia. And the reason we had servers in Asia, because we wanted our Indian or Japanese users to have a shorter time to the servers. When we launched Agent, their experience got significantly worse. And we're like, what happened? Like, it's supposed to be faster. Well, it turns out it's worse is because the AIs are sitting in the United States. And so the programmers actually in the United States, it's you're sending the request to the programmer and the programmer is interfacing with a machine across the world. And so, yes, suddenly the agent is the programmer.

• 9:54Marc Andreessen?

  Okay, so that's new terminology. Agent is a software program that is basically using the rest of the system as if it were, as if it were a human user, but it's not. It's a bot.

• 10:03Amjad Masad?

  That's right. It has access to tools such as write a file, edit a file, delete a file, search the package index, install a package, provision a database, provision of object storage. It is a programmer that has the tools and interface, it has sort of an interface that is very similar to a human programmer.

• 10:22Marc Andreessen?

  And then, you know, we'll talk more about how this all works, but a debate inside the AI industry is with these, was kind of this, you know, this idea now of having agents that do things on your behalf and then go out, you know, go out and kind of accomplish missions. There's this, you know, kind of debate, which is, okay, how, like, obviously, you know, it's a big deal even to have an AI agent that can do relatively simple things. To do complex things, of course, is, you know, one of the great technical challenges of the last 80 years, you know, to do that. And then there's this sort of this question of like, can the agent go out and run and operate on its own for five minutes? You know, for 15 minutes, for an hour, for eight hours? And, and meaning like, sort of like, how long does it maintain coherence? Like, how long does it actually like stay in full control of its faculties and not kind of spin out? Because at least the early agents or the early AIs, if you set them off to do this, they might be able to run for two or three minutes.

• 11:07Marc Andreessen?

  And then they would, they would start to get confused and go down rabbit holes and, you know, kind of, kind of spin out. More recently, more recently, you know, we've seen that agents can run a lot longer and do more complex tasks. Like, where are we on the curve of agents being able to run for how long and for what complexity tasks before they break?

• 11:25Amjad Masad?

  That's absolutely the, I think, the main metric we're looking at. Even back in 2023, you know, had the idea for software agents, you know, four or five years ago now. The problem, every time we attempt them, the problem of coherence, you know, they'll go on for a minute or two and then they'll just, you know, they're to compound in errors in a way that they just can't recover.

• 11:47Marc Andreessen?

  And you can actually see it, right? Because they actually, they actually, if you watch them operate, they get increasingly confused and then, you know, maybe even deranged.

• 11:53Amjad Masad?

  Yeah, very deranged and they go into very weird areas and sometimes they start speaking Chinese and doing really weird things. But I would say sometime around last year, we maybe crossed a three, four, five minute mark and it felt to us that, okay, we're on a path where long, you know, long horizon reasoning is getting solved. And so we made, we made a bet and I tell my team.

• 12:20Marc Andreessen?

  So long horizon reasoning, meaning reasoning, meaning like dealing in like facts and logic in a sort of complex way. And then long horizon being over a long period of time.

• 12:32Amjad Masad?

  Yes.

• 12:32Marc Andreessen?

  With many, many steps to a reasoning process?

• 12:34Amjad Masad?

  Yeah, that's right. So if you think about the way large language models work is that they have a context. This context is basically the memory, all the text, all your prompts and also all the internal talk that the AI is doing as it's reasoning. So when the AI is reasoning, it's actually talking to itself. It's like, oh, now I need to go set up a database. Well, what kind of tool do I have? Oh, there's a tool here that says Postgres. Okay, let me try using that. Okay, I use that. I got feedback. Let me look at the feedback and read it and it'll read the feedback. And so that prompt box or context is where both the user input, the environment input and the internal thoughts of the machine are all within. It's sort of like a program

• 13:16Amjad Masad?

  memory in memory space. And so reasoning over that was the challenge for a long time. That's when AIs just like went off track. And now they're able to kind of think through this entire thing and maintain coherence. And there's there's now techniques around compression of contexts. So there's still the context length is still a problem, right? So I would say LLMs today, you know, they're marketed as a million token length, which is like a million words almost. In reality, it's about 200,000 and then they start to struggle. So we do a lot of, you know, we sop, we compress the memory. So if a memory, if if a portion of the memory is saying that I'm getting all the logs from the database,

• 14:00Amjad Masad?

  you can summarize, you know, paragraphs of logs with one statement or the database setup. That's it, right? And so every once in a while, we'll compress the context so that we make sure we maintain coherence. So that there's a lot of innovation happened outside of the foundation models as well in order to to enable that long context coherence.

• 14:19Marc Andreessen?

  So what was the what was the key technical breakthrough in the foundation models that made this possible, do you think?

• 14:24Amjad Masad?

  I think it's RL. I think it's reinforcement learning. So the way pre-training works is, you know, they pre-training is the first step of training a large language model. It reads a piece of text, it covers the last words and tries to guess it. That's how it's trained. That doesn't really imply long context reasoning. You know, it it it turns out to be very, very effective. It can learn language that way. But the reason we weren't able to move past that limitation is that that modality of training just wasn't good enough. And what you want is you want a type of problem solving over a over long context.

• 15:07Amjad Masad?

  So what reinforcement learning, especially from Codex Union, gave us is the ability to for the machine to for the LLM to roll out what we call trajectories in AI. So a trajectory is a step-by-step reasoning chain in order to reach a solution. So the way, as I understand, reinforcement learning works. Works is they put their in a programming environment like Replit and say, hey, here's a here's a code base. Here's a bug in the code base and we want you to solve it. Now, the human trainer already knows what the solution would look like. So we have a pull request that we have on GitHub. So we know exactly, or we have a unit test that we can run and verify the solution.

• 15:53Amjad Masad?

  So what it does is it rolls out a lot of different trajectories. They sample the model. And maybe one of those trajectories will reach, and a lot of them will just go go off off track. But one of them will reach the solution by solving the bug and it reinforces on that. So that that gets a reward and the model gets trained that, okay. You know, this is how you solve these type of problems. So that's how we're able to extend these reasoning chains.

• 16:17Marc Andreessen?

  Got it. And how, so two part question is how, how, how good? How good are the models now at long, long, long reasoning? And I would say, and how do we know? Like, how is that established?

• 16:30Amjad Masad?

  There is a nonprofit called Meter that is measuring useful. Has a benchmark to measure how long a model runs while maintaining coherence and doing useful, useful things. Whether it's programming or other top benchmark tasks that they've done. And they put up a paper, I think, late last year that said, every seven months. The, the minutes that a model can run is doubling. So you go from two minutes to, you know, four minutes in seven months. I think they vastly underestimated that.

• 17:08Marc Andreessen?

  Is that right?

• 17:08Amjad Masad?

  Vastly.

• 17:09Marc Andreessen?

  It's doubling, it's doubling more often than seven months.

• 17:11Amjad Masad?

  We, so agent three, we measure that, you know, very closely. And we measure that in real tasks from real users. So we're not doing benchmarking. We're actually doing A B tests and we're looking at the data that how users are successful or not. For us, the Absolute sign of success is you made an app and you published it. Because when you publish it, you're paying extra money. You're saying this app is economically useful. I'm going to publish it. So that's as clear cut as possible. And so what we're seeing is in agent one, the agent can run for two minutes. And then, and then perhaps struggle. Agent two came out in February. It ran for 20 minutes. Agent three, 200 minutes. Okay. 200 minutes. Some users are pushing it to like 12 hours and things like that.

• 17:53Amjad Masad?

  I'm less confident that it is as good when it goes to these stratospheres. But at like two, three hours timeline, it is really, it's, it's, it's insanely good. And, and the main innovation outside of the models is a verification loop. Actually, I remember reading a research paper from Nvidia. So what Nvidia did is they're trying to write GPU kernels using DeepSeek. And that was like perhaps seven months ago when DeepSeek came out. And what they found is that if we add a verifier in the loop, if we can run the kernel and verify it's working, we're able to run DeepSeek for like 20 minutes. And it was generating actually optimized kernels.

• 18:37Amjad Masad?

  And so I was like, okay, the next thing for us, obviously as an, as a sort of a agent's lab or like app layer company, we're not doing the foundation model stuff, but we're doing a lot of research on top of that. And so, okay, we know that agents can run for 10, 20 minutes now. Our LLMs can stay coherent for longer. But for you to push them to 200, 300 minutes, you need a verifier in the loop. So that's why we spend all our time creating scaffolds to make it so that the agent can spin up a browser and do computer use style testing. So, once you put that in the middle, what's happening is it works for 20 minutes. It spins up, another agent spins up a browser, tests the work of the previous agent.

• 19:21Amjad Masad?

  So it's a multi-agent system. And if it is. If it founds a bug, it starts a new trajectory and says, okay, good work. Let's summarize what you did the last 20 minutes.

• 19:33Marc Andreessen?

  Right.

• 19:33Amjad Masad?

  Now, that plus the bug that we found, that's a prompt for a new trajectory.

• 19:37Marc Andreessen?

  Right.

• 19:38Amjad Masad?

  So you stack those on each other and you can go endlessly.

• 19:40Marc Andreessen?

  So it's like setting up a marathon or like a relay race.

• 19:43Amjad Masad?

  Mm hmm.

• 19:44Marc Andreessen?

  As long as each step is done properly, you could do an infinite number of steps.

• 19:47Amjad Masad?

  That's right. You can always compress the previous step into a paragraph and that becomes a prompt. So it's an agent prompting the next agent.

• 19:54Marc Andreessen?

  Right, right, right. That's amazing. So and then when an agent, like when a modern agent, like running on modern LMs that are trained this way, let's say it runs for 200 minutes, like when you watch the agent run, is it like running, is it like processing through like logic and tasks at the same pace that like a human being is? Or slower or faster?

• 20:11Amjad Masad?

  It's actually, I would say it is faster, but not that much significantly faster. It's not at computer speed, right, what we expect computer speed to be.

• 20:21Marc Andreessen?

  It's like watching a person, like if you watch, if it's describing what it's doing, it's sort of like watching a person work.

• 20:26Amjad Masad?

  It's like watching John Carmack on cocaine work.

• 20:30Marc Andreessen?

  The world, okay, the world's best programmer.

• 20:33Amjad Masad?

  Yeah.

• 20:34Marc Andreessen?

  The world's best programmer on a stimulus.

• 20:36Amjad Masad?

  Yeah, that's right.

• 20:37Marc Andreessen?

  Okay, working for you, working for you.

• 20:40Amjad Masad?

  Yeah, it's very fast and you can see the file diffs running through. But every once in a while, it'll stop and it'll start thinking, I'll show you the reasoning.

• 20:48Marc Andreessen?

  Yeah.

• 20:48Amjad Masad?

  It's like, I did this and I did this. Am I on the right track? It kind of really tries to reflect.

• 20:53Marc Andreessen?

  Right.

• 20:53Amjad Masad?

  And then it might review its work and decide the next step. Or it might kick into the testing agent. Or, you know, so you're seeing it do all of that. And every once in a while, it calls a tool, for example, it stops and says, well, we ran into an issue, you know, Postgres 15 is not

• 21:12Amjad Masad?

  compatible with this database ORM package that I have.

• 21:17Amjad Masad?

  Okay, this is a problem I haven't seen before. I'm gonna go search the web. So it has a web search tool, go do that. And so it looks like a human programmer.

• 21:25Marc Andreessen?

  Right.

• 21:26Amjad Masad?

  And it's really fascinating to watch it. So one of my favorite things to do is just to watch the tool chain and reasoning chain and the testing chain. It is, yeah, it is like watching a hyper productive programmer.

• 21:37Marc Andreessen?

  Right. So, you know, we're kind of getting into here, kind of the holy grail of AI, which is sort of, you know, generalized reasoning, you know, by the machine. So you mentioned this a couple of times with this idea of verification. So so just for folks on the listening podcast who maybe aren't in the details, let me try to describe this and see if I have it right. So like just a just a large language model, the way you would experience, you would have experienced with like chat GPT out of the gate two years ago or whatever. Would have been, it's like, it's incredible how fluid it is at language. It's incredible how good it is at like writing Shakespearean songs or rap lyrics. It's amazing how good it is at human conversation. But if you start to ask it like problems that involve like rational thinking or problem solving, all of a sudden, like

• 22:18Amjad Masad?

  Or math.

• 22:18Marc Andreessen?

  Or the math, the whole show. And in the very beginning, it was you could ask, if you ask a very basic math problems that, you know, it would, it would not be able to do them.

• 22:24Amjad Masad?

  That's right.

• 22:25Marc Andreessen?

  But then even when it got better at those, if you started to ask it to like, you know, it could maybe add two small numbers together, but it couldn't add two large numbers together. Or if it could add two large numbers, it couldn't multiply them.

• 22:34Amjad Masad?

  Yeah.

• 22:34Marc Andreessen?

  And it's just like, all right, this is true. And then it had this, there was this famous, the famous was the strawberry, the strawberry test, the famous strawberry test, which is how many Rs are in the word strawberry?

• 22:41Amjad Masad?

  That's right.

• 22:42Marc Andreessen?

  And there was this long period where it kept, it would, it would just guess wrong. It would say there were only two Rs in the word strawberry. And it turns out there are three. So, so it was this thing. And so people were, and there was even this term that was being used, kind of the slur that was being used at the time was stochastic parrot.

• 22:57Amjad Masad?

  Yeah, I was thinking clanker.

• 23:00Marc Andreessen?

  The clanker is the, is the new slur. Clanker, clanker is just the full on racial slur against AI as a species. But the technical critique was so called stochastic parrot, stochastic means random.

• 23:11Amjad Masad?

  Yeah.

• 23:11Marc Andreessen?

  So sort of random parrot, meaning basically that this thing was sort of, the large language models were like a, they were like a mirage. Where they were like repeating back to you things that they thought that you wanted to hear, but they didn't.

• 23:20Amjad Masad?

  And in a way, it's true in the, in the pure pre-training LLM world.

• 23:23Marc Andreessen?

  Right. For the, for the very basic layer. But then what happened is, as you said, over the last year or so. There was this layering of reinforcement learning. And then, but the key.

• 23:31Amjad Masad?

  It's not new, crucially, it's like it's AlphaGo, right?

• 23:34Marc Andreessen?

  Right. Describe, so describe that for a second.

• 23:36Amjad Masad?

  Yeah, so we had this breakthrough before and 2015 was the AlphaGo breakthrough. I think 2015, 2016, where it is a merging of sort of, you know, the, you would know a lot better than me. The old AI debate between the connectionists, the people who thinks neural networks are the true sort of way of doing AI. And symbolic systems, I think, or like the people that think that, you know, discrete reasoning, so F statements and knowledge bases, whatever. This is the way to go. And so there was, there was a merging of these two worlds where the way AlphaGo worked is it had a neural network. But it had a Monte Carlo tree search algorithm on top of that.

• 24:17Amjad Masad?

  So the neural network would generate, would like, generate a list of potential moves. And then you had a more discrete algorithm sort those moves and find the best based on just tree search. Based on just trying to verify, again, this is sort of a verifier in the loop. Trying to verify which move might yield the best based on more classical way of doing algorithms. And so that's a resurgence of that movement where we have this amazing generative neural network. That is the LLM. And now let's layer on more discrete ways of trying to verify whether it's doing the right thing or not. And let's put that in a training loop.

• 24:59Amjad Masad?

  And once you do that, the LLM will start gaining new capabilities such as reasoning over math and code and things like that.

• 25:06Marc Andreessen?

  Exactly right. Okay. And then, that's great. And then, and then the key thing there, though, for our L to work, for LMs to reason, the key is that it be a problem statement. That there is a defined and verifiable answer.

• 25:17Amjad Masad?

  That's right.

• 25:17Marc Andreessen?

  Is that right? And so, and you might think about this as like, let's give a bunch of examples. Like in medicine, this might be like, you know, a diagnosis that like a panel of human doctors agrees with. Or, by the way, or a diagnosis that actually, you know, solves the condition. In law, this would be a, you know, a argument that in front of a jury actually results in an acquittal. Or something like that. In math, it's an equation that actually solves properly. In physics, it's a result that actually works in the real world. I don't know, in civil engineering, it's a bridge that doesn't collapse. Right? So, so there's always some, some test of.

• 25:49Amjad Masad?

  Caveat is that the first two do not work very well.

• 25:53Marc Andreessen?

  Okay.

• 25:53Amjad Masad?

  Just yet. Like the, like, I would say. Law and healthcare, they're still a little too squishy, a little too soft.

• 26:01Marc Andreessen?

  Okay.

• 26:02Amjad Masad?

  It's unlike math or code. Like the way that they're training on math, they're using this sort of like a program language, provable language called Lean for proofs, right? So you can run a Lean statement, you can run a computer code. Perhaps you can run a physics simulation or civil engineering sort of physics simulation, but you can't run a diagnosis.

• 26:22Marc Andreessen?

  Okay.

• 26:22Amjad Masad?

  So I would say that-

• 26:24Marc Andreessen?

  But you could verify it with human answers or not.

• 26:26Amjad Masad?

  Yeah, so that's a more edge half in a way.

• 26:29Marc Andreessen?

  Okay.

• 26:30Amjad Masad?

  So it is not the like sort of autonomous RL train, like fully scalable autonomous.

• 26:35Marc Andreessen?

  Okay.

• 26:36Amjad Masad?

  Which is why coding is moving faster than any other domain is because we can, we can generate these problems and verify them on the fly.

• 26:45Marc Andreessen?

  But there's two, with coding, as anybody who's coded knows, there's coding, there's two tests, which is one is, does the code compile?

• 26:50Amjad Masad?

  Right.

• 26:50Marc Andreessen?

  And then the other is, does it produce the right output?

• 26:52Amjad Masad?

  Right.

• 26:52Marc Andreessen?

  And just because it compiles doesn't mean it produces the right output.

• 26:55Amjad Masad?

  Right.

• 26:55Marc Andreessen?

  You tell me, but verifying that it's the correct output is harder.

• 26:58Amjad Masad?

  Yeah, so Sweetbench is a collection of verified pull requests and states. So it is, it is not just about compiling. We, so they, a group of scientists, so Sweetbench is the main benchmark used to test whether AI is good at software engineering tasks. And we're almost saturating that. So last year at like maybe 5% early 24 or less. And now we're like 82% or something like that with Clothson at 4.5. That's state of the art. And that's like a really nice hell climb that's happening right now. And basically they went and looked on GitHub. They found the, you know, most complex repositories.

• 27:42Amjad Masad?

  They found bug statements that are very clear. And they found pull requests that actually solve those bug statements with the unit tests and everything. So there is an existing corpus on GitHub of tasks that, that the AIs can solve. And you can also generate them. Those are not too hard to, to generate, you know, what's called synthetic data. But, but you're right, it's not infinitely scalable. Because you, some human verifier still need to kind of look at the, at the task. But maybe the foundation models have found a way to have the synthetic training go all the way.

• 28:18Marc Andreessen?

  Right. And then what's happening, I think, I think because what's happening is the foundation model companies are, in some cases, they are higher, they're actually hiring human experts to generate new training data.

• 28:26Amjad Masad?

  Yes.

• 28:26Marc Andreessen?

  So they're actually hiring mathematicians and physicists and coders to basically sit and, you know, they're, they're, they're hiring, they're, they're, they're hiring human programmers, putting them on the cocaine.

• 28:34Amjad Masad?

  Yes.

• 28:36Marc Andreessen?

  Having them probably coffee.

• 28:37Amjad Masad?

  Yeah.

• 28:38Marc Andreessen?

  Um, and, and having them actually write code and then, and then write code in a way where there's a known result of the code running such that the, this RL loop can be trained properly.

• 28:45Amjad Masad?

  That's right.

• 28:46Marc Andreessen?

  And then the other, the other, and then the other thing these companies are doing is, is they're, they're building systems where the software itself generates the training data, generates the tests, generates the valid, the validated results. And then that's so-called synthetic training data.

• 28:58Amjad Masad?

  That's right. And, but, yeah, but, but again, those, those work in the very hard domains. It works to some extent in the software domains. And I think there's some transfer learning. You can, you can see the reasoning work when it comes to, you know, tools like deep research and things like that. But we're not making as fast as progress in the, in the more soft domains.

• 29:18Marc Andreessen?

  So it's, it's, it's, it's, it's software domains, meaning like domains in which it's hard, it's harder, it's harder or even impossible to actually verify correctness of, of result.

• 29:26Amjad Masad?

  Yeah.

• 29:26Marc Andreessen?

  In a sort of a deterministic factual grounded.

• 29:29Amjad Masad?

  Yeah.

• 29:29Marc Andreessen?

  Non-controversial way.

• 29:30Amjad Masad?

  Like if you have a chronic disease, you could, you could have, you know, you have a POTS or, uh, you know, whatever, EDS syndrome or. And, and they're all, they're all clusters and it's because it is the domain of abstraction. It is not as concrete as code and math and things like that. So I think there's still a long ways to go there.

• 29:51Marc Andreessen?

  Right. So sort of the more concrete, the problem, like it's the concreteness of the problem that is the key variable, not the difficulty of the problem. Would that be a way to think about it?

• 29:58Amjad Masad?

  Yeah. I think the, the, uh, concreteness in a sense of can you get a true or false verifiable?

• 30:05Marc Andreessen?

  Right. But like in any domain, in any domain of, of human effort in which there's a verifiable answer, we should expect extremely rapid progress.

• 30:10Amjad Masad?

  Yes.

• 30:11Marc Andreessen?

  Right.

• 30:11Amjad Masad?

  Yes, absolutely. And I, I think that's what we're saying.

• 30:13Marc Andreessen?

  And that, and that for sure includes math, that for sure includes physics, for sure includes chemistry, for sure includes large areas of code.

• 30:18Amjad Masad?

  That's right.

• 30:19Marc Andreessen?

  Right. What, what else does that include, do you think?

• 30:21Amjad Masad?

  Bio, like we're seeing with a protein.

• 30:23Marc Andreessen?

  Like genomics, yeah, yeah.

• 30:25Amjad Masad?

  Yeah, yeah. Things like that. I think some, some, uh, areas of robotics.

• 30:29Marc Andreessen?

  Right.

• 30:29Amjad Masad?

  Um, uh, there's a clear outcome.

• 30:33Marc Andreessen?

  Right.

• 30:34Amjad Masad?

  But, but it's not that many. I mean, surprisingly.

• 30:36Marc Andreessen?

  Well, it depends.

• 30:37Amjad Masad?

  Yeah.

• 30:38Marc Andreessen?

  Depends on your point of view. That's, that's, some people might say that's a lot. Um, so, uh, and then, um, you, you mentioned that, you mentioned the pace of improvement. So what would you expect from the pace of improvement going forward for this?

• 30:48Amjad Masad?

  I think we're ripping on coding. Like, I think it's just, it's going. Like, I think it's gonna be, like, what we're working on with Agent 4 right now is by next year, we think you're gonna be sitting in front of Raplet, and you're shooting off multiple agents at a time. You're like, planning a new feature. So I want a social network on top of my storefront. And another one is like, hey, refactor the database. And you're running PDL agents. So you have about 5, 10 agents working in the background, and they're merging the code, and it'll take care of all of that. But you also have a really nice interface on top of that, that you're doing design, and you're interacting with AI in a more creative way,

• 31:33Amjad Masad?

  maybe using visuals and charts and things like that. So there's a multimodal angle of that, of that interaction. So I think, you know, creating software is gonna be such an exciting

• 31:46Amjad Masad?

  area, and I think the layperson will be as good as what a senior software engineer that works at Google is today. So I think that's happening very soon. But, you know, I don't see them, and I'd be curious about your point of view, but like, my experience between, as a sort of a, you know, on the, let's say, healthcare side, or more, you know, write me an essay side, or more creative side, I haven't seen as much of a rapid improvement as what we're seeing in code. So I think code is gonna go to the moon, math is probably as well. Some, you know, scientific domains, bio, things like that, those are gonna move really fast.

• 32:25Marc Andreessen?

  Yeah, so there's this weird dynamic, see if you agree with this, and Eric also, I'd be curious your point of view on this. Like, there's this weird dynamic that we have, and we have this in the office here a lot, and I also have this with like, the leading entrepreneurs a lot, which is this thing of like, like, wow, this is the most amazing technology ever, and it's moving really fast, and yet we're still like, really disappointed. And like it's not moving fast enough and like it's right like maybe right on the verge of stalling out and like, you know, we should both be like hyper excited, but also on the verge of like slitting our wrists because like, you know, the gravy train is coming to an end.

• 32:54Amjad Masad?

  Right.

• 32:55Marc Andreessen?

  And I always wonder, it's like, you know, on the one hand, it's like, okay, like, you know, not all, I don't know, letters go to the moon, like just because something, you know, looks like it works or, you know, doesn't mean it's going to, you know, be able to scale, you're going to be able to scale it up and have it work, you know, to the fullest extent. Um, uh, you know, so like it's important to like recognize practical limits and then not just extrapolate everything to infinity. Um, on the other hand, like, you know, we're dealing with magic here that we I think will probably all would have thought was impossible five years ago, or certainly 10 years ago. Like I didn't, you know, like I, you know, I got my CS degree in the late eighties, early nineties. I never, I didn't think I would live to see any of this, right? Like, oh, this is just amazing that this is actually happening in my lifetime.

• 33:29Amjad Masad?

  But there's a huge bet on AGI, right? Like whether it's the foundation models, uh, I think, you know, now the entire US economy is sort of a bet on AGI. And there are crucial questions to ask whether are we on track to AGI or not, because there are some ways that I can tell you it doesn't seem like we're on track to AGI because we, uh, because there doesn't seem to be transfer learning across these domains that are, that are, you know, significance, right? So if we get a lot better at code, we're not immediately getting better at like generalized reasoning. We need to go also, you know, get training data and create RL environment for bio or chemistry or physics or math or law or so. And this, this has been the sort of point of discussion now in the AI

• 34:14Amjad Masad?

  community after the Dvarkesh and Richard Sutton interview where, uh, you know, Richard Sutton kind of poured this cold water on the, on the bitter lesson. So everyone was using this essay that he wrote called the bitter lesson. The idea is that there are infinitely scalable ways of uh, doing uh, AI research and, and, and anytime you can pour more compute and more data and get more performance out, you're just, you know, that's the ultimate way of getting to AGI. And some people, you know, interpreted that interview that perhaps he's doubtful that even, we're even on a, on a bitter lesson path here and perhaps.

• 34:59Amjad Masad?

  The current training regime is actually very much the opposite in which we are so dependent on human data and human annotation and all of that stuff. So I think that I agree with you. I mean, as a company, we're excited about where things are headed, but there's a question of like, are we on track to AGI or not? And curious what you think.

• 35:21Marc Andreessen?

  So, and you know, I think Ilja Sesciver makes a specific form of this argument, which is basically like we're just literally running out of training data.

• 35:28Amjad Masad?

  It's the fossil fuel argument.

• 35:29Marc Andreessen?

  Right, like we've slurped all the training data. Fundamentally, we've slurped all the data off the internet. That is where almost all the data is at this point. There's a little bit more data that's in like, you know, private dark pools somewhere that we're going to go get, but we have it all. And then, right, we're in this business now trying to generate new data, but generating new data is hard and expensive, you know, compared to just like slurping things off the internet. So there are these arguments. Um, you know, having said that, you know, you get into definitional questions here really quick, which are kind of a rabbit hole. But having said that, like you mentioned, transfer learning. So transfer learning is the ability of the machine to, right, to be an expert in one domain and then generalize that into another domain. My answer to that is like, have you met people? And how many people do you know who are able to do transfer learning?

• 36:05Amjad Masad?

  Yeah, not many.

• 36:06Marc Andreessen?

  Not many, right? Well, because there's

• 36:08Amjad Masad?

  Quite the opposite, actually. The nerdier they are in a certain domain, the kind of, you know, often they have blind spots.

• 36:14Erik Torenberg?

  We joke about how everyone's just retarded in one area or they make some like massive mistake and don't trust them on this, but another other topic, you know?

• 36:21Marc Andreessen?

  Right. Yeah, well, and this is a well-known thing among like, for example, public intellect. So this happens, but there's actually been whole books written about this on so-called public intellectuals. So you get these people who show up on TV and they're experts. And what happens is they're like an expert in economics, right? And then they show up on TV and they talk about politics and they don't know anything about politics, right? Or they don't know anything about like medicine, or they don't know anything about the law, or they don't know anything about computers. You know, this is the Paul Gerkman talking about how the internet is going to be no more significant than the fax machine.

• 36:44Amjad Masad?

  Fax, yeah.

• 36:45Marc Andreessen?

  The brilliant economist has no idea what a computer

• 36:47Amjad Masad?

  Is he a brilliant economist?

• 36:48Marc Andreessen?

  Well, at one point.

• 36:51Marc Andreessen?

  At one point, at one point, let's get, even if, even if he's a brilliant, well, this is the thing, like, what does that mean? Should a brilliant economist be able to extrapolate, you know, the internet is a good question, but, um, but the point being like, even if he is a brilliant, you know, or take anybody, take anybody, oh, by the way, or like Einstein, like Einstein's like actually my favorite example, like, I think you'd agree Einstein was a brilliant physicist. He was like, he was, he was a Stalinist, like he was, yeah, he was a socialist and he was a Stalinist and he was like, well, he thought like Stalin was fantastic.

• 37:18Amjad Masad?

  Well, he's already out, so.

• 37:20Marc Andreessen?

  Yeah, okay, all right.

• 37:21Amjad Masad?

  True socialism.

• 37:22Marc Andreessen?

  All right, all right, Einstein, you know, I'll, I'll, I'll, I'll, I'll, I'll take your word for it. But like, once he got into politics, he was just like totally loopy or, or, you know, or even right or wrong is just, he just sounded like all of a sudden like an undergraduate lunatic, like somebody in a dorm room, like he, there was no transfer learning from physics into politics. Like he, right or wrong, he didn't, there was no, there was clearly, there was nothing new in his political analysis. It was the same rote routine bullshit you get out of, you know.

• 37:47Amjad Masad?

  Yeah. So in a way, the argument you're making is like, we may be already a human level AI. I mean, perhaps the definition of AGI is, is, is something totally different. It's like above a human level that something that truly generalizes across domains is, is not something that we've seen.

• 38:01Marc Andreessen?

  Yeah, like we've ideal, yeah, I was, I was saying, so we've, we've, and you know, look, we should, we should shoot big, but we've idealized a, we've idealized a goal, um, that may be idealized in a way that like, number one, it's just, it's, it's like so far beyond what people can do, that it's, it's no, you're no longer, it's no longer a relevant comparison to people. And, and usually AGI is defined as, you know, able to do everything better than a person can. And it's like, well, okay, so if doing everything better than a person can, it's like if a person can't do any transfer learning at all.

• 38:25Amjad Masad?

  Right.

• 38:26Marc Andreessen?

  Doing even a little, little bit, a marginal bit might, might actually be better, or it might not matter, just because no, no human can do it. And so therefore you just, you just stack up the domains. There's also this well-known phenomenon in AI, you know, typically this works the other way, which is phenomenon in AI, AI engineers always complain about, scientists always complain about, which is the definition of AI is always the next thing that, that the machine can't do. And so like the definition for AI for a long time was like, can it beat humans at chess? And then the minute it could beat humans at chess, that was no longer AI. That was just like, oh, that's just like boring.

• 38:53Amjad Masad?

  That's computer chess.

• 38:54Marc Andreessen?

  Computer chess. So it's just like boring and now it's an app on your iPhone and nobody, and nobody, and nobody cares, right? And it's immediately the.

• 39:00Amjad Masad?

  Chewing test was the.

• 39:01Marc Andreessen?

  The Turing test.

• 39:02Amjad Masad?

  And then we passed it and nobody.

• 39:03Marc Andreessen?

  We blew, this is a really big deal.

• 39:04Amjad Masad?

  There was no celebration.

• 39:06Marc Andreessen?

  There was no parties. That's exactly right. There was no party. For 80 years, the Turing test, I mean, they made a movie about it, like the whole thing. That was the thing. And like, we blew right through it and nobody even registered it. Nobody cares. It gets no credit for it. We're just like, ah, it's still, you know, a complete piece of shit.

• 39:18Amjad Masad?

  Like I say.

• 39:19Marc Andreessen?

  Right. And so there's this thing where, so the AI scientists are used to complaining, basically, that they're, that they're, they're being, they're always being judged against the next thing as opposed to all the things they've already, they've already solved.

• 39:30Marc Andreessen?

  But, but that's maybe the other side of it, which is they're also putting out for themselves.

• 39:34Amjad Masad?

  An unreasonable goal.

• 39:35Marc Andreessen?

  An unreasonable goal and then doing this sort of self-flagellation kind of along the way.

• 39:38Amjad Masad?

  That's right.

• 39:39Marc Andreessen?

  And, and I kind of wonder, yeah, I wonder kind of which way that cuts.

• 39:42Amjad Masad?

  Yeah, yeah. It's an interesting question. Like, I started thinking about this idea of like, it doesn't matter whether it's truly AGI. And the way I define AGI is that you put in a AI system in any environment and efficiently learns. You know, it doesn't have to have that much prior knowledge in order to kind of learn something, but also can transfer that knowledge across different domains. But, you know, we can get to like functional AGI. And what functional AGI is, is just, yeah, collect data and every useful economic activity in, in the world today. And train an L M on top of that, or train the same foundation model on top of that. And, and we'll go, we'll target every sector of economy and, and you can automate a big part of labor that way.

• 40:27Amjad Masad?

  So I think, I think, yeah, I think we're on that track.

• 40:30Marc Andreessen?

  Right.

• 40:30Amjad Masad?

  For sure.

• 40:30Marc Andreessen?

  Right.

• 40:31Erik Torenberg?

  You tweeted after G P T five came out that you were feeling the diminishing returns.

• 40:35Amjad Masad?

  Yeah.

• 40:35Erik Torenberg?

  What, what were you expecting and, but, and what needs to be done? Do we need another breakthrough to get back to the pace of growth or what are your thoughts on?

• 40:41Amjad Masad?

  I mean, this, this whole discussion is, is sort of about that. And, and my feeling is that, you know, G P T five got good at verifiable domains. It didn't feel that much better at anything else. The more human angle of it, it felt like it regressed and like you had this sort of Reddit pitchfork. Sort of movement against, against SAM and OpenAI because they felt like they lost a friend. G P T four oh felt a lot more human and closer.

• 41:12Amjad Masad?

  Whereas G P T five felt a lot more robotic, you know, very in its head, kind of trying to think through, through everything. And, and so I, I would have just expected like when we went from G P two, two to three. It was clear it was getting a lot more human. It was a lot closer to our experience. It can, you can feel like it's actually all it gets me. Like, there's something about it that understands the world better. Similarly, three to four, four to five, didn't feel like it was a better overall being.

• 41:46Marc Andreessen?

  But is that, is that, is that, is that a, is the question there like, is it emotionality? Is it?

• 41:53Amjad Masad?

  Partly emotionality, but again, partly, like, I like to ask models, like, very controversial things. Can it reason through, I don't know how deep I want to go here, but like, what happened with the World Trade Seven?

• 42:11Marc Andreessen?

  Right, right, sure.

• 42:12Amjad Masad?

  It's an interesting question, right? Like, I'm not, I'm not putting out a theory, but like, it's interesting. Like, how did it, you know, and can it, can it think through controversial questions? In the same way that it can go think through a coding problem? And there hasn't been any movement there, like, all the reasoning and all of that stuff. I haven't said, and not just that, you know, that's a cute example, but like, COVID, right? Like, you know, the origins of COVID.

• 42:39Marc Andreessen?

  Right.

• 42:39Amjad Masad?

  You know, go, you know, dig up GPT four or other models and go to GPT five. You're not going to find that much difference of, okay, let's reason together. Let's try to figure out what was the origins of COVID. Because it's still an unanswered question, you know? And I don't see them making progress in that. I mean, you play a lot with them, do you feel like?

• 42:57Marc Andreessen?

  Yeah, I use it differently. I don't know, maybe I have different expectations. I, I'm, I, the way I, my main use case actually is sort of, sort of PhD and everything at my beck and call. And so I'm, I'm trying to get it to explain things to me more than I'm trying to like, you know, have conversations with it, maybe.

• 43:10Amjad Masad?

  Yeah.

• 43:11Marc Andreessen?

  Maybe I'm just unusual with that, but.

• 43:12Amjad Masad?

  And that, that gets better.

• 43:13Marc Andreessen?

  Well, so what I, what I, what I found specifically is a combination of like GPT five Pro plus deep reasoning, or like GPT four heavy, like, you know, the, the highest end models like that. You know, they now basically generate thirty to forty page, you know, essentially books on demand on any topic. And so anytime I get curious about something, you're just taking it, maybe it's my version of it, but it's something like, I don't, like, here's a good example. When, when, when an advanced economy puts a tariff on, on a raw, you know, on a raw material or on a finished good, like, who pays? You know, is it, is it the consumer? Is it the, is it the importer? Is it the exporter? Or is it the producer? And then, and this is actually a very complicated, it turns out, very complicated question. It's a big, big, big thing that economists study a lot. And it's just like, okay, you know, who pays? And what I found like for that kind of thing is it's outstanding.

• 43:56Amjad Masad?

  Well, but it's outstanding at sort of going out of the web, getting information, synthesizing it.

• 44:02Marc Andreessen?

  Correct. It gives me, it gives me a synthesized 20, 30, 40 page, basically tops out of 40 pages, 40 pages of PDF.

• 44:08Amjad Masad?

  Yeah.

• 44:09Marc Andreessen?

  But I can get it, I get up to 40 pages of PDF, but it's a completely coherent. And as far as I can tell, for everything I've cross checked, a completely like, like world class, like if I hired, you know, for a question like that, if I hired like a great, you know, econ PhD postdoc at Stanford, who just like went out and did that work, like it would maybe be that good.

• 44:26Amjad Masad?

  Yeah.

• 44:26Marc Andreessen?

  But then, of course, the significance is it's like, it's like, you know, at least for, this is true for many domains, you know, kind of PhD and everything. And so.

• 44:33Amjad Masad?

  But this is synthesizing knowledge, not trying to create new knowledge.

• 44:36Marc Andreessen?

  Well, but this, this, this gets to the, the sort of, you know, of course, you get into the angels dancing on the head of a pin thing, which is like, what, you know, what's the difference? How many, how much new knowledge ever actually is there anyway? What do you actually expect from people when you ask them questions? And so what I'm looking for is like, yes, explain this to me in like the clearest, most sophisticated, most complex, most like complete way that is possible for somebody to, you know, for a real expert to be able to, to explain things to me. And that's what I use it for. And again, as far as I can tell from the cross checking, like I'm getting, you know, like almost like basically a hundred out of a hundred. Like, I don't even think I've had an issue in months where it's like, had a problem in it.

• 45:11Amjad Masad?

  Yeah.

• 45:11Marc Andreessen?

  And it's like, yeah, you can say, yeah, synthesizing is supposed to create new information, but like it's, it's generating a 40 page, it's basically generating a 40 page book.

• 45:18Amjad Masad?

  That's amazing.

• 45:18Marc Andreessen?

  That's like incredibly like fluid. It's, you know, it's, it's, it's, you know, the, the logical coherence of the entire, like it's, it's a great, right? Like if you, if you evaluated a, a human author on it, you would say, wow, that's a great author.

• 45:31Amjad Masad?

  Yeah.

• 45:32Marc Andreessen?

  You know, are people who write books, you know, creating new knowledge? Well, yeah, well, sort of not, because a lot of what they're doing is building on everything that came before them is synthesizing.

• 45:40Amjad Masad?

  Money?

• 45:41Marc Andreessen?

  But also like a book is a creative accomplishment, right? And so.

• 45:44Amjad Masad?

  Yeah. One of the things I'm, I'm, I'm interested in, I'm hoping AI could help us solve this. Just like how confusing the information ecosystem right now. You know, everything feels like propaganda, like it doesn't feel like you're getting real information from anywhere. So I, I really want an AI that could help me reason from first. Principles about what's happening in the world for me to actually get real information. And and maybe that's an unreasonable sort of ask of of the AI researchers, but but I don't think we're we have made any progress there.

• 46:14Marc Andreessen?

  So maybe I'm over focused, maybe I'm over maybe in my my line, or maybe I'm over focused on arguing people as opposed to trying to get as a trust, trying to get underlying truths. But well, here here's the thing I do a lot with this is I just say like, take a provocative point of view and then steel man the position. Take your COVID thing, steel man. So I often I often pair these steel man the position that it was a lab leak and the steel man the position that it was natural origins. And and again, like, is this creativity or not? I don't know. But like, what comes back is like 30 pages each of like, wow, like, that is like the most compelling case in the world I can imagine with like every, you know, everything marshaled against it, like the argument structured in like the most possible.

• 46:47Amjad Masad?

  Part of the reason that's hard happening is because it stopped being taboo to talk about a human origin. When it was taboo.

• 46:54Marc Andreessen?

  Yes.

• 46:54Amjad Masad?

  The the eyes would like talk, well, you know, talk down to you. It's like, oh, you're a conspiracy theorist. And so.

• 47:00Marc Andreessen?

  Yes.

• 47:00Amjad Masad?

  As there's a, you know, yes, period of time. And so it takes something truly controversial and they actually they can't reason about it because of all our L. A. Chefinons.

• 47:10Marc Andreessen?

  So some of the limitations and as you know, I will pick them specific ones here, but like there there are certain, certain big models that will still lecture you that you're a bad person for asking that question. But but you know, like, I was just there. Some of them are just like really, really open now to, you know, being able to do these things. And then, um, uh, yeah, so, um, okay. Yeah, so, okay, so, yeah, so there's this. Yeah, so so basically, like, ultimately, what you're looking for, like, the ultimate thing would be if there's something that's like, I don't think anybody's really defined this really well, because it's not, because, again, it's like, conventional, all the conventional definitions of AGI are like, basically comparing to people.

• 47:43Amjad Masad?

  Yeah.

• 47:43Marc Andreessen?

  And there, there, and there, it's always like, you know, it's, it's the conventional explanations of of of of AGI always, for me, strike me a lot like the debate around like, whether a self driving car works or not, which is, is, does a self driving car work because it's a perfect driver? Or does it work because it is is better than the human driver? And better than the human driver, I think, is actually quite, you know, just like with the chess thing and the go thing, I actually think like that.

• 48:06Amjad Masad?

  That's like a real thing. And then, and then, and then, and then there's the like, is it a perfect driver, which is, you know, what they're obviously the self-driving car companies are working for. But then I think you're looking for something beyond the perfect driver. You're looking for the car who like knows where to go. So I, I'm of two minds. So one mind is the sort of practical entrepreneur. And I just, I have so many toys to play with to build like, stop AI progress today, and Revit will continue to get better for the next five years. Like, wait, there's so much we can do just on the app, app layer and the infrastructure layer. So, you know, I, but, but I think that will, you know, the, the foundation models will continue to get better as well. And so it's in a very exciting time in our industry. The other mind is more academic, because as a kid, I've always been interested in the nature of consciousness, nature

• 48:51Amjad Masad?

  of intelligence. I was always interested in AI and reading the literature there. And I would point to the RL literature. So Richard Sutton, there's another guy, I think co-founder of DeepMind, Shane Lag, wrote, wrote a paper trying to define what AGI is. And in there, I think that the definition of AGI, I think, is the, is the original, perhaps correct one, which is efficient continual learning. Like, if you, if you truly want to build an artificial general intelligence that you can drop in any domain, you can drop in a car without that much prior knowledge about cars. And within, you know, how long does it take a human to, to learn how to drive? Within months, be able to drive a car really well.

• 49:36Marc Andreessen?

  Right. So generalized skill, sort of generalized skill acquisition, generalized understanding acquisition, generalized reasoning acquisition.

• 49:44Amjad Masad?

  And I think that's the thing that will, like, truly change the world. That's the thing that would give us a better understanding of, of the human mind, of human consciousness. And that's the thing that will, like, propel us to the next level of human civilization, I think. So, so on a civilization level, I think that's, that's a really deep question. But separate from the economy and the industry, which is all exciting. But, but there's an academic aspect of it that I'm really.

• 50:10Marc Andreessen?

  And so what, and what odds, if we're on, if we're on CalShe today, what, what odds do we place on that?

• 50:14Amjad Masad?

  I, I, I'm kind of bearish on, on, on true AGI breakthrough, because what we built is so useful and economically valuable. So, in a way.

• 50:27Marc Andreessen?

  Good enough, good enough is the enemy.

• 50:29Amjad Masad?

  Yeah, yeah. Do you remember that essay? Worse is better. Worse is better. Worse is better. Worse is better. And.

• 50:35Marc Andreessen?

  So there's like a local, there's like a trap. There's like a local, local maximum trap.

• 50:38Amjad Masad?

  We're in a local maximum.

• 50:39Marc Andreessen?

  Local maximum trap where it's because it's, because it's good enough for so much economically productive work.

• 50:44Amjad Masad?

  Yes.

• 50:45Marc Andreessen?

  It relieves the pressure in the system to create the generalized answer.

• 50:49Amjad Masad?

  Yes. And then you have the weirdos like Rich Sutton and others that are still trying to go that down that path. And maybe they'll succeed.

• 50:55Marc Andreessen?

  Right.

• 50:55Amjad Masad?

  But there's enormous optimization energy behind the current thing that we're hell climbing on this like local maximum.

• 51:01Marc Andreessen?

  Right, right, right. And the irony of it is, everybody's worried about like the, you know, the gazillions of dollars going into building out all this stuff. And so the, the, the, the most ironic thing in the world would be if the gazillions of dollars are going into the local maximum.

• 51:11Amjad Masad?

  That's right.

• 51:12Marc Andreessen?

  As, as opposed to a counterfactual world in which they're going into solving the general problem.

• 51:15Amjad Masad?

  But, but it's also potentially irrational. Like maybe the general problem is actually, you know, not within our lifetimes. Who knows?

• 51:22Erik Torenberg?

  How, how much further do you think, like, do you think we squeezed most of the juice out of, out of L L M's in general then? Or are there any other research directions that you're particularly excited about?

• 51:32Amjad Masad?

  Well, that, that's the thing. I think the problem is there aren't that, that many. I, I think the, the, the breakthroughs in R L are incredibly exciting, but we also knew about them now for like over 10 years. Where you marry generative systems with, with tree search and things like that. But, but there's a lot more to go there. And I think, again, the, the, the, the original minds behind reinforcement learning are trying to go down that path and try to kind of bootstrap intelligence from scratch. Carmack is, is going down that path. As, as far as I understand Carmack, you guys may be invested, but the, the, you know, they're, they're not trying to go down the L L M path. So there are people that are trying to do that, but I'm not seeing a lot of progress or outcome there. But I, I watch it kind of from far.

• 52:17Marc Andreessen?

  Although, you know, for all we know, it's already, there's already a bot on X somewhere.

• 52:20Amjad Masad?

  Maybe.

• 52:21Marc Andreessen?

  You know, you know, you never, you never know. It might not be a big announcement. It might just be a, you know, one day there's just like a bot on X that starts winning all the arguments.

• 52:27Amjad Masad?

  Yeah, it'd be.

• 52:28Marc Andreessen?

  Or a code, as I, a code, a user Reddit and all of a sudden it's.

• 52:32Amjad Masad?

  Yeah.

• 52:33Marc Andreessen?

  Generating incredible software. Um, okay. Let's, uh, let's spend our remaining minutes. Let's, let's, let's talk about you. So, uh, so, uh, so how, so yeah, take us and start from the beginning with your, with your life. And how, how did you get, how did you get from being born and being in Silicon Valley?

• 52:45Amjad Masad?

  Okay.

• 52:47Marc Andreessen?

  In two minutes.

• 52:48Amjad Masad?

  Yeah.

• 52:48Marc Andreessen?

  I'm just joking, but.

• 52:50Amjad Masad?

  Yeah, I got into just computers very, very early on. And so for whatever reason, so I was born in Amman, Jordan. And for whatever reason, my dad, who was just a government engineer at the time, decided that computers were important. And he didn't have a lot of money, took out a debt, bought a computer. It was the first computer in our neighborhood, first computer of anyone I know. And I just, one of my earliest memories, I was six years old, just watching my dad unpack this machine and sort of open up this huge manual and kind of finger type CD, LS, MKDIR. And like, I would, you know, be behind his shoulder and just like watching him,

• 53:33Amjad Masad?

  you know, type these commands and seeing the sort of machine kind of respond and do exactly what he's asked it to do.

• 53:41Marc Andreessen?

  Pop in Tylenol.

• 53:44Amjad Masad?

  Exactly.

• 53:46Marc Andreessen?

  Autism activated.

• 53:47Amjad Masad?

  Of course. You have to.

• 53:50Amjad Masad?

  You have to.

• 53:52Marc Andreessen?

  What kind of, what kind of computer was it?

• 53:56Amjad Masad?

  It was an IBM, as far as I remember.

• 53:59Marc Andreessen?

  IBM PC.

• 54:00Amjad Masad?

  It was IBM PC.

• 54:01Marc Andreessen?

  So what year was this?

• 54:02Amjad Masad?

  DOS, nineteen ninety-three.

• 54:03Marc Andreessen?

  Nineteen ninety-three. Okay, so DOS, so did it have Windows at that point?

• 54:06Amjad Masad?

  No, it didn't have Windows.

• 54:08Marc Andreessen?

  It was right before Windows.

• 54:09Amjad Masad?

  Right before Windows, but I think Windows had been out, but you would.

• 54:12Marc Andreessen?

  It was an add-on.

• 54:12Amjad Masad?

  It was an add-on. You wouldn't boot it up. So I think we bought the disk for Windows and you had to kind of bootload it, you know, from the disk and it will open Windows and you can click around. It wasn't that interesting because there wasn't a lot on it. So a lot of time I just spent in DOS and writing batch files and opening games and messing around with that. But it wasn't until Visual Basic that I started. So like after Windows ninety-five, that I started making real software.

• 54:41Marc Andreessen?

  Right.

• 54:41Amjad Masad?

  And the first idea I had was, I used to be a huge gamer. So I used to go to these LAN gaming cafes and play Counter-Strike. And I would go there and, you know, the whole thing is full of computers, but they don't use any software. Business. It was just like people running around, just like writing down that your machine number, how much time you spent on it and how much did you pay and kind of tapping your shoulders like, hey, you need to pay a little more for that. And I asked him like, why don't you like just build a piece of software that allows me to log in and have a time or whatever. And I was like, yeah, we don't know how to do that. And I was like, okay, I think I know how to do that. So I spent, I was like 12 or something like that. I spent like two years building that and then went out and tried to sell

• 55:23Amjad Masad?

  it and was able to sell it and was making so much money. I remember McDonald's opened in Jordan around the time when I was 13, 14. I took my entire class to McDonald's. It was very expensive, but I was bawling, you know, all this money and I was showing off. And, and so that was the first business that I created. And then when it came to, and at the time I started kind of learning about AI, you know, reading sci fi and all of that stuff. And when it came time to go to college, I didn't want to go to computer science because I felt like coding is on its way to get automated. I remember using these wizards. Do you remember wizards?

• 56:06Marc Andreessen?

  Yes, wizards, basically. It's like extremely crude early bots.

• 56:10Amjad Masad?

  That generated code.

• 56:11Marc Andreessen?

  That generated code, yeah.

• 56:12Amjad Masad?

  Yeah, I remember you could like, you know, type in a few things like, here's my project, here's what it does, whatever, and then click, click, click, and which is like scaffold a lot of code. I was like, oh, I think that's the future. Like coding is such a.

• 56:23Marc Andreessen?

  Double solved.

• 56:23Amjad Masad?

  Yeah, it's solved, you know, why, why should I go into coding? I was like, okay, if AI can do the code, what should I do? Well, someone needs to build and maintain the computers. And so I went to the computer engineering and, and did that for a while. But then rediscovered my love for, for programming, reading polygon masses on Lisp and things like that. And I started messing around with scheme and programming languages like that. But then I found it incredibly difficult to just like learn different programming languages. I didn't have a laptop at the time. And so every time I go to like wanting to learn Python or Java, I would go to the computer lab. Download gigabytes of software, try to set it up, type a little bit of code,

• 57:06Amjad Masad?

  try to run it, you know, run into missing DLL issue. Or I was like, man, this is so primitive. Like at the time, it was 2008, something like that. You know, we had Google Docs, we had Gmail, you could like open the browser. And partly thanks to you and be able to kind of use software on the internet. And I thought the web is the ultimate software platform. Like everything should go on the web. Okay, who's building an online development environment? And no one. Right. And it felt like I found like a hundred dollar bill on the, you know, on the floor of Grand Central Station. Like surely someone should be building this. But no, no one was building this. And so it's like, okay, I'll try to build it.

• 57:49Amjad Masad?

  And I got something done in like a couple of hours, uh, which was a text box. You type in some JavaScript, we, and there's a, there's a button that says eval. You click eval and evaluates, shows you in a, in an alert box. So one plus one, two. I was like, oh, I have a programming environment. I showed it to my friends, people started using it. I added a few additional things like saving the program. I was like, okay, all right, this is, there's, there's a real idea here. People love it. And then again, it took me two, two or three years to actually be able to build anything. Because, you know, the browser can only run JavaScript. And it took a breakthrough at the time. Mozilla had a resource project called M Scriptin that allowed you to compile

• 58:32Amjad Masad?

  different programming languages like C, C++ into JavaScript. And for the browser to be able to run something like Python, I needed to compile C, Python, or JavaScript. So I was the first to do it in the world. So built, contributed to that project and built a lot of the scaffolding around it. And we, uh, my friends and I compiled Python into JavaScript. And I was like, okay, we did it for Python, let's do it for Ruby, let's do it for Lua. So, and that's how the emergence of the idea for REPL it came is that when you need a REPL, you should get it, you should REPL it. And so REPL is the most primitive programming environment possible. So I added all these programming languages. And again, all this time, my friends were using it and excited about it. And I was on GitHub at the time.

• 59:17Amjad Masad?

  And just my standard thing is like when I make a piece of software, just open source it. And so I was open sourcing all the things. I was, you know, years building just like this underlying infrastructure to be able to just run code in the browser. And then it went viral, went viral on Hacker News and it coincided with the MOOC era. So massively online courses, Udacity was coming online, Coursera and, and most famously, Code Academy. So Code Academy was the first kind of website that allowed you to code in the browser interactively and learn how to code. And they built a lot of it on my software that I was open sourcing all the way from Jordan. And so I remember seeing them on Hacker News and they were going super viral. And I was like, hey, that's, you know, I recognize this, what are you using? And so I left the Hacker News comments. I was like, oh, you're

• 1:00:00Amjad Masad?

  using my, my open source package. And so they reached out to me. They, they're like, hey, would like to hire you. I was like, I'm not interested. I want to start a startup. I want to start this thing called Replit. And, and they're like, well, no, you know, you should come work with us. We can, we can do the same stuff. And I kept saying, no, I was like, okay, I'll contract with you. They were paying me $12 an hour. I was really excited about it. Back from Oman. But they came out to their, to their credit. They came out of Jordan to recruit me and spent a few, a few days there. And then, you know, I kept saying no. And then at the end, they gave me an offer I can't refuse. And they got me an O1 visa, came to the United States.

• 1:00:36Marc Andreessen?

  That's when you moved. So when, when was the first, because you were born, what year?

• 1:00:40Amjad Masad?

  1987.

• 1:00:40Marc Andreessen?

  Eighty seven. What was the first year that you could remember where you had the idea that you might not live your life in Jordan and you might, you might actually move to the US?

• 1:00:47Amjad Masad?

  When I watched Pirates of Silicon Valley.

• 1:00:49Marc Andreessen?

  Is that right? Okay, got it. All right.

• 1:00:51Amjad Masad?

  Maybe 98 or 99. I don't know when it came out.

• 1:00:55Marc Andreessen?

  That might be a good place to.

• 1:00:56Amjad Masad?

  Yeah.

• 1:00:57Erik Torenberg?

  Is it worth telling the hacker story? Because there's a version of the world where you didn't actually, like, if that changed differently, maybe you wouldn't have gone to America.

• 1:01:03Amjad Masad?

  Right, right. Yeah. So in, in school, I was programming the whole time. So I, I just want to start businesses. I just like, I'm exploding with ideas all the time. And like, the reason Replit exists is because I have ideas all the time. I just want to go type on the computer and like, build them. So I wasn't going to school. It was like incredibly boring for me. And part of the reason why Replit has a mobile app today is because I always wanted to program under the desk. So I just do things. And so the, at school, they kept failing me for attendance. So I would get As, but I just didn't show up. And so they, they would fail me. And so I felt that was incredibly unfair. And all my friends were graduating now. This year is was like 2011. I'd been like for

• 1:01:45Amjad Masad?

  six years in college. It should be like a three or four year. And I was like incredibly depressed. I really wanted to be in Silicon Valley. And so. I was like, oh, what if I change my grades?

• 1:01:58Marc Andreessen?

  There we go.

• 1:01:58Amjad Masad?

  The university database.

• 1:02:02Amjad Masad?

  And so I went into my parents' basement and implemented the polyphasic sleep. Are you familiar with that?

• 1:02:13Marc Andreessen?

  I am.

• 1:02:15Amjad Masad?

  Leonardo da Vinci's polyphasic sleep. I didn't hear from Leonardo da Vinci. I heard it from Seinfeld because there's an episode where John Kamer goes on polyphasic sleep.

• 1:02:24Marc Andreessen?

  What, 20 minutes every four hours?

• 1:02:26Amjad Masad?

  Yes, 20 minutes every four hours.

• 1:02:28Marc Andreessen?

  Yeah. And yes, and this somehow is going to work well.

• 1:02:30Amjad Masad?

  Yeah, and hacking, if you've ever done any-

• 1:02:33Marc Andreessen?

  As the meme goes, this has never worked for anybody else, but it might work for me.

• 1:02:37Amjad Masad?

  Yes.

• 1:02:39Amjad Masad?

  And a lot of what hacking is, is that you're coming up with ideas for like finding certain security holes and like writing a script and then running that script. And that script will take like a 20, 30 minutes to run. And so you'll take that, you know, 20, 30 minutes to sleep and go on. So I spent two weeks just going mad, like trying to hack into the university database. And finally, I found a way, I found a SQL injection somewhere on the site. And I found a way to like be able to edit the records. But I didn't want to risk it. So I went to my neighbor who was going to the same school. I think till this day, no one caught him. But I went to him and I said, hey, I have this way to change grades. Like, would you want to be my guinea pig? And I was honest about it. I was like, I'm not going to do it. Are you open to do it?

• 1:03:24Amjad Masad?

  He's like, yeah, yeah.

• 1:03:25Marc Andreessen?

  They call this human trials.

• 1:03:27Marc Andreessen?

  This is how medicine works.

• 1:03:30Amjad Masad?

  So we went and we went and changed his grades. And he went and pulled his transcript and the, you know, the update wasn't there. And went back to the basement. Well, it turned out that I had access to the slave database. I didn't have access to the master database. So find a way through the network, privileged escalation. It was an Oracle database that had affordability. And then found the real database. And then I just, you know, did it for myself, changed the grades and went and pulled my transcripts. And sure enough, it actually changed. Went and bought the gown, went to the graduation parties, did all that. We're graduating.

• 1:04:13Amjad Masad?

  And then one day I'm at home, it's like maybe six or seven PM. I get a, you know, the telephone at home rings. I'm gonna, I'm gonna ring something. Yes. Well, hello. And he's like, hey, this is the university registration system. And I knew the guy that run it. He's like, look, you know, we, we're having this problem. The system's been down all day and it keeps coming back to your record. There's an anomaly in your record where you're both pass, you have a passing grade, but you're also banned from that final exam with subject. I was like, oh shit. Well, it turns out the database is not normalized. So typically that when they ban you from an exam, the, the grades resets to thirty five out of a hundred. But apparently there's a Boolean flag.

• 1:04:58Amjad Masad?

  And by the way, all the column names in the database are single, are single letters. So that was the hardest thing, you know, security by obscurity. Right. And turns out there's a flag that I didn't check. So when, when, when you go over attendance, um, yeah, when you don't attend and they, they, they wanna fail you, they, they ban you from the final exam. So I changed the grades and that, that, that created a, an issue and brought down the system. So they were calling me and I thought at the time, I was like, you know, I could, I could potentially lie and it'll be a huge issue or I just like, I'll just, I'll just. So I said, hey, listen, look, um, yeah, I might know something about it. Hey, let me, let me come, uh, tomorrow and kind of talk to you about what happened.

• 1:05:43Amjad Masad?

  So I go in and I open the door and it's the deans of all the, all those schools. It's like computer science, computer, they were all working on it for like days because it's like, it's like, it's a very computer heavy, you know, university and it was like a problem. And they're all kind of really intrigued about what happened. And so I pull up a whiteboard and started explaining what I did and, and everyone was engaged. I gave them a lecture, basically. Your oral exam for your Ph D. Yeah. This is great. They were, they were, they were really excited and, uh, and I think it was endearing to them. I was like, oh wow, this is a, this is a very interesting problem. And then I was like, okay, great, thank you. And I, I'm hitting out, I was like, hey, wait. Wait, we don't know what to do with you. Yeah. Do we send you to jail?

• 1:06:27Amjad Masad?

  And I was like, hey, we have to escalate to the university president. And he was a great man. And I think he gave me a second chance in life. And I went to him and I, you know, I explained the situation. I said, like, I'm really frustrated. I need to graduate. I need to get on with my life. I've been here for six years and I just can't sit in in in school. The stuff I already know, I'm a really good programmer. And and he gave me a Spider-Man line at the time. It's like, with the great power comes great responsibility. And you have a great power. And you know, and it really affected me. And I think he was right at the moment. And and so he said, well, we're gonna let you go, but you're gonna have to help the system administrator secure the system. There we go. For the summer. I was like, happy to do it.

• 1:07:11Amjad Masad?

  And I show up and all the programmers there hate me. Yeah. Hate my guts. Yes, 100%. And they would lock me out. Like, I would see them, they would be outside. I would knock on the door and no one would listen. It was like, they don't want to let me in. I tried to help them a little bit. They weren't collaborative. And so I was like, all right, whatever. And so it came time for me to actually graduate. It was the final project. And one of the computer science dean came to me and he said, look, I need to call a favor. I was a big part of the reason we kind of let you go and we didn't kind of prosecute you. So I want you to work with me on the on the final project. And it's gonna be around security and hacking. I was like, no, I'm I'm done with that shit. I just wanna I just wanna build programming environments and things like that. And he's like, no, you have to do it.

• 1:07:56Amjad Masad?

  I was like, OK. So I thought I'd do something more productive. So I wrote a security scanner that I was very proud of, that that kind of crawls a different side, that tries to do SQL injection and all sorts of things. And actually, my security scanner found another vulnerability in the system. Amazing. And so I went to the defense and he's like, you need to run this security scanner live and show that there's a vulnerability. And I didn't understand what was going on at the time, but I just, OK. So I give the presentation about how the system works. And I was like, oh, let's run it. And it showed that there's a security vulnerability. OK, let's get, let's try to get a shell. So the system automatically runs all the security stuff and gets you gets you a shell. And then the other dean, that turned out he was giving the mandate to secure the system.

• 1:08:40Amjad Masad?

  And now I started to realize I'm a pawn in some kind of. Bribery here and and his his face turned red and it's like, no, it's impossible. You know, we secure the system. You're lying. I was like, you know, you're accusing me of lying. All right, what should we know? Should we know your um your salary or your password? What do you want me to look up? And I was like, yeah, I look up my password. So I I look up his his password. Uh, and it was like Gyp Rashid was encrypted. And I was like, oh, that's not my password. See, you're lying. I was like, well, there's a decrypt function that programmers put it there. So I do decrypt and it shows his password. And it was something embarrassing. I forgot. I forgot what it was. And so he gets up really angry, shakes my hand and leaves to change his password.

• 1:09:25Amjad Masad?

  Uh, so that I I was able to hack into the university another time. Luckily, I I was able to graduate, give them the software, they secured the system. But um, but yeah, later on, I would realize that, yeah, he wanted to embarrass the other guy, which is why I was in the middle.

• 1:09:40Marc Andreessen?

  I think the moral, the moral of the story is if if you can successfully hack into your school system and change your grade, you deserve the grade and you deserve to graduate.

• 1:09:48Amjad Masad?

  I I think so.

• 1:09:49Marc Andreessen?

  And and and just for any for any parents out there, just your children out there, you can you can cite, you can cite, cite me.

• 1:09:54Amjad Masad?

  Marc Andreessen.

• 1:09:54Marc Andreessen?

  As the moral, you can cite, you can cite on me as the moral, moral authority, moral authority in this.

• 1:09:58Amjad Masad?

  One maybe lesson, I think that is very relevant for the AI age. Uh, I think that the traditional sort of more conformist path is paying less and less dividends. And I think, uh, you know, kids coming up today should use all the tools available to be able to discover and chart their own paths because I feel like just, you know, listening to the traditional advice and doing the same things that people have always done is just not as it's not working out as much as as we'd like.

• 1:10:31Marc Andreessen?

  Yeah, that's right.

• 1:10:31Amjad Masad?

  Thanks for coming to the podcast.

• 1:10:33Marc Andreessen?

  Thank you. Fantastic.

• 1:10:36Speaker 02

  Thanks for listening to this episode of the A16Z Podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast.

• 1:11:20Speaker 02

  For more details, including a link to our investments, please see a16z.com/disclosures.