Amjad Masad - The Future of Software Creation - [Invest Like the Best, EP.310]

My guest today is Amjad Masad. Amjad is the founder and CEO of Replit, whose mission is to bring the next billion software creators online. Replit has built a browser-based coding environment that makes coding more fun, collaborative, and approachable. We discuss how that is possible and why the way most of us interact with computers today is suboptimal. We then go into the effects of AI on software creation and its broader impacts on technology. Please enjoy my conversation with Amjad Masad. For the full show notes, transcript, and links to mentioned content, check out the episode page here. ----- This episode is brought to you by Tegus. Tegus streamlines the investment research process so you can get up to speed and find answers to critical questions on companies faster and more efficiently. The Tegus platform surfaces the hard-to-get qualitative insights, gives instant access to critical public financial data through BamSEC, and helps you set up customized expert calls. It’s all done on a single, modern SaaS platform that offers 360-degree insight into any public or private company. I’ve been so impressed by the platform that my firm, Positive Sum, recently made an investment in Tegus. We did so because we feel that Tegus will be the gold standard platform for investing research for decades to come. As a listener, you can take Tegus for a free test drive by visiting tegus.co/patrick. ----- Invest Like the Best is a property of Colossus, LLC. For more episodes of Invest Like the Best, visit joincolossus.com/episodes. Past guests include Tobi Lutke, Kevin Systrom, Mike Krieger, John Collison, Kat Cole, Marc Andreessen, Matthew Ball, Bill Gurley, Anu Hariharan, Ben Thompson, and many more. Stay up to date on all our podcasts by signing up to Colossus Weekly, our quick dive every Sunday highlighting the top business and investing concepts from our podcasts and the best of what we read that week. Sign up here. Follow us on Twitter: @patrick_oshag | @JoinColossus Show Notes[00:02:18] - [First question] - The Steve Jobs black-pill [00:06:02] - Speculation on the near future of programming [00:09:38] - Potential convergence of simple software and coding tools [00:11:23] - What an IDE is and how it works [00:12:44] - The definition of REPL and the role of Replit in the space [00:14:21] - Decreasing friction in a programming environment using primitives [00:19:47] - Real-world effects of Replit’s low-friction design [00:23:27] - His perspective on new coding and AI technologies [00:30:29] - Promises and limitations of the user-friendly programming movement [00:33:16] - The dynamic nature of IDE technology and its challenges [00:39:53] - How he’s priming his team to react to new technologies like the upcoming GPT-4 [00:43:58] - Recommended skills and training for the AI world of the future [00:47:21] - The impact of IDE and AI tech innovations on existing tech giants [00:51:56] - His mixed but optimistic views on the trajectory of AI [00:54:40] - Recommendations for the curious listener without a programming background [00:56:50] - The role of smartphones in the IDE movement [00:58:28] - The kindest thing anyone has ever done for him

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Patrick O'Shaughnessy?

  I know firsthand how complex the tech stack is for asset management firms. And seemingly every new tool and data source makes the problem even worse, adding more complexity, more headcount and more risk. RidgeLine offers a better way forward. One unified platform that automates away the complexity across portfolio accounting, reconciliation, reporting, trading, compliance and more, all at scale. RidgeLine is revolutionizing investment management, helping ambitious firms scale faster, operate smarter and stay ahead of the curve. See what RidgeLine can unlock for your firm. Schedule a demo at ridgeline.ai. This episode is brought to you by Tegus. Over the years of our partnership with Tegus, they have evolved from a pure expert network into a full company intelligence platform. I've been so impressed by the platform that my firm, Positive Sum, recently made an investment

• 0:45Patrick O'Shaughnessy?

  in Tegus. We did so because we feel that Tegus will be the gold standard platform for investing research for decades to come. Tegus streamlines the investment research process so you can get up to speed and find answers to critical questions on companies faster and more efficiently. The Tegus platform surfaces the hard to get qualitative insights, gives instant access to critical public financial data through BAM SEC and helps you set up customized expert calls. It's all done on a single, modern SaaS platform that offers 360 degree insight into any public or private company. As a listener, you can take Tegus for a free test drive by visiting tegus.co/patrick.

• 1:25Patrick O'Shaughnessy?

  Hello and welcome, everyone. I'm Patrick O'Shaughnessy and this is Invest Like the Best. This show is an open-ended exploration of markets, ideas, stories and strategies that will help you better invest both your time and your money. Invest Like the Best is part of the Colossus family of podcasts and you can access all our podcasts, including edited transcripts, show notes and other resources to keep learning at joincolossus.com.

• 1:50Speaker 01

  Patrick O'Shaughnessy is the CEO and founding partner of Positive Sum and the CEO of O'Shaughnessy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of Positive Sum or O'Shaughnessy Asset Management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of Positive Sum or O'Shaughnessy Asset Management may maintain positions in the securities discussed in this podcast.

• 2:22Patrick O'Shaughnessy?

  My guest today is Amjad Masad. Amjad is the founder and CEO of Replit, whose mission is to bring the next billion software creators online. Replit has built a browser-based coding environment that makes coding more fun, collaborative, and approachable. We discuss how that is possible and why the way most of us interact with computers today is suboptimal. We then go into the effects of AI on software creation and its broader impacts on technology. Please enjoy my conversation with Amjad Masad.

• 2:51Patrick O'Shaughnessy?

  So, Amjad, I was trying to think of a fun place to begin, and I've heard you talk elsewhere about the Steve Jobs Black Pill, which I think is such an interesting idea. Everyone loves Steve Jobs. Obviously, he was incredible, incredible designer. All the great things everyone says about him are true. But what is the Steve Jobs Black Pill interpretation?

• 3:09Speaker 02

  So, Black Pill and Internet nomenclature, if you're terminally online, you would know that from the matrix, you get the green pill and red pill. Red pill is sort of reality. The Internet has expanded these concepts further. So now we have a white pill. White pill is like something that is optimistic, and a black pill is something that is pessimistic or bad. So Steve Jobs' legacy is obviously hugely positive, but I think there's some dark aspects of his impact that wasn't fully explored. And I would say our relationship to computing is possibly changed forever because of his impact, again, positively

• 3:54Speaker 02

  and negatively. I'm going to focus on the negative because that's what you asked me for. But computers in the early PC revolution, and even before that, were not seen as this purely entertainment consumption devices. Even Steve Jobs said, "Computer's the bicycle for the mind." It is this thing that helps you think. Where we are today is that a lot of people use computers and by computers I mean broadly phones, anything that has a processor in a television-like manner. I think most people are consumers. Even when you sit on the desktop and you do your work, there's an aspect of it where you're actually just consuming someone else's software.

• 4:39Speaker 02

  In the past, when you sat down, say in the seventies or eighties, when you sat down on your PC, you were writing programs, you were writing whatever it is, DOS, Command, Lisp, Fortran. Every professional that learned how to do computers and use software for work were making and creating software. Whereas today, even professionals are at the mercy of the designers of the software they're creating. Anytime you hit a limit with your software, you have two choices. You either go on Twitter and complain, or you go switch to another piece of software. Whereas in the past, every piece of software is programmable in some sense.

• 5:21Speaker 02

  So the Steve Jobs black belt is that he made computers so easy as to condition people that the relationship between them and their devices is a purely consumptive relationship and a relationship where you're dependent on the software creator and you don't really have the agency to make better use of your data, of your software, of your device. I mean, just a simple example, you buy a thousand dollar iPhone and say you want to program this thing. You just want to write code for it, you just bought this very expensive computer. Well, then you have to buy another expensive computer, you have to buy a two thousand, three thousand dollar MacBook. And it was like, okay, now I want to program my iPhone.

• 6:06Speaker 02

  It's like, no, you have to now pay Apple a hundred dollars for a developer license. All right, okay, now I paid a hundred dollars, so now we're up four thousand, one hundred dollars up. And it's like, okay, I want to program my iPhone. It's like, okay, you can program your iPhone, but you can't publish your software until you ask for our permission. And now you have to wait for their permission. So that's the Steve Jobs black belt. It's like our relationship to computers have changed fundamentally because of his impact. And now we're mostly consumers of computers.

• 6:35Patrick O'Shaughnessy?

  So if you have anything to say about it, what will change over the next 10 to 20 years? What could the future of software be? Maybe harkening back to that distant past, where it was much more interactive and creative, I guess is the right word to use, where I've got this powerful thing in my pocket. It does whatever I tell it to do versus me choosing from a menu of stuff that I can do.

• 6:56Speaker 02

  There are multiple answers to this. There's the present and then there's the future. And we're seeing glimpses of the future right now. But let me focus on the present first. The sort of white pill today is there's a generation of startups that started in the mid aughts in the past decades that intuited these problems. And Notion, Airtable, every kind of no-code tool company, we started with this premise. I mean, if you go look at Notion's About page, they talk about this problem. And you can extend this to Figma and a lot of these creative prosumer tools as well. So that's been really awesome, that for the first time, this generation of entrepreneurs that want to create software that's fundamentally end-user programmable

• 7:40Speaker 02

  and gives users more freedom. That's one optimistic note. But I don't think it goes far enough. I think you're still fundamentally limited. You need a full programming experience to actually experience the full capacity of computers. You have things like Replit now, where Replit is, in a lot of ways, when we started the call, you called me a traditionalist because I used to live in Silicon Valley. And there's more traditionalist things about me. Another example is, I actually think that one of the golden age of computers was when you boot it up a computer and the first thing that you saw was a command line prompt. So the DOS, the Commodore 64 era of computing was a fundamentally programmable,

• 8:25Speaker 02

  user freedom enhancing era of computing. So I think Replit, in a lot of ways, is going back to that era, where just you open a browser tab and you get this prompt, where it's like you can make any kind of software in the world. The thing is, we're not deluded at Replit. We don't think that everyone's world will just spend all the time writing software. It is still somewhat of a niche scale. We think a lot more people could learn it. And a lot of people are learning it. And our success at Replit kind of shows that. But I actually think that there's another phase shift that is happening now, which is on top of code, we are finally able to layer a natural language interface, which is now you get into generative AI.

• 9:10Speaker 02

  So G P T three knows how to write code better than most people. Deep Mind just submitted a paper called Alpha Code, where they beat something like fifty percent or more than fifty percent. The computer did better than average at programming competitions like lead code, and it's getting better really quickly. I think that by next year, AI will be better at these algorithmic interview type questions than any human. So basically, computers are going to get to a point where they're as good as people at generating code. So humans will do what humans do best, which is we're going to be creative with code. We're going to be creative with software.

• 9:54Speaker 02

  At Replod, we think that we want to make programming more accessible to everyone. We also want to create this layer of AI that allows even the ostensibly non-technical person to be able to generate and create software in the same way that a professional programmer would.

• 10:11Patrick O'Shaughnessy?

  It almost makes me think of like a word processor, like anyone can open a word processor and write an essay or write whatever they want. It's not hard. There's very little friction. You have to know how to type, I guess, and speak a language. It sounds like that's the convergence you're talking about, that if I can write an essay, I should be able to write a program in the same sense that most people write bad essays. So writing a good essay or a good program is a very different question. Am I thinking about that the right way? Is that sort of the convergence that you would hope to drive?

• 10:38Speaker 02

  I think at the limit, a Google Doc, a JavaScript file, and a chat G P like interface are all the same thing. You're creating knowledge, you're computing, you're querying, you're visualizing, you're doing all this thing within the same document. I don't know what the design of this thing looks like, but I think these things will converge. So you have the notebook style, which is like Google Colab, Jupiter. These are tools for data scientists where they use Python to generate graphs and do operations on data. And then you have the regular IDE like Replid where you're writing code and running that code.

• 11:19Speaker 02

  And then you have a chat interface where you're talking to most famous now ChatGPT. You're talking to some agents, could be a person by the way, where they're generating knowledge for you. They're querying the internet for you. They're doing menial text generation tasks for you. They've even generating images for you. And I think if you put all these things together, I think you're going to have a very powerful software system that blurs the distinction between prose, code, knowledge generation, knowledge querying and all these things. Describe what an IDE is for people that don't know that term. So an IDE stands for Integrated Development Environment.

• 12:04Speaker 02

  It's basically a programming environment. So basically, when you're programming, you are obviously typing text into a text editor, similar to like a word processor. There are fundamental differences between a code editor and a word processor in that the grammar in the code editor is a lot more strict. So we have things like spell check, but they're fundamentally different than the prose spell check or grammar check. If you misspell in a Google Doc, another human reading it will like understand what you're trying to say. If you misspell in code, the program will not run. Therefore, the tools that you need to have around that environment

• 12:49Speaker 02

  tend to be different and more algorithmic and more strict. So an IDE is a set of tools around a text editor that gives you autocomplete, checks your code for errors, check your code for the equivalent of a grammar. We call this linting for some reason. Allows you to run your code, allows you to bring libraries from GitHub. Basically, a never-ending set of tools that helps you write software.

• 13:17Patrick O'Shaughnessy?

  And where does the name Replit come from in the IDE context?

• 13:22Speaker 02

  Replit actually started as the opposite of IDE. In the same way that Word became this monstrosity of a piece of software, and it crashes all the time, and it has this triple chin of a toolbar. IDEs tend to grow in the same way, and they become unwieldy and really hard to learn. The opposite of a word processor is something like a notepad. Replit started more on the notepad end, which is called a REPL. So REPL stands for Read, Eval, Print, Loop, R E P L, REPL. And that is actually a program in a language called Lisp that bootstraps a programming environment, a very simple notepad-like programming environment.

• 14:06Speaker 02

  So it's a single line of code that bootstraps a programming environment. And the way it does that, read is a command that reads code from the user. Eval is a command that evaluates that code. Print prints the results of that evaluation. Loop restarts the process. Almost like a mystical line of code, because that line of code bootstraps an entire programming environment on top of it. So I love that concept and love this idea of a REPL. The idea behind Replit came from, if you were to use REPL as a sort of verb, like Replit, that became the name of the company. Of course, Replit became more of an IDE now, although it's a lot simpler than traditional IDEs, and it still functions like a REPL.

• 14:49Speaker 02

  A lot faster to start, a lot simpler to use. You can use it on your phone, so on.

• 14:54Patrick O'Shaughnessy?

  It seems like, again, for you to achieve the mission of just a lot more people being able to do this more code-based creation, that the friction to get going in something like your system needs to ultimately feel like a doc to the normal person out there. How do you do that? How are you building something that has the simplicity of a notepad? Maybe that's, it can't be that simple, but feels simple enough and feels non-intimidating. So that, let's say, someone like me who's done, you know, some work in the past. Like, I'm smart enough, I can figure things out, but I don't have a formal computer science training or something. So what are the key pieces or building blocks that you think will allow you and others to unlock that access for normal people like me?

• 15:38Speaker 02

  At Replit, we're fans of this primitive concept. In programming, computing and science, I guess, a primitive is a sort of basic data type or basic operation or basic object for which you can build up more complex objects. So for example, computers are actually made of a single object. Do you know what that object is?

• 16:01Patrick O'Shaughnessy?

  Binary, I guess.

• 16:03Speaker 02

  NAND, a NAND gate. So NAND gate is a binary operation gate. So it takes two inputs and produces one output. If you give it one and one, it produces zero. One and one is one, not one is zero. So NAND. You can create a AND gate from an AND gate. You can create an OR gate from an AND gate by chaining these different NAND gates. You can create all operations in mathematics from a single gate. That's a super powerful concept. At Replit, we stress about this idea of what are the powerful primitives that allows us to construct these features, products and services on top of them

• 16:46Speaker 02

  that makes the product more and more powerful over time? So for example, one of the core primitives of Replit is the REPL. It is something that takes user input, evaluates it and gives you a result. So all the Replit environment, it's bootstrapped from that REPL. Every programming environment, even the most advanced one is bootstrapped from that. Another concept is universal package manager. So we built this thing that you type any type of code into it. If you depend on a library on GitHub or any other popular package repository, it figures that out and installs it for you. So that way you removed a lot of barriers.

• 17:27Speaker 02

  A lot of software today is actually just gluing together existing pieces of software. So now we've taken care of using open source packages. Then you just go down the line. For example, for hosting, we host your code automatically. So previously with services like AWS or Heroku or any of the big cloud providers, the operations necessary to get a piece of code into a production environment are so insane as to be inscrutable by any person who's not like a DevOps sort of engineer. With Replit, you write the code and again, the environment, the Replit detects that you are trying to host a web server and we just host it for you.

• 18:13Speaker 02

  A lot of the stuff is either using machine learning or using basic Linux administration and detection and just like a bunch of automation layers. So we built up all these primitives. You get something like ChatGPT. So ChatGPT went super viral over the past couple of weeks. And Replit grew by virtue of ChatGPT going viral. You ask the question, why are we growing? Our revenue is growing, our usage is growing, everything is growing. It's not our product, it's someone else's product. And it's because ChatGPT is really great at generating software. So you can go to ChatGPT and ask it, hey, make me a Python program that plots the chart of Bitcoin. You copy that code

• 18:57Speaker 02

  and there's one place on the internet where you can paste that code where it just works without you configuring something and that's Replit. All the primitives that we built up are ready for this AI moment. And this AI moment where you can just talk to the computer and the computer can generate the code better than you. So in the future, I think we'll layer in a ChatGPT like thing into Replit itself. And you can have conversational interface. You still have access to the code. I think it's a little while away before you're able to not worry about the code at all. But I think we're going to keep layering AI tools that help you debug, that help you explain, that help you to get to a point where I think you would have a rudimentary knowledge of code, but we'll take care of the rest.

• 19:41Speaker 02

  And again, I just want to say that I think people should learn how to code. And I think everyone should learn that skill. But at some point, it's not going to be the bottleneck for creating software. But every now and then you still have to, in the same way that you pop over the hood of your car and having a simple understanding of the mechanics.

• 19:59Patrick O'Shaughnessy?

  I can certainly find myself doing it here, sitting here imagining like, okay, what would I do if there was no friction and I could open up the equivalent of a dock? It just effectively program it with natural language. If language is the limiter, geez, there's a lot I could do that I can't do today. I could imagine building charts by just describing them. I could imagine pulling in data points by just describing them. What have you seen happen by people using Replit, given that you've reduced friction? I'm imagining what I would do, but what have you seen others do? Like, what are some of the use cases or user types that have surprised you on top of this simplifying toolkit that you've built for people?

• 20:40Speaker 02

  I actually got a text from a friend, maybe you know him, Jonathan Swanson, who was the founder of PlumTack and he's running a new company called Athena now. He said that he posted a bounty on Replit for $20. So bounties are sort of another way to make software on Replit without actually writing code. So you just pay a bit of money and people from our community will make the software for you. He said that the task was to create some OpenAI-based fine-tuning interface or something like that. That's like a pretty complicated task and it got done in 48 hours. And he was like, after this was done, they found out that the creator is a 14-year-old kid.

• 21:24Patrick O'Shaughnessy?

  For $20?

• 21:25Speaker 02

  Yes, for $20, he made an entire program, which, by the way, who should charge these people a lot more? But it's wild. Now you have kids who can be productive when they're 13, 14, and they can make a lot of money. So our bounties program is merely three, four weeks old and we paid out $15,000 to creators and that's three Xing week over week. It's not going to be the thing that's going to make or break Replit. The reason we did it is to actually just showcase the talents and just create an economy around Replit and just show that there's this new generation of software creators that are eager to do work, that can make a lot of things and can build startups and can do all these things that people are asking them for. I think that wouldn't have been possible before.

• 22:09Speaker 02

  For you to find a 14-year-old kid to do it for $20 would have been at least not like a 48-hour thing. That's on the one hand, you have open access to people all over the world. Yeah, people who are tech professionals. Someone wrote an article in a medium that they built an investing Twitter bot, actually, that retweeted some investor accounts using ChatGPT and Replit in a matter of hours with zero prior coding experience. I just can go on for hours of these examples of just people gaining this new superpower, either earning money using this superpower or supercharging everything that they do.

• 22:52Speaker 02

  There's a Y C company that has their marketers code in Replit to automate a lot of their marketing and their data and things like that. This idea of coding being a thing that only programmers do, I think, is just disappearing in the same way that with Figma, the concept of a designer would just completely change. There was a pie chart going around around the time of the acquisition that showed the customer set for Figma. And it actually turns out designers are forty percent of the active users of Figma. That's the interesting thing about the creation tools. This new trend that I think is a super white pill is that it expands the concept of a creator. You go from a professional designer, you had to learn for, you had to go college for

• 23:37Speaker 02

  it and learn, earn all these degrees to something that anyone can really participate in. And I think that changes companies and how companies work fundamentally. When everyone is involved in the creation process, your marketing people, your salespeople, less siloing, more collaboration, I think you would need less people as well. Because there's a lot of glue people in organizations that are not really necessary.

• 24:01Patrick O'Shaughnessy?

  I think since ChatGPT was released, there's been this tsunami of reactions from other technologists, from other people building software, where you kind of have to answer the question, like, how does this change what we do? You're in a unique position because you've obviously deeply embraced the new AI technologies that have come out and how they may affect Replit. But maybe you could walk us through your perspective of watching these things come out. What has it been like for you in your unique position to watch Dolly come out, to watch Stable Diffusion come out, to watch ChatGPT be released? What are the things that you're thinking in the background about these technologies? Because I think your perspective is very different than anyone I've asked this of before.

• 24:44Speaker 02

  I started writing code that manages code more than 10 years ago. There's this thing called compiler in computer science. Compiler takes the code and runs it, does operation on it, generate machine code. And the fundamental insight that I had back when I say worked at Yahoo was I was responsible for like doing a lot of libraries and managing a lot of code that my coworkers wrote. I always thought that what we're doing with compilers is sort of a fundamentally manual process. You're basically writing all these traditional algorithms to manage code. It's very manual process. And then you look at the world of machine learning.

• 25:28Speaker 02

  So what is machine learning automating? Machine learning is taking every traditional algorithmic process and just making it statistical and making it learned. So you look at chess. Chess, you could write computer chess programs in this very algorithmic fashion. You could look at the board and calculate things. But then you had a bunch of machine learning programs that you can write on your calculator. At this point, it's so simple to write that just will beat any human just by using neural network and using statistical approaches. You had the game of Go with AlphaGo, I think 2015, 16, or even before that, that beat the world champion. Obviously, you had self-driving.

• 26:12Speaker 02

  Everything that we thought required this traditional algorithmic thing is now this huge, powerful bulldozer effect thing that is really flattening everything. And it's just this powerful tool. And it's just obvious to me that it's going to be applied to code. Now, when we started Rappel in 2016, I started toying around with the state of the art on applying machine learning code, and it was very, very primitive. Part of the reason it was primitive is because we actually were missing the insight that

• 26:49Speaker 02

  Code is very similar to natural language. There's this field called natural language processing that is actually fairly advanced, but it wasn't really applied to programming. So at OpenAI, they, yeah, the first experiment that really foreshadowed what's to come on language modeling was they had this machine learning system that learned how to do sentiment analysis. So sentiment analysis, basically, you give it an Amazon review and I'll tell you positive or negative. They had this paper called, I think, the Sentiment Neuron, where a machine learning system learned sentiments completely unsupervised. And what does unsupervised mean?

• 27:34Speaker 02

  So supervised, you give it an example. You say this is positive, this is negative, here's the text, this is positive, here's the text, this is negative. You need to collect a large data set. In this case, they actually didn't have a supervised data set where it's labeled. They only fed it the entire text. And the main operation of learn is to predict the next word. In that paper in 2017, they had this insight, they said, oh, interesting, this machine learning program learned language modeling on its own. What happens if we scale this up? Maybe we'll get some interesting effects. So GPT-2 was the first sort of scaled up version of that, where they trained a machine learning network that was one billion parameters. And so what one billion parameters

• 28:19Speaker 02

  meant is that there are two billion numbers inside the network that could be changed and tuned in order to learn something. So these are called the weights and biases. And GPT-2, anyone with a good imagination would look at GPT-2 and realize that the world has fundamentally changed when that technology came out, because GPT-2, you could give it a piece of text and it would complete it in a coherent way. You could train it to do very simple things like sentiment analysis, like translation, like generating small pieces of code. It was very primitive compared to GPT-3. But you could tell that this really changed things. Because up until that point, machine learning was a very tedious task of collecting millions of labeled samples.

• 29:07Speaker 02

  For the first time, you could construct a machine learning model where you threw a corpus at it and it just learned things on its own. It's almost like a child. It's a fundamental jump in generality. So that was 2019. 2020 rumors of GPT-3 started coming out and then when we got access to it, it was obvious that this thing was a really fundamental breakthrough in computer science and AI. It had this interesting phenomena where it had the thing called in-context learning. With GPT-3, it's called now prompting. So this idea of you could prompt it to do any task. You can prompt it to do translation, to do sentiment analysis, to do parsing, to send emails, to generate, copy.

• 29:52Speaker 02

  So you don't have to retrain the network. You actually just program it using natural language. So you go from a world where every machine learning model had to be constructed to do one thing in one thing alone, to a world where you have machine learning models that learn language in a general way and then can be programmed using natural language. So for us, it was pretty obvious that we needed to invest in this because our mission is to bring the next billion software creators online. We were building these primitives and we were having a success making programming more accessible. But at the same time, it just felt like there's a phase jump that needed to happen that

• 30:37Speaker 02

  you're not gonna get, no matter how simple you're gonna make the tools. When we looked at GPT-3, we're like, okay, this is it. So we started investing in it and building around it. Now, we have this product called Ghost Rider that is growing really fast, which is using foundation models and using GPT-3-like models to generate code on your behalf. And we're investing more in chat and interfaces and things like that.

• 31:02Patrick O'Shaughnessy?

  What are the limitations? I want to come back to the possibilities of GPT-4 and 5 and 6 and beyond. And they're incredibly exciting and I think people can get their hands around why this is so interesting because the interface itself, especially the chat one, is just so simple and easy to use. But we just came off of a tech boom, which may still be one in all things crypto where there was this, everyone felt the same way at the start of this is this unbelievable way to package value digitally in a way that we could package information digitally that led to the internet boom. And the boom just hasn't happened. Maybe it still might, but certainly to this point, I think it's underwhelmed in terms of end user applications.

• 31:42Patrick O'Shaughnessy?

  Whereas this one seems to be far more explosive in the consumer signal, the user signal, how much you're seeing it, what's being done with it and so on. I watched a hilarious Home Alone version of the White Lotus 2 introduction the other day, which was just mind blowing how creative it was. A multi positive was created by someone at Repli that, don't you make people create something when they start or something like that or using your toolkit?

• 32:08Speaker 02

  Yeah.

• 32:08Patrick O'Shaughnessy?

  The question I have then is what are the limitations? Limitations have emerged from Web 3 that maybe we couldn't have planned for. What do you think we need to be careful to not get too excited about? When it comes to this new technology, before I return to the stuff to get excited about and the opportunities?

• 32:25Speaker 02

  The center of AI is actually academia, the universities, the big labs. And the center of AI is Archive X. I think that's how you pronounce it. This is where you go read papers. When people are sharing things about AI on Twitter, they're sharing papers. They're academics and I'm not saying academics just means obviously true. The technical underpinning of this movement is very, very solid. And unlike what we had, I think with crypto. I still think crypto is gonna play a part in this future world. And I think there's some synergies in AI. I wrote a thread about it a while back. Happy to get into it. And what you said about the consumer stuff is very important.

• 33:11Speaker 02

  I think the hallmark of a tech bubble is something that makes investors a lot of money, but doesn't affect consumer life. Today, you go in any social media network. Everyone has AI avatars. That's like already fundamentally made people an interesting product that wouldn't be otherwise possible. Monkey avatar is notwithstanding because monkey avatars are just monkey JPEGs that you just pay for. It was like a pure financial play. This is a piece of technology that's actually interesting, that puts you in interesting places. This guy on Twitter, where he takes selfies across time. It's like an amazing piece of technology.

• 33:49Patrick O'Shaughnessy?

  So if you think about the positive side of this and where this goes, because if it's anything, it's not static. I think anyone that's been watching this has been bowled over. And obviously, there's a ton of research work, a decade plus or two decades plus of incredible research work being done at universities, at companies, at whatever, that led to this moment. But the realities were now are in this moment and new versions of everything are being released at sort of a terrifying pace. How do you think about keeping up with this? And what do you expect from, maybe we'll keep it simple, GPT four versus GPT three? Like, what does each successive iteration of this very general purpose technology gonna give us?

• 34:31Speaker 02

  There's the financial answer here, which is like, how do you make money off of this? It's actually quite difficult, even if you just absolutely believe this is gonna change the world. How do you construct a portfolio to do that? The simple answer, and I think what I'm attracted to is you invest in infrastructure companies. Invest in AI infrastructure, invest in hardware infrastructure like Nvidia and things like that. OpenAI, which is like the intelligence layer. You create a sort of an index portfolio on all the AI progress that's gonna happen. Then there's the product question, which is, okay, if you're an entrepreneur, what kind of product do you build? And that's actually also difficult because if you want to build a consumer product,

• 35:14Speaker 02

  things like Lenzel or these AI avatars, a lot of these things tend to be trends that come and go. Can you build some consumer product that's lasting? I think that chat's product, the AI assistance product is probably the best idea in the space. But then it's like, what is the monetization plan there? Can you build a Google alternative? A lot of people are thinking about this as a Google alternative. The cost is still pretty high. The cost needs to come down a thousandfold to be able to just have the same economics as a search engine. There's a timing question here. I mean, there's a lot of really interesting search engines. Proplexity AI, if you haven't checked it out, is really cool. You can ask it questions, you can also ask it to plot things.

• 35:59Speaker 02

  It can generate SQL, it can query Twitter, it does a lot of really cool things. So there's a lot of search stuff there. And maybe the answer is you start at the top end with some kind of subscription service. And then over time, maybe there's like an ads thing as the cost comes down. Then there's the futurists, what is the future of humanity sort of question here? Where we go from here? OpenAI's goal is to create AGI, so artificial general intelligence. And the idea behind AGI is that you get to a point where AI surpasses human intelligence. Now, this is a whole field of philosophy and technology. And so you have people, most famously this guy called Eliezer Yudkowsky,

• 36:43Speaker 02

  and people who blog on the social network slash blog called Less Wrong. And they're super convinced that this will kill everyone. They think that this is the death of us. And Elon said once, this is like unleashing the demon. So Elon kind of in this camp a little bit. And this is because of this problem called alignment. It's that you create this super powerful force, but we don't know how to align it to human interests. So they'll point out evidence, look at ChatGPT. ChatGPT will misinterpret you and give you false information pretty easily. Despite what OpenAI has done to align it and make it more responsive to you. And people are hacking it in all sorts of ways.

• 37:28Speaker 02

  Now imagine that a million times more powerful and is actually able to do things in the world. The classic example that's sort of a reductio ad absurdum a little bit, which is I am a paperclip entrepreneur and I want to create as many paperclips as possible. As a paperclip entrepreneur, you also happen to be the first guy for some reason who invents an AGI. And so you ask the AGI, hey, I want you to make paperclips. AGI is like, oh, how many paperclips? It's like maximize paperclips as many as possible. So the AGI starts making the first factory, you sell paperclips. You build another factory and AI keeps going and going and going. And then it takes over the entire country and you're like, no, stop. We don't need any more paperclips.

• 38:12Speaker 02

  It's like you didn't give me instructions to stop. You told me to maximize paperclips. So it harnesses all the resource on earth to create paperclips. And then it's done with earth, it goes to the moon, it goes to Mars. The entire universe becomes a paperclip factory. That's the dystopian vision of that. The utopian vision is the Sam Altman OpenAI view of things is that alignment is possible. We're making progress on that. And actually, these things like ChatGPT will actually help us align and build the next version of it. Now people at OpenAI are using ChatGPT to help them with their work and to ask a question and say, okay, how would you align the next version of yourself?

• 38:58Speaker 02

  So this bootstrap answer of we're going to make slow progress, slowish progress. By the time we have AGI, we're going to have learned a lot because we have these tools in order to align them. But the idea is that we're going to get to a point where it's similar to the movie Her, where everyone has this assistant AI that's better at everything. Your role in the world is going to be to be more creative, to have fun, to find. Fulfillment somehow, but for any given task, an AI will be better than you. These are the dominant views. I can give you my view, which is somewhere in the middle. I think there are limits to how intelligent computers can get.

• 39:42Speaker 02

  I think they can get pretty raw intelligence. But I think we will not be able to cross the chasm into a human-like agent. Until we actually understand the nature of consciousness and they understand the nature of human intelligence. It's almost like you expect to get abs without going to the gym. I'm gonna sit around and just get abs. It's like, how do you expect to build a human-like agent if we don't understand human agency to start with? We don't understand the nature of consciousness. We barely understand the brain. So I feel like we're so far away from that question that it's inconceivable for me to create things that act like humans in the world until we understand what humans are.

• 40:27Patrick O'Shaughnessy?

  What do you think about just the practical need for you as a CEO of a company, especially that's so ingrained in the space, to sort of prepare yourself and your team for, let's just make it simple, GPT-4? This is something everyone can do. Everyone can have an opinion on AGI, but it's kind of hard to do much beyond have an opinion. Whereas we kind of know GPT-4 is coming. It's going to be like what we've experienced, but better. Or maybe even different in the way that ChatGPT was different than GPT-3. What are you doing with your team to get them in the right mindset to react to these changes in technologies in productive ways? Because I think all companies have to start thinking about this. Whether it's something simple like customer success, which is sort of made to have AI laid upon it or something like that.

• 41:13Patrick O'Shaughnessy?

  What are you doing as an example to prepare for GPT-4 specifically?

• 41:17Speaker 02

  We're becoming a lot more iterative as a company. I recently made this funny tweet that went a little viral, which is after six years of running Replod as CEO, I decided to make a change. I am stepping into a new role as head of engineering. And then I said, that's in addition to my role as CEO. So now I have two roles. And people freaked out about it. The reason I did this, I actually did take on head of engineering. And so now I have all of engineering reporting to me. It's not like Replod is a huge company, but it's a lot more reports. And part of the reason I did this is because I think we need to be a lot more iterative and reactive in this environment. I think you need to be able to change your roadmaps pretty quickly.

• 42:02Speaker 02

  So ChatGPT caused a huge update in my mind. I was not actually convinced that a chat like interface is going to be useful for coding, for example. I was like, okay, this is useful for consumer apps, but it starts out programmers love that interface. Absolutely love it. They use it instead of going to Stack Overflow, which was the main way to look up programming questions. And now we're building a chat interface. We're actually in the middle of building it. I think we're going to have something in two or three weeks. That wasn't our roadmap before ChatGPT. You had to change your roadmap.

• 42:40Speaker 02

  So I mean, we're probably in a more of a special case because we're tracking this and turns out coding is the best application of this technology. But I think it affects everyone. At minimum, any company needs to prepare for catching up on employee productivity and productivity tools. If you're a CEO of a super big company, you should be thinking about how can I increase productivity using these tools? Especially in the market condition we're in, where investors reward more efficiency. I think you can bring a lot more efficiency using AI. You can fine tune these models for domain specific applications.

• 43:23Speaker 02

  You can create the co-pilot ghostwriter of whatever thing of accounting, of finance, of whatever thing it is. And just prepare to be surprised. What comes next? For example, I got a message from a CEO of a interview prep company, one of the biggest ones. And he's like, I feel like the world is changing from under our feet. They've been preparing people for algorithm-like questions that are asked at Google, like reverse a binary tree. And he's like, I feel like we need to react. What do you think the future of this is? And so I told my view, and my view is that I think no one will use these questions in the next two years. Literally, no one will use these questions.

• 44:07Speaker 02

  The traditional Google-type interview questions are gone. I think there's an entire industry there that needs to change. So prepare to be surprised. Prepare to be more iterative. And think about how you can adopt this technology to bring more efficiency to your organization.

• 44:32Patrick O'Shaughnessy?

  It implies if Google's going to stop asking those questions, that the kinds of people that they're trying to find using those questions may change. What skills do you think should people cultivate in themselves, given this new emergent toolkit? And what things are going to be less valuable than in the past as a result of this emerging toolkit?

• 44:54Speaker 02

  I think post-war world, and particularly America, was this very interesting time in the history of humanity. And a lot of the boomer and maybe Gen X generation think that that's how the world is. But there was a time where you sort of went to school, you did what you were told to do. You learned these basic sort of rote way of doing things, like multiplication table and writing essays and things like that. It didn't require a lot of creativity. You just checked some boxes and went through life and had a pretty good life. You were able to buy a house and put your kids through college to kind of repeat the cycle and buy a car and things like that.

• 45:39Speaker 02

  And that world is gone. That world has changed, and I don't think people understand that it's gone. We're still clinging on to this notion of this traditional past.

• 45:50Speaker 02

  I think the worst thing you can, like as a parent, the worst thing you can contrive in your children is to be completely. Follow the rules. Yeah, yeah, just follow the rules. I think the worst thing in the world they can do is really devolve that in them. I'm not saying you should go and make them disruptive or things like that, but they need to think for themselves. Teach independent thinking, teach being adaptable, teach being different. Teach them to be more okay with a changing world because we're, I think, in the middle of this massive shift. I think creativity is going to be more important.

• 46:32Speaker 02

  Being able to generate ideas is going to be more and more important. You know, in tech, we made fun of idea people for a long time. They're laughing now because if you're an idea person, you can go to stable diffusion and DALL-E and ask it for a poster. You can go to G P T three and ask it for an essay. You can go to Repland and ask it for software. Ideas people are actually coming back in a strong way. Be prepared to generate a lot of ideas and just be able to do that on the fly. And then your question about Google, how the Google interview will change, will go back to being more focused on problem solving. As opposed to just jumping through hoops.

• 47:16Speaker 02

  I actually think it's going to be a positive development. I think it's going to be more honest as a working environment as to who gets promoted. And it's going to be more obvious who does a better job. For better or worse, I think wealth and equality will grow at a much faster rate. Because I think all these technologies that accentuate existing differences. Access to opportunity will increase, but at the same time, people think that access to opportunity and wealth and equality are unrelated. But actually, the more access to opportunity, the more honest the market.

• 47:51Patrick O'Shaughnessy?

  The more natural wealth inequality is, yeah.

• 47:53Speaker 02

  Exactly.

• 47:55Patrick O'Shaughnessy?

  Well, how does this all impact the big existing dominant technology companies? In some sense, there's very simple things you could say like, ah, it's probably good for Nvidia. If we're going to be training a lot more of these models on GPUs, seems like they'll be in a good position. But there's probably hard to imagine effects that I'm sure you've thought about for the big technology companies. And I'm just curious what you think here, because some of them have become so dominant in their respective areas. The traditional FAANG companies, as everybody calls them, they're just so big and so profitable and so powerful. Some of them like Facebook are trying to create new interfaces, which might impact all this too. How do you think about the impact of these new technologies? And what you're doing and sort of the creative boom or explosion that might result from it? On the existing leading technology companies?

• 48:41Speaker 02

  Let's start with Google. So everyone is saying that chat GPT is the first. So let me take the contrarian position on this. I actually think that Google's been training in the gym every day for the past 15 years. And this is their moment now. That doesn't mean that they're going to be able to capture it, but I don't think it's foregone conclusion that they're going to blow it. For instance, the transformer machine learning model that birthed this entire industry. So everything from stable diffusion to GPT is based on the transformer technology. Was invented at Google in 2017.

• 49:21Speaker 02

  Google search uses BERT, a transformer based language engine, going back to 2019. Google has been adopting this technology since the start. But the thing is, the economics at Google just doesn't make sense right now. What they're working on is they're making these technologies a lot cheaper. It really needs to be a thousand fold cheaper for them to apply it on a massive scale. Again, that doesn't mean that they're going to win, but also doesn't mean that they're going to lose. I don't think that it's a foregone conclusion that Google's going to lose. They've built so much AI muscle. It really is their moment and Sundar is like a smart guy. He knows that.

• 50:00Speaker 02

  And I think they have a really good chance to create another S curve in their growth story. So that's Google. Have no insight about Facebook. I don't know how strong their AI stuff is, like the metaverse is interesting. It'll probably benefit from some of the AI stuff. I think there's a potential that there's a social media company that innovates on this idea of completely AI influencers. So if you think about influencers, our generation grew on Facebook and the Facebook model was a one-to-one model. You friended someone and they friended you back. The follow model, which is TikTok, Instagram, Twitter is all based on, is a one-to-many model. You can build a relationship with someone who doesn't even exist.

• 50:45Speaker 02

  There's a potential for AI to disrupt that. Seems like TikTok is probably ahead. I could imagine entirely new ways of generating completely virtual people that you can follow on these places. And maybe for the first time, tech will actually have a real challenge to Hollywood. Where you can follow the Marvel Universe on TikTok. I suppose like in one of the movie theater and watching that. So that's gonna be interesting to watch. I have no insight as to whether Facebook will be able to do that. And I don't know if the metaverse is the right bet there. Microsoft is super interesting because Microsoft is adopting these technologies super fast, like co-pilot.

• 51:27Speaker 02

  It's a rapid competitor, but they were ahead of us to market because their relationship with OpenAI. I think they're making really big bets there. Satya, the CTO of Microsoft, the CEO of GitHub, they're all non-stop talking about AI. They're selling their platforms as AI platforms. They're especially focused on developer productivity and efficiency and things like that. I think they're also gonna do a great job at no code, low code with the rest of AI. I think they have this huge platform called Power, I think. They're gonna infuse basically everything with AI. I think they're gonna do a great job on the product side. They're gonna use the OpenAI infrastructure and they're gonna infuse all their products with AI. Might be the first to market to do that.

• 52:10Speaker 02

  So I think that's a bull case on Microsoft. I don't think the rest have any advantage or disadvantage. I think AWS will be fine. I think a lot of people are just using GPUs on AWS. I don't think they have any particularly big AI bet, but they'll be fine.

• 52:30Patrick O'Shaughnessy?

  Are there any other major black pills that you think about often? That's where we started our conversation, which I think was just so fascinating. On the one sense, computers got accessible to us all. But the cost of that was that we couldn't really control what they do. Although the market does to some extent, control what they do and it gives people what they want over time. Any other major things, I'll call them black pills again, that you just think very differently about than is the common narrative out there?

• 52:57Speaker 02

  In this moment, I'm actually more optimistic than pessimistic.

• 53:01Patrick O'Shaughnessy?

  So white pills then?

• 53:02Speaker 02

  I only can give you white pills at this point. I think the bottleneck to human flourishness has been our output. And how much we can get done. And the most productive people on earth should be able to do more, should be able to have more capital. We should double, triple down on the most productive people in the world. And I think AI really accentuates that. AI allows people the equivalent of commanding an army of people. So once we have these AI agents that's going to help you with generating software, making businesses and making decisions and automated things.

• 53:44Speaker 02

  The most productive people will get even more productive, like crazy more productive. I think like a Elon type person will be a thousand X more productive. Can you imagine that?

• 53:54Patrick O'Shaughnessy?

  No.

• 53:55Speaker 02

  Well, maybe that's a black pill at this point, given the ethics. But for the first time, we're going to see a real impact on productivity. For the average person, but also more extreme on the tail as well. And I think that's a good thing. I think that'll get humanity to the next set of answers, to the next set of scientific conclusions, to the next set of energy breakthroughs. All the challenges that we have today, I think, are fundamentally constrained by human intelligence and productivity. I think we'll be able to break through all these things with the right technology. Now, the potential black pill here is

• 54:36Speaker 02

  I actually think that there are some people in AI that are building these technologies to be less cooperative and more competitive. So less co-pilot and more pilot. And I think that's bad. I think entrepreneurs should be building AI systems that are meant to accentuate existing human creativity and intelligence. If there's a black bill, it would be something like these companies from their hubris just think that they can build the equivalent of an AI monarch or dictator or something like that. And I think that would be a bad outcome.

• 55:13Patrick O'Shaughnessy?

  If you could encourage everyone listening to go engage with Replit in some way, what would you recommend that they do? And the person listening on average is probably like very smart, but probably not terribly computer science technical. What's a great way for them to just go realize the power of what's unfolding right now?

• 55:32Speaker 02

  Whatever idea you have, just whatever idea you have, go to Replit Bounties, create a bounty, brain dump your idea into this text box, just in English. Pick someone from the community to do it for you. And in a matter of hours, maybe days, they're going to get it done for you. And you're going to get a Repl. You're going to get a project that is the software, it's running, but it's also the code. And then go into that software and try to change it. You can subscribe to Ghost Rider. Ghost Rider is our AI tool. You can highlight any piece of code inside that software and right click explain. You get an English explanation for that code. And then you can do right click generate and you can generate more code.

• 56:14Speaker 02

  And you can also, inside the editor, you can get suggestions from the AI. If any of this was too hard, open ChatGPT and ask it the question you want to ask it. If you ran into an error, copy that error, put it into ChatGPT and ask it, what happened here? How can I fix it? We also have a very simple history scrubber thing where you can go back. If you made an error to a previous version, that we're going to help you with that as well. But I think Bounties is actually like a very good entry point to the technically curious. Because you get an idea, you have a piece of software and you have the backing code behind it and you can start changing it and doing something different with it. Alternatively, you just go to Replit and we have this program called a hundred days of Python,

• 56:59Speaker 02

  where if you commit to twenty minutes of Python per day, we'll get you to full proficiency encoding if you commit to a hundred days. So just twenty minutes a day for a hundred days, you could do it on your phone, and we'll get you to full proficiency and you can use the AI tools throughout this to learn more. So I would say these two alternative paths, either a hundred days of Python or bounties as a good entry point.

• 57:23Patrick O'Shaughnessy?

  What roles do phones play in all this? It seems like one limiting factor is people are on their phones all the time, they're not on their desktop all the time. You told the opening story about having to buy another computer to program the smaller computer. What role do you think phones play in programming in the future?

• 57:38Speaker 02

  Ten percent of our engineering team is working on programming environments for the phone. I think now the most popular phone app for programming, we just got in a couple of months, two hundred fifty thousand downloads. It's growing super virally, especially in places like India and Brazil. With AI, I think it's tolerable to be able to write code on your phone and to be able to generate code, interact with code. Again, I think the ultimate interface for these things is one that really blurs the distinction between chat, docs and code. We're probably ways away from that, but I think once we get there, and that's like mostly a UX innovation, and as the technology gets better,

• 58:20Speaker 02

  and I don't know if we're going to be the first to do that, we have a lot of other priorities, but I think at some point, interacting with code by talking to an AI agent on your phone is probably the right modality for the average person.

• 58:36Patrick O'Shaughnessy?

  Incredibly powerful to think about the impact of everything that we've talked about today. I mean, it messes with the mind a little bit that so many of the frictions between our imagination and outcomes. Back to your point about we used to knock the idea people and say it's all about execution, but it seems like technology is taking more and more of that execution away from us. The need for it is disappearing through tools like the one you're building. I've loved this conversation, I've loved exploring what you all are building. I ask everybody the same traditional closing question, what's the kindest thing that anyone's ever done for you?

• 59:07Speaker 02

  My mind keeps going back to a teacher in first grade back in Jordan where I'm from. And there was a computer teacher and it was kind of odd for a school to have computers at the time and place, but we had one and he saw some kind of talent in me by just being able to write a little bit of code even at six years old. I also was a big troublemaker in school, so I would get in trouble all the time. I would just get bored in class, start issues and fights and things like that. And so I was always in trouble. I was always in detention and all of that. He took interest in me and he would take me out of math and physics classes and would

• 59:51Speaker 02

  teach me advanced algebra. In first grade, I was doing fourth grade, fifth grade level math and things like that. And it didn't go on for so long. It went on for two years, but he taught me enough math and programming that set me, I think, on a path that is fundamentally different than what I would probably otherwise have gone down. I can't remember the teacher's name. I try to look him up and things like that, but if it's not the kindest thing, it's definitely the most impactful thing that anyone's done for me.

• 1:00:23Patrick O'Shaughnessy?

  I'm John. It's been so much fun. Thank you so much for your time.

• 1:00:25Speaker 02

  Thank you. I appreciate it.

• 1:00:28Patrick O'Shaughnessy?

  If you enjoyed this episode, check out joincolossus.com. There you'll find every episode of this podcast complete with transcripts, show notes and resources to keep learning. You can also sign up for our newsletter, Colossus Weekly, where we condense episodes to the big ideas, quotations and more, as well as share the best content we find on the internet every week.