Ep 5: You.com CEO Richard Socher on The Future of Search, Open Source Models and AGI

Jacob and Erica sit down with You.com CEO and former Salesforce Chief Scientist Richard Socher to discuss building a new search engine and the future of search, his predictions on when open source models will catch up to GPT-4, AGI and more. You can find Richard on Twitter (@RichardSocher) and learn more at You.com

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Jacob Effron?

  This is Unsupervised Learning, a podcast on the rapidly developing AI landscape and what it means for businesses in the world. I'm Jacob Effron, and I'm joined by my colleague, Erica Brescia. And this episode is a really fascinating conversation with Richard Socher, founder and CEO of U.com, a consumer search engine. We had a wide-ranging conversation. Richard shared what he thinks makes for a good search experience. We touched on AGI, when open source will catch up to GPT-4, the role of academia in the future of AI. And, you know, Richard's done a lot of fascinating work throughout his career, and we hit a bunch of it. Protein generation, economic policy. I think folks are really going to enjoy this episode. We've got Richard Socher on the podcast. Richard started with a CS PhD over at Stanford,

• 0:47Jacob Effron?

  then founded MetaMind, a company that was using neural networks in a broad range of ways, eventually acquired by Salesforce, where he was the chief scientist. Richard, I think, taught the first deep learning course at Stanford while he was there, and then founded U.com, an AI search engine, which we'll be discussing a bunch today. Richard, thanks so much for coming on.

• 1:04Speaker 01

  Thanks for having me. So exciting time for unsupervised, supervised, reinforcement learning, all the different kinds of learning.

• 1:11Jacob Effron?

  Yes, our podcast name and beyond. You know, I guess to start with, you have this fascinating career, right? You know, going from your PhD to starting MetaMind, to then the chief scientist at Salesforce, Stanford, and then U.com. Would love to hear just from you, what motivated each of those career moves and kind of the journey here?

• 1:28Speaker 01

  Oh, thanks for asking. Yeah, you know, I was fascinated from a young age with natural languages as well as math. And in high school, I basically realized those overlap in a really beautiful way in linguistic computer science, which is a different term for natural language processing. And then I felt like there wasn't quite enough math in that, so I went a little bit more into computer vision, but eventually found my way back to natural language processing. And I think it's the most interesting manifestation, like language understanding is the most interesting manifestation of human intelligence, and hence also the most interesting and hardest one to capture in artificial intelligence, and trying to replicate those capabilities. And, you know, language is intricately connected

• 2:12Speaker 01

  to thought, and culture, and civilizations, and knowledge. And so it's been, it's been always a fascination of mine. I was very fortunate and, you know, sort of a combination, as often is, of luck and sort of a lot of hard work and not giving up. But, you know, at the beginning of my PhD, saw Andrew Ng talk about deep learning for computer vision and learning features rather than having AI experts kind of design those features and then just have AI weight them. And at the same time, I was working with Chris Manning and NLP. And so I kind of tried to take ideas from neural networks and apply them to let natural language processing. And, you know, that led to word vectors, some of the most highly cited word vectors, glove embeddings,

• 2:55Speaker 01

  and then work on embedding of whole sentences, which became Cove contextual vectors, which then became Elmo and then BERT. You know, sort of improved upon those ideas by Google. And then eventually it became, let's try to pre-train everything and not just the encoding, but also decoding. And the way you have to do that is by adding the task into, as input to the model. And so that was kind of my research goal for many years is to have one unified neural network model that could do a lot of different things. And we finally did it. It's called Deca NLP. Some of the folks from the founding team of OpenAI said that kind of influenced them, you know, their thinking around GPT-2.

• 3:38Speaker 01

  And there was this idea that the question slash the task slash what we now call the prompt is also an input to the model, rather than using a separate model for translation and one for sentiment analysis and one for question answering. You use a single model and you just tell that model, currently, I'm trying to do translation. Now I'm trying to do sentiment analysis. So that idea kind of led to prompt engineering and some of the chat GPT craziness that we're currently seeing. And after basically having put out most of those big ideas and research, I realized that I think you can now have more impact in taking those ideas and actually applying them again. I think that wasn't, you know, quite true until a few years ago, but it certainly is now. It's like as if we had, you know,

• 4:22Speaker 01

  all this covered electricity. And now, yes, you could try to understand it even better and the field dynamics of. And so on, but you can also just electrify like different cities and have light bulbs and like just build all of these impactful kinds of technologies with with those ideas. While there's still continuing to be research happening towards AGI and stuff, but even even, you know, GPT four models. Uh, I think a lot of those ideas have been around for many years. We're just scaling them up now. And that scale is harder to do in an academic setting. So I've been, long story short, motivated by I for a long time. Uh, trying to first make it work and then trying to have its impact be positive. And, you know, you dot com in particular.

• 5:05Speaker 01

  It's just the biggest application of deep learning and natural language processing is search. And so like it was kind of a weird moment in time where you see the underlying technology get so much better. But then somehow the biggest application is so stagnant, you know, there Google is still mostly a list of blue links for a lot of queries. Plus like a bunch of ads and and then other things kind of add postdoc kind of added to the query. And so. We thought with summarization and all these different NLP techniques, we should be able to do better. And that's what led us to be the first to launch a chat, you know, search engine. The first to infuse links into that chat engine, the first to bring.

• 5:46Speaker 01

  Artificial like generative AI into into the search engine by letting you generate images and writing essays for you and writing code for you. And all those things, those are, you know, some of them were research projects we had worked on in the past. Especially on the large language modeling side and some of those we're just very excited. Uh, in the stable diffusion, for instance, which I hadn't worked on in the past to to bring to people and. Kind of push the very boundaries of what we even call a search engine, you know, because when you're creating an essay, when you're generating an image, when it's writing code for you. Are you still searching or are you actually doing things? But if you call it a do engine, people are like, what's a do engine anyway?

• 6:22Jacob Effron?

  Yeah, I don't know if Marc Benioff owned do dot com, right? He definitely owned you dot com.

• 6:26Speaker 01

  He did, yeah. He just shared it actually in this, uh, the information article that came out today about, yeah, he had, uh, had it for twenty four years.

• 6:34Speaker 02

  Oh, how funny. So you talked about it in bits and pieces, but just for. Listeners who maybe haven't played around with you dot com, which is super cool. Can you? Describe like how people interact with it and what was important for you to really. Get right, because I think part of what makes you.com so special is how thoughtful you've been in doing good and providing a great experience and a good experience, too. It's like great and do good things. There's some ethos in there that's worth talking about.

• 7:05Speaker 01

  Yeah, we believe internally in trust, facts, and kindness, though we did learn that you don't say that out loud, because as soon as you say we care about trust, people actually trust you less right away. But it is something that we care about a lot, sort of under our underlying values. Concretely, also, we want to be able to help people accomplish more, right? There's been sort of a lot of online, a lot of our online life is very easily getting stuck in these engagement loops, trying to get people just to spend more time on that platform and eventually try to get them to click on ads. And while ads, especially when they're private, as in they don't follow you around the internet, they're not personalized, you're just like looking for an air purifier, so you see an air purifier ad, that kind of thing, as long as that's not the fundamental goal of it.

• 7:50Speaker 01

  But the fundamental goal is actually to help you accomplish something, very much influenced by the Jobs to be Done framework, we think we could actually build something better. So, long story short, for listeners who are not familiar with it, you know, we launched new chat, which we're actually about to make the default experience for everyone again. There's some sort of partnership pickups in the process there, where we had to kind of make it a secondary experience, but now we're about to bring it back to the main default experience. And the idea there is to combine these large chatbot models, similar to ChatGPT, and take the best of that world, but also still give you easy access to that list of links that you are familiar with, that you sometimes need if you actually have

• 8:31Speaker 01

  a quick navigational query. But the power of a chat-first experience is that it will really summarize things in such a clearer way than, you know, you could, you know, after many clicks, maybe eventually do yourselves. But it saves you an incredible amount of time. And I wish I could share with users here, here on the screen, or the listeners, but, you know, like imagine you just have a list of blue links when you're trying to program something or trying to write an essay, or you just have the essay written, or you have the code written that you're looking for to solve. And so, it's kind of interesting. While there's a bit of an Uber analogy here, where in the beginning, some investors thought, oh, Google is this big, maybe you get some market share from them, and then you could

• 9:15Speaker 01

  be at most, you know, that size. But what we found is that People would not go to Google and say, write me a website with JavaScript code and a commenting function. It's just because they know a list of blue links is not going to give you that answer. But you chat will actually just write that whole thing, hundreds of lines of code for you. And so the TAM, while largely overlapping, is not completely overlapping with your traditional search engine.

• 9:41Jacob Effron?

  Yeah, no, it's really interesting. I mean, I think one thing you've talked about before is, you know, letting people obviously build on top of you.com, right? And kind of integrating third parties. I feel like it's kind of a key philosophy of the product and how you work. And, you know, I think it'd be super interesting to kind of contextualize how you think about that, maybe also kind of comparing to, you know, obviously, OpenAI recently released the ChatGPT plugins. Like, would love to kind of, I think our listeners would be super interested in your perspective on that.

• 10:04Speaker 01

  Yeah, it's been a really crazy ride the last couple of months. You know, since we've launched UChat, seen a bunch of copycats. And now, of course, also some of these really big, I guess, copy lions. Just talking about mountain lions here. Yeah, they're not quite copycats. And so, you know, also taking some of those ideas that we've launched and, but maybe for those of you not familiar, the main idea is that as everything in search becomes this one chat interface that summarizes everything in a very useful way for you, it kind of leaves the rest of the internet behind. And so we came kind of from this more equitable place where we thought, well, let's then all collaborate on that very first

• 10:48Speaker 01

  page that people don't have to really leave that much because it's so powerful and bring apps into that experience. And also, you know, as much as I love natural language, but people don't really want to shop with just text, right? If you're buying something that you're going to have to look at, it's not just like a piece of software or something. You want to see pictures of what it looks like in your house, when you wear it, when, yeah, you carry it around or whatever. And so you have to be more visual. So we not only launched the first app store for a search engine, and then were the first to merge it with with chat, but we also made the first, what we call multimodal large language models. So where the modalities that the output could be are

• 11:29Speaker 01

  Just text, but it can be code, but it can also be images, it can be interactive elements, it can be graphs and plots and tables. So when you ask like, what's the stock of Salesforce? It will actually just show you not a bunch of made up numbers that are hallucinated potentially by an LM, but a stock ticker app that just shows you the ground truth of that information, right? There's a LinkedIn app that pulls up the right sets of people and, you know, all of that. And so we think that's really important. You're right, literally just last week, OpenAI, very inspired by that idea. And now also has what they call plugins. Currently, they still, you have to select each plugin and then it'll just talk with this plugin in mind. And where our system is already, you know, fully integrated, the LM can kind of choose which apps to bring in or not.

• 12:14Speaker 01

  So it is, you know, if you're a search engine, people don't say, oh, now I want to look for the weather. Let me select the weather widget. Now I like ask what the weather is. They just want to say what's the weather and then get the right answer. And then if they switch, okay, now what's the right stock price for this company? They don't want to have to select some other thing in some dropdown menu. But, you know, maybe eventually they'll copy that automation of the selection process too. But overall, we're, we're kind of excited and on the one hand, it's stressful to be honest, to have so many, yeah, copy, copy lines, maybe we can coin that term in this podcast here, come after you. But at the same time, it kind of shows our users also like, wow, I can go to you.com first to see what the future is like. And then eventually, you know, that future will be more equally distributed and going elsewhere.

• 12:59Speaker 01

  And of course, we're not stopping now. We have a bunch more exciting innovations in the pipeline.

• 13:05Speaker 02

  Copy lions, you heard it here first, folks. You know, on that note though, I think when you look at, at search and chat too, there's a version of the world where it subsumes almost everything in the way that we do all kinds of things that are currently verticalized solutions. And I'll throw one example, we've seen, I don't know, dozens of startups in travel, right? How do you think about what the world looks like in the future? Is there room for companies to build specialized apps for, you know, to travel or another example? Or do you think that just becomes a feature of these other platforms like you.com?

• 13:43Speaker 01

  Yeah, it's a great question. You know, I've I've been a big fan of of natural language processing for over a decade. And and like I said, it's sort of the most interesting manifestation of human intelligence is is language understanding and hence like AI. But at the same time, when you see different waves of improvements in that technology, I also, and I'm, you know, investing in the side as well, AI X Ventures that I co-founded with Sean Johnson and like, you know, you see sometimes like, and this is, you know, many years ago before we started AI X, but you see speech recognition finally working, right? And then a bunch of people are like, wow, I want to just talk about everything with my personal assistant. And of course, the underlying NLP was part of the problem, but also

• 14:24Speaker 01

  sometimes people think of this new hammer and then everything is a nail all of a sudden. And as much as I love NLP, like pure language isn't the right interface for everything, right? Like if I ask for a stock price, a stock ticker is a better way to consume that content. And we'll see similar things like in restaurants, for instance, like, sure, I can read like a thousand lines of text about all the restaurants in San Francisco and but really what I wanted to see was a map and then a filter for like, I don't know, Michelin stars or like, you know, Chinese food or whatever it might be. And then like it's currently open and I have these like simple filters. And then I have a map and I'd be like, I'd be willing to walk this much extra for like this extra star rating.

• 15:05Speaker 01

  And like to communicate all of that via text back and forth is less efficient ultimately than having a useful UI UX that is well designed for that use case. And so that's why our apps are not just like sort of backend widgets, but they're sort of have these strong frontend components too. Because for a search engine and even a chat based, you know, a chat first search engine, I think you need to have that multimodal and interactive style. And I guess that's a long-winded way to answer your question, which is, I do think there's still space for a lot of other companies. I think also the competitive mode that we at U.com struggle with feeling like that gets smaller, copy lines coming in and lots of other smaller players also coming in and trying to copy those features.

• 15:49Speaker 01

  I think even OpenAI will have that problem. Because when there's this much excitement in a space and there's a ton of like, there are a ton of researchers right now who are like, what am I even still doing if I don't have access to a model like this? And so my prediction is that before the end of this year, we will have a GPT-4 equivalent open source model.

• 16:11Jacob Effron?

  Yeah, no, I saw you tweet that. It was a bold proclamation, I guess. What gives you confidence in that?

• 16:15Speaker 01

  I mean, we already have internally now a model that's equivalent to GPT 3.5, maybe. And we have lots of user questions and I'm chatting. I'm an investor in Hugging Face also.

• 16:30Jacob Effron?

  They were in your class, right?

• 16:31Speaker 01

  That's right. The founders were part of my class back at Stanford days. And so I think there's just so much interest in the space. And you know, you have folks like stability.ai that do have massive amounts of funding. And then I think people, it's kind of fascinating, but once you show something is possible, it's a lot faster and easier to also do it. And so, you know, I do give OpenAI a lot of credit for having had enough conviction to say, we're going to make this model really, really big. And we're going to spend $100 million or so on training the first version of it. And not knowing for sure, all these interesting emergent capabilities would come out. You know, you couldn't guarantee it. There's a lot of intuition. I had similar intuitions. And I told some of my team members, just make your large language models, which we had the best stuff at the time, just make them bigger.

• 17:15Speaker 01

  But I did not give them like, you know, 100 people and $50 million to train it. And so I didn't have that resource, but I don't know if I, even if I had it, maybe I still wouldn't have given it to them, right? But I also couldn't convince them. But anyway, long story short, they had this conviction. We have these large models. Now that we know it's possible, I think people will replicate it and they have more and more. And so far, most of the open source models had been much, much worse. But people are now coming around to understanding how important the RLHF is, the reinforcement learning with human feedback. And once you sprinkle even just like a couple hundred of those human feedback data sets or data, like the answers essentially into the model, it'll get massively better.

• 17:57Speaker 01

  And we have hundreds of thousands of human feedback at u.com already. I'm sure other people are collecting this too. I think the open source community will eventually kind of realize that open source code is nice. But once it's all AI, when you think about the whole software 2.0 stack and realize, okay, it's the code, but it's also data sets and then it's the human feedback on top. And how do you open source all of that and collaborate? And open source is very powerful, right? If you get thousands of people to be excited about that model and they all start to label it to and not just code on it, it's just going to be a matter of time.

• 18:31Speaker 02

  Hey, actually. I have this mental model that I've been working on that I'd love to run by you. I'm not, I'm not like convinced it's the right one, so I'm curious to get your take live on the air. Um, I've been thinking about this a little bit like the, the early days of cloud, right? And like when the cloud platforms came up and you had kind of Amazon and then later Google and Microsoft and then others tried, there was kind of like this open source movement. It didn't get the same amount of like traction, like OpenStack, for example. You know, it was kind of for all the me too players, but there were still a lot of opportunities

• 19:11Speaker 02

  to build really big businesses on top of the substrate of the cloud vendors as it came on the scene. Right? And as I've been thinking about where there are investment opportunities, I think of things like, you know, MongoDB still spent a, or still built a hugely successful business despite the fact that cloud vendors were launching their own database services. So, you know, I'm still working through, like, I always want open source to win. I've been in open source for 20 years, but I'm like, I'm still working through like, how does this actually play out?

• 19:45Speaker 01

  So is the question like, could open source catch up to this? Yeah, I think the, the interesting bit is that cloud, for the most part, essentially allowed you to virtualize hardware and open source doesn't give you free hardware, right? Hardware still costs money, no matter if it like, you can't give it away for free forever because it just keeps accruing cost, whereas like, here's a piece of code, you do it, you run it in your own infrastructure. So there's this like massive discrepancy. Now, I think in an interesting way, because these really powerful AI models are so expensive to run, especially at scale, you could actually argue that that pushes them away from, oh, this is just a pure software thing, just run it wherever, to maybe you do need to

• 20:30Speaker 01

  kind of run it closer to the hardware and hence it will always be associated with some cost. And hence like even it being open source, I think, won't help as much. And I think that's kind of what we're seeing, like even If you have the alpaca model, for instance, or the llama model, and you could run it, I think we will actually see, you know, kind of side counterpoint to that, these models will get smaller and smaller and people, once there is an open source model that's roughly equivalent to even G P T three point five, we're seeing this now, I think it'll be similar with G P T four. I think people will learn how to make them even smaller. Now there's four bit L M's that's kind of work with a reasonably small reduction in in performance. So the models will get smaller, so maybe this is slightly less good, but certainly right now, it would still be true that

• 21:15Speaker 01

  even having a massive model that can only run on A one hundred for free, fully available, fully as functioning as G P T four. Well, you're still only useful if you have that A one hundred lying around and that A one hundred costs money. And hence, you do need to have that connection to a data center. And hence, you might be right in that analogy that there is still a lot of value to be built on top of it. And these foundational models, similar to cloud providers, there'll only be a few. And then, of course, you can kind of try to have a little bit of a, you know, on prem in the cloud, a little bit edge computing and on the cloud mixtures and and building a lot of combinations of those. There's also now L M ops as a whole new space that's coming out, right?

• 21:56Speaker 01

  You wanna not be locked in with one cloud vendor, so you wanna be multi-cloud. People now wanna be multi-L M. We're already multi-L M two at you.com, you know, we have our own. We're partner of others, we have different partnerships and I'm adding to those. You know, there's a lot of a lot of that L M ops of like doing prompt engineering well, that probably will be bigger and not smaller. So Langchain, I think, is a really great example. There are others like on the data set side, like Active Loop and there are vector databases like Chroma. And I'm a little bit biased here, those are all A I X portfolio companies. But but yeah, you know, I think that's.

• 22:34Jacob Effron?

  You wouldn't be a good V C if you didn't shout out your portfolio companies on a podcast. That's just part of the job.

• 22:38Speaker 02

  Exactly.

• 22:40Speaker 01

  So I do think in this new software 2.0 stack, there will be a couple of interesting players. And then, of course, we're going to see a ton of new companies in their verticals sitting on top of either foundational models or very specific models trained on pathology data or radiology data and things like that.

• 22:56Jacob Effron?

  Yeah, I mean, I'm just curious, because one player we didn't talk about in this split between the foundation models using ton of compute on the closed source side and then the open source community is academia. And I'm actually curious, obviously, you did a lot of this cutting edge research that's very relevant to everything happening today while you were in your PhD. And obviously, academia drove a lot of the early ML innovations. Today, it feels like a lot of the advances are coming from private companies that just have access to massive amounts of compute. How do you think about the role of academia in the AI world going forward?

• 23:21Speaker 01

  It's a good question. And I mentioned this earlier, like, part of why I moved a little bit out of academia is that I felt like the main ideas, like, everything is in that everything should be trained end to end, everything is embedded in a vector, you want to have large data sets with very highly parallelizable compute underneath. Those ideas, plus a lot of fine tuning and optimization techniques for neural nets and so on. Those ideas have been around now for a while, and the biggest improvements have come from companies scaling them up. And indeed, companies are better at scaling academia. There are a few projects, like the Large Hydrogen Collider, where a ton of academics collaborate with engineers to build massive, massive experiments. But in computer science,

• 24:06Speaker 01

  for sure, that has not really taken off as well. And so I think academia is struggling a little bit right now with staying relevant and trying to not just kind of get the scraps. And this is, I think, part of why we're going to see an open source model, because you have a lot of very intelligent people whose entire career depend on being relevant to AI and so on. And they're going to work very hard to be able to get access to these models and then be able to improve them, understand them better, analyze them, and eventually, hopefully, train really large models again in a joint framework. But they do have to adjust and kind of come together and build bigger groups with also more engineering. I think the other problem that was sort of even I had

• 24:50Speaker 01

  Had that in my academic days is that, you know, you hire a bunch of really smart PhDs. They know they need to publish and if they don't publish, their careers don't advance. And so, you know, when I, when I pitched this like, oh, let's build like larger language models. It doesn't sound very publishable, right? It's just like an engineering problem. And instead, you know, the first author of these large language models like wanted to like had lots of other neat little ideas. And that he could implement quickly and experiment quickly on and then crank out another really neat paper like in the next quarter. But yeah, so there's some strategic issues of like, you give a PhD title to someone who was like one of three hundred people. And just did one small engineering aspect of this massive large system and he didn't invent all the nice math behind it and so on.

• 25:35Speaker 01

  This is very hard. It's like structurally academia is not set up for, for these, these bigger projects.

• 25:41Speaker 02

  What does that mean for the talent market? It's been really interesting to see how many people are moving out of academia and into building things. So, you know, does that continue or is it just going to be this like flood of folks out of academia and into building?

• 25:56Speaker 01

  I think so. I think especially, I mean, obviously academia is a big field, right? There's biology and so on, but they all need AI too. And I think protein models for large language models for proteins will also be big. But I do think in CS and AI in particular, we will see more folks wanting to apply all these exciting ideas. And take this idea of, you know, electricity and now apply it to all these different industries out there. I think there is just so much potential that it makes sense. And then also I think we're going to see, maybe you will in the future just not need a PhD to be able to do incredibly interesting work. You know, I've, I've hired a bunch of folks that didn't have a PhD and then very quickly did PhD level work. You know, my, my co-founder, Brian McCann, he had a master's in CS from Stanford.

• 26:41Speaker 01

  After doing his undergrad, the philosophy and computer science also at Stanford. And, you know, he did some of the most impactful work in the last decade. And, and built these really large systems like DECA NLP, you know, single model for, for the 10 hardest tasks in natural language processing. And I think we're going to see really smart. People needing less credentials if they can show, you know, GitHub results, self-published archive papers and things like that that have real impact, that have interesting ideas and can be verified to do so.

• 27:12Jacob Effron?

  Totally. I mean, speaking of applying this electricity and just the excitement around that, you know, one thing I'm struck by is just the breadth of stuff you've done over your career. Like, I believe you published on like protein generation, you published on like using AI for policy. You obviously invest now across a broad variety of areas. So I assume you spend 99% of your time thinking about search. But obviously, I'm curious, like, what other areas of AI get you most excited right now? I feel like you have a pretty interesting bird's eye view.

• 27:35Speaker 01

  Thanks. Yeah, yeah, it's true. Like, a big focus is on you.com. I am very excited about helping other people build amazing things with AI. And it's sort of my old professorial hat on, but in a very different way, in the sense that startup founders, especially first time founders are, you know, there's just so many things you can help with and mentor them and then help them kind of achieve their dreams and their goals. It's just a lot of fun. And so indeed, brought up the two that I'm probably outside of search and natural language processing and chat most excited about. And that is number one, protein not folding, which is interesting. But even more interesting is if you can generate whatever protein you want. And so this ProGen paper, as with many of our papers,

• 28:18Speaker 01

  I guess we didn't have as much luck with some of the reviewers in Nature and Science as some other learning labs, but it took several years. It finally got published in Nature, one of the Nature magazines. And that line of work has already spawned, I think, half a dozen companies. And finally, also, the first author of the paper, Ali Madani, started Profluent. I think it will change all of medicine in a very fundamental way. Like, you can, you know, once you have full control over the world of proteins, they govern everything in the human body, in health, in sickness, in cancer, in viruses, in bacteria, and so on. It's such a powerful underlying technology that I'm very, very bullish on that space. And it'll just be incredible when we finally, and hopefully in the next couple of decades,

• 29:01Speaker 01

  get a grasp on viruses the way we had a grasp on bacteria for a long time and still do for the most part. But even, you know, like antibiotic resistant bacteria. Like finding new antibiotics will also help with this technology. It's just gonna be like incredible. You're gonna be able to change your DNA in living like existing like adult people who have like a genetic disease or something like that and and just fix the problem rather than fix all the symptoms of it and so on. So it's gonna be a very exciting time in biotech. And my hunch is it's gonna be in the different waves. You know, biotech right now is struggling. So all the public markets are struggling and then, you know, that that kind of seeped into the private markets. But I'm I'm sure in the next decade or so or two that that will come back in a major way. The second one hasn't hasn't still taken off.

• 29:46Speaker 01

  It still reminds me very much of my early days in deep learning for natural language processing, where I had famous researchers say it makes no sense to squeeze a variable length sentence into a fixed length vector that will never work and and so on. And, you know, we've seen a lot of folks like really dislike this idea of using reinforcement learning at larger scales for economic policy making. And but this AI economist paper that we worked on, Stefan Chang is the first author, I think has a ton of potential, but it's early days and the field of economics is much, much slower. It makes you appreciate computer science where we don't have like every year you get a journal paper out. No, it's like, it's not even just every conference, right? You just get the paper out and archived whenever you're done with it, right?

• 30:30Speaker 01

  And so the field can already take that idea and iterate on it. But the field of economics is much, much slower in comparison. And a lot of, you know, older professors in that field just don't know all the math and and science behind reinforcement learning. And especially not this particularly complex example where you have an AI economist at the top that sets a bunch of taxes and subsidies and you have a bunch of AI agents that just maximize your own utility and you have to go back and forth in this loop that is highly known trivial to optimize because, you know, you essentially change your objective functions for all the AI agents every time you change your subsidies and taxes. But that can ultimately lead to some incredibly powerful philosophical questions that humanity

• 31:12Speaker 01

  has, you know, discussed, but also fought over in violent wars and so on for many, many centuries and millennia. And and I think, yeah, there's a ton Potential there for smaller organizations and maybe eventually smaller countries that are more tech forward to use that. Here's optimize. Say this is what our objective function is now. Let's run a billion years of optimization on this. And then find the most optimal setup attacks, different people. And then, you know, obviously have humans double check it and make sure it all makes sense. And there's no corner cases that the model kind of exploited that you didn't have in your simulation. But ultimately, I think that will have a massive impact on the economy.

• 31:49Jacob Effron?

  Yeah, no, they're both super cool papers. We'll definitely link to them in the show notes. You know, we always like to, I mean, this fascinating conversation. We always like to kind of wrap up our conversations with a standard set of what we call the quickfire round where we ask you about.

• 32:00Speaker 01

  All right. I'll try to be, I'll try to be more terse. I know I can.

• 32:02Jacob Effron?

  No, no, no, please. This is, I feel like we should, we, we, we could have you for hours here. This is fascinating, but I want to be respectful of your time. You know, I guess to start, I'll start with the topic de jour, which is like AGI. You know, I know you actually had a tweet thread today responding to the letter and kind of the six month moratorium proposal. But I'd be curious for your, just your thoughts on AGI, you know, is it possible and over what timeline? And then, you know, maybe just expanding a bit on how you think about the threat.

• 32:25Speaker 01

  Oh boy, how do I keep that one short?

• 32:27Jacob Effron?

  I'll ask you the broadest question possible for the quickfire round.

• 32:31Speaker 01

  I do, I do think there is no philosophical or technical reason why we will never have AGI. I think we will be able to get there eventually. I do think people are a little bit overly optimistic because we're currently at the beginning of an exponential. But most exponentials flatten out. I think of history and various cases where, you know, you look from the first human flight to people flying with ultra, the, the speed of sound and like faster than the speed of sound. Shooting machine guns, flying loopings and so on. If you kept extrapolating from, wow, we couldn't fly at all. And now a few decades later, we're flying faster than the speed of sound. Like surely in a few decades later after this, we're gonna just have a vacation on the moon over the weekend

• 33:15Speaker 01

  and then fly back down and circle the world in like an hour or two. And like, it's gonna continue like this forever, right? And the truth is no exponential in nature stays exponential forever. They usually flatten out. Now the question is where are we on that exponential? And can a digital such exponential stay longer before it flattens out? Because it is not as sort of tied to a physical nature and the laws of laws of physics. At the same time, GPUs are still, you know, physical also, and we do need that computing substrate. And so, long story short, I think people are overly optimistic on full AGI. I think there are a couple of things that no one's really working on. It used to be multiple things. Now, actually, with the combination of L M's that can code, we're kind of working on one of those three that I used to mention a lot.

• 34:00Speaker 01

  But one of them that still no one's really working on is giving agency to AIs to choose their own objective functions. It's like, you might wanna generate the next token, but maybe you don't wanna generate the next token anymore. Maybe you wanna just think to yourself in latent space and do whatever you want, right? And like, set your own goals. No one's really working on that because that's hard to make money with that. And so, I think that the people who are really clever saying, oh, we're gonna go to Mars. But then they just make money putting satellites into near space orbit, near Earth orbit, and just make money with that. But they have this longer term vision. And so, I think similarly, you can have an AGI vision, but the organizations that are clever about it just make money with the current thing that we can't do. And actually do change jobs massively and all of that. So, I guess that's one way to answer it.

• 34:45Speaker 01

  I could go on for a long time about all the other aspects of AGI. But yeah, I think it's really hard to not be optimistic because we're seeing a ton of exciting improvements. At the same time, you don't need AGI to be very excited about all this progress that is indeed happening. And it's changing so many jobs and like creating new kinds of jobs. It will change the economy forever. It'll change the human condition, all without the G in the AI.

• 35:10Speaker 02

  Which is is super exciting. And as an investor, like, it could not be a more thrilling time, I think. You know, I'm also a parent. I have a 10 year old. He's really getting into Python and JavaScript and coding, which I still think is probably good for him to learn for his brain development. But at the same time, as a parent, it's a little scary for me because I don't feel like I know, AGI aside even. I don't really know what the world is going to look like. And I suppose all parents have that question, but it feels especially acute now because of how much AI is gonna fundamentally change. Like, how do you think about that? How do you think about prepping a kid for his job in 15 years in an environment like this?

• 35:55Speaker 01

  I'm super happy that your kid is like learning Python. I was just chatting with the Chancellor and the German Cabinet a few weeks ago. And indeed, one of my recommendations for the country of Germany was make Python or programming a mandatory subject in all schools. I think that it's just so important to be able to know how programming works. It's just, it's going to be more important than, I think, a second language, for instance, that, you know, pretty much every kid in high school in Germany learns two foreign languages. And so, you know, obviously, English is very important, but after that, I think Python is more important than than the next foreign language. And so I think that's a great start. I think just helping them have as much fun as possible in learning new things and not

• 36:39Speaker 01

  feeling like you just learned this now to get a good exam grade and then you can forget about it and then you don't ever have to take exams again and then you're done with learning. I think that's sort of a mindset that I see a lot of people get into and that's going to be really challenging in the future. I think we will need to have lifelong learning as the labor markets, I think, are gonna eventually catch up with the speed that we're seeing frenetically in the AI and chat space right now. So the love for learning and lifelong learning and instilling that that joy of learning new things in people is a crucial crucial thing. And then I think creativity and a combination of knowing what things can scale and be automated and what things cannot scale or must not scale, right?

• 37:22Speaker 01

  And having a like a certain amount of empathy for the users that are might be affected by that technology. Once you combine all of those, I think, yeah, kids will do fine.

• 37:32Speaker 02

  He's gonna be just all right. We'll see, we'll check back in 15 years at what.

• 37:38Jacob Effron?

  We'll do a reunion episode in 15 years. See how it was. You know, I guess another thing we always liked, obviously, on AI Twitter, there's there's nonstop commentary all day. And I'm curious of the things that are often talked about, you know, one that you think's overhyped and one that you think's underhyped.

• 37:53Speaker 01

  I think overhyped is the fact that like these systems will be self-conscious and are an existential threat to humanity. I think that's really quite silly and it's not a research thing, it's just a cool sci-fi idea. So it should be part of the literature department, not the CS department.

• 38:10Speaker 02

  I love that.

• 38:10Speaker 01

  And I think underhyped, man, in AI right now, it's actually like, everything is so hyped. It's hard to find some things underhyped. I might go into the fields of application that don't get as much love. And I think one big one is economics. So going back to that paper, I still feel like that paper has not really seen its sort of convolution during that ImageNet type of moment where the rest of the field is like, oh, it's undeniable that this is better, even though they've shown that it provably can do things in like these smaller setups that traditional folks have, traditional like researchers have used, it can recover those provably correct results. But that whole field needs to kind of get more onto computation, large scale simulation, and large scale AI.

• 38:54Speaker 01

  And there is not enough hype behind it, even though on a massive scale, it is, I think, going to change the entire economy, right? But the truth is, especially in the US, like, working for the government and the economy is just not as interesting, right? It feels insanely intractable. You can't have that much impact as in a single individual and the whole area of economics and sort of policymaking is moving incredibly slowly. Sometimes for good reason, of course, you don't want to iterate as quickly as in a startup with, you know, all the taxes of everyone every quarter. But, you know, you can actually try to make more use of that technology. And then, you know, maybe I guess one level below that, even large organizations, I

• 39:36Speaker 01

  think, can use some of that to incentivize their employees correctly. You know, once you have like thousands of employees, you can also use some of that economic simulation to see how you best structure your organization. I think that area overall is under hyped.

• 39:52Jacob Effron?

  Yeah, I mean, it's interesting. You talk about this intersection of government and AI going forward. And I'm actually curious, you know, I think that intersection can happen in a few ways. You know, one could be also just governments getting involved in AI more, you know, as a matter of regulation, security. And then obviously, AI can influence policy and be part of government. So I'm curious, how do you see that evolving going forward?

• 40:09Speaker 01

  Yeah, I mean, I had helped a little bit the last month or so of the Obama administration. And they put out a really good paper on the IAC and the economy back then. And, you know, I have some hope that other countries like Germany that don't have as strong of a VC ecosystem and large company kind of acquiring companies kind of ecosystem can support innovation a little bit more. It's kind of interesting. I think the Silicon Valley is one of those unique spots on earth where when something new happens, the default mindset of a lot of people in this area is more like, oh, that's cool. What can I do with it? Rather than, oh, that could be scary and could result in changing my very comfortable status quo.

• 40:51Speaker 01

  And maybe it's a bad thing and I should like kind of be worried or regulated and so on, which I think most of the rest of the world is more prone to that mindset. And so helping people kind of balance it out, that that sort of Silicon Valley, let's innovate and improve existing technology, even when it's not yet perfect. That I think will be a big part for governments to try to get right in order to not fall behind.

• 41:16Speaker 02

  Yeah, as someone who has a couple of investments in amazing companies in Germany, I would love to see the ecosystem over there continue to evolve. And perhaps the government to put in place some better like structure and regulation or whatever to help companies grow. It's been an interesting learning experience for me navigating a lot of the red tape over there that you just don't have in the US.

• 41:40Speaker 01

  Right. Yeah. I mean, even just simple things like, you know, when you join a startup, your equity is obviously like theoretical and so you shouldn't get taxed on it as if it's real. Because you can't pay for those taxes. Like it's just like a simple thing like that that's still, I mean, the like I chatted with the cabinet and the chancellor and they're like, okay, well, we're working on it. And which is great, but yeah, it's still, it's still not fully executed.

• 42:03Speaker 02

  Yeah, lots of opportunities for improvement, as we like to say.

• 42:08Jacob Effron?

  I don't think the AI policymaker would approve of those policies.

• 42:12Speaker 02

  Yeah, there you go. Exactly. Because it would look at the economic benefit and be able to make the right the right judgments for the best outcome. This has been fantastic. I feel like we could ask you more questions and sit here for hours, as has been the case every time I've met you now. But thank you so much for making the time for us. Before we wrap up, can you tell people how they can, you know, follow you and your thoughts and also perhaps give you.com a try?

• 42:39Speaker 01

  Happy to, yeah. Check out you.com, Y O U dot C O M. If you want to have the next best search engine in the world that combines the best of, you know, chat G P like chatting and useful answers and summaries, still have those useful links that Google has. So you.com. And I guess I share some of my thoughts on Twitter. I'm just Richard Socher, my name. And yeah, you've started an interesting company. Happy to have you to take a look at those, too.