EP 87: Luis von Ahn (CEO, Duolingo) on The Marketing Formula that Propelled Duolingo to 500M Users

Luis von Ahn is the CEO and Co-Founder of Duolingo, the world’s #1 language learning application. In the episode, Luis shares the marketing formula they use to drive rapid user growth, including the backstory of Duolingo’s famous owl mascot that has become a viral internet sensation. Luis also dives into the A/B testing and product gamification experiments that have helped make Duolingo a $9B public company, as well as a handful of valuable lessons from his entrepreneurial journey. (0:00) Intro (0:54) Building a Culture of Iteration and Continuous Improvement (2:29) The Importance of User Retention in Duolingo's Growth (3:24) The Power of A/B Testing in Product Development (7:57) Balancing User Feedback and Data in Product Design (10:24) The Role of Marketing in Duolingo's Growth (15:54) The Importance of Setting Constraints in Product Development (18:35) The Role of Metrics in Decision Making (24:02) The Power of Viral Marketing and Memes in Brand Building (30:44) The Impact of Social Media Marketing on User Acquisition (35:41) The Impact of Passive Aggressive Notifications (37:42) Dealing with Controversial Content and Backlash (42:40) The Importance of Being Scrappy in Marketing (45:13) The Challenges of Teaching Languages (49:25) The Importance of User Retention in Duolingo (51:14) The Journey to Monetizing Duolingo (56:40) The Challenges of Introducing Monetization in Duolingo (1:04:16) The Expansion of Duolingo Beyond Languages (1:09:13) Growing User Base and Utilizing AI (1:11:16) Impact of Going Public on Duolingo's Operations (1:16:00) The Importance of Employee Selection in Duolingo (1:22:42) The entrepreneurial journey (1:33:46) Improving the Education System (1:37:21) The honeypot website to catch cheating students Produced: Rashad Assir & Leah Clapper Mixed and edited: Justin Hrabovsky Executive Producer: Josh Machiz 🎙 Listen to the show Apple Podcasts: https://podcasts.apple.com/us/podcast/the-logan-bartlett-show/id1606770839 Spotify: https://open.spotify.com/show/5WqBqDb4br3LlyVrdqOYYb?si=3076e6c1b5c94d63&nd=1 Google Podcasts: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zaW1wbGVjYXN0LmNvbS9zb0hJZkhWbg 🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1 Follow on Socials 📸 Instagram - https://www.instagram.com/theloganbartlettshow 🐦 Twitter - https://twitter.com/loganbartshow 🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow About the Show Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-based VC with $6B AUM and investments in Snowflake, DraftKings, Twilio, and Netflix. In each episode, Logan goes behind the scenes with world-class entrepreneurs and investors. If you're interested in the real inside baseball of tech, entrepreneurship, and start-up investing, tune in every Friday for new episodes. Executive Producer: Rashad Assir Producer: Leah Clapper Mixing and editing: Justin Hrabovsky Check out Unsupervised Learning, Redpoint's AI Podcast: https://www.youtube.com/@UCUl-s_Vp-Kkk_XVyDylNwLA 🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1 Follow on Socials 📸 Instagram - https://www.instagram.com/theloganbartlettshow 📱 X - https://twitter.com/loganbartshow 🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow About the Show Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-based VC with $6B AUM and investments in Snowflake, DraftKings, Twilio, and Netflix. In each episode of The Logan Bartlett Show, we sit down with the people behind today’s most important startups and extract the tactics, lessons, and frameworks they’ve learned the hard way. Conversations span hiring to GTM, product, growth, fundraising and everything in between - collectively forming the ultimate playbook to make you a better CEO, investor or board member. Tap follow and enable notifications to stay ahead of the game.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Logan Bartlett?

  Welcome to Logan Bartlett Show. On this episode, what you're going to hear is a conversation I had with Louise Von On. Now Louise is the co-founder and CEO of Duolingo, a $9 billion business in the public markets that builds an application for language learning. Louise and I talked about a number of different things, including how they built a culture of iterating on products and A/B testing different features, as well as how they got their mascot duo to become a meme and go viral on different social channels. Really fun conversation with someone that is very, very bright, including having a PhD in computer science, but applied that to building both Duolingo, as well as the prior company that he sold to Google called Recaptcha,

• 0:47Logan Bartlett?

  which you might be familiar with. You'll hear that conversation with Louise here now. I want to talk about like building a culture of iteration, A/B testing, continuous improvement, all that stuff. I kind of have an opinion that in some companies or some markets, there's an opportunity that opens up and it's just a debate of which company is going to run through and build the equity value that comes out of that. I'm not sure if you agree with this, but my sense is that you guys were in the right place, right time with regard to mobile and you were right about your thesis around language, but it was actually the culture of incremental improvement and just continuously getting better with the product

• 1:28Logan Bartlett?

  that has led to the distance, how big of a business you've been able to build. I don't know if you agree with that, that thesis, that it wasn't necessarily inevitable that there would be an $8 billion public company built around language learning.

• 1:44Speaker 01

  Yeah, I don't think it's necessarily the case that an $8 billion company needed to exist in language learning. It's true that we were at the right time with mobile for sure and we capitalized for that, capitalized on that. But I really think we were able to create a product that...

• 2:06Speaker 01

  Got people very engaged and such a thing didn't exist for language learning. And I'll tell you that the biggest step that supports that 80% of our users in the United States were not learning a language before Duolingo.

• 2:20Speaker 01

  We basically massively grown the market in certain countries, not in every country, but in certain countries, we've massively grown the market. And it's, I think it's just because, I mean, the main reason for it is because for the first five years of the company, the only thing we did was work on making Duolingo have higher user retention. And that's not paid user retention, that's just free. We were not making any money for the first five years of Duolingo, we made zero dollars. We were not making any money. And because we were not making any money, we also weren't spending any money on marketing. So there was kind of nothing to do. I mean, we weren't spending any money on marketing, so there was not much to do there. We weren't making any money, so there was not much to do in terms of like, you know,

• 3:04Speaker 01

  working on the monetization of Duolingo or anything like that. The only things we could work on are teach better and keep people engaged. And so that's all we did for five years. We simply made the product stickier and stickier and it really made a big difference.

• 3:21Logan Bartlett?

  How do you build a culture of like A B testing in iteration and people willing to take chances, knowing that you might make mistakes and that you can undo them if so?

• 3:32Speaker 01

  It's not like when we started, we thought we're going to build a company around A B testing. This is not, you know, the way it started is we started, we started with the product and we would make some changes to it. And we would make like whatever the changes were like, okay, we're going to add a screen at the end of a lesson that has fireworks or whatever, random changes. And two things were happening. Number one.

• 4:02Speaker 01

  We would add the screen and we couldn't tell if the thing was better or not. Because, you know, we would, we would see that the metrics were like either kind of flat. But if they were flat, we always found an explanation like, oh, it's because, it's because this week is Thanksgiving. Or whatever it is, and that's why normally we would have gone down. But because we, you know, added the screen, we didn't go down. Or something like that, like whatever, you just couldn't tell if things were actually helping or not. And in addition to that, we, we couldn't tell which idea was good and which wasn't. Because we're like, well, if we couldn't tell if that was a good thing or not, we, you know, I don't know, was that a good idea or not? And so at some point, it occurred to us, you know what?

• 4:44Speaker 01

  We should actually run one A B test and figure out if, if that thing was actually good.

• 4:51Speaker 01

  And we run, I remember we ran a test. I don't remember exactly what the feature was, but I remember there were like four options. And we, each one of us bet on which the option was going to be best. And we thought of ourselves as very good product people. And it turned out, none of us were right. And then after that, we thought, oh crap, I guess we really had no idea. You know, we ran the A B test, the thing that won was not the thing any of us thought was, was gonna win. And then for the next few weeks, we ran a number of other A B tests. And we always would say, try to predict what was going to win. And we realized we're actually quite crappy predicting what was going to win.

• 5:30Speaker 01

  And that's when we really convinced ourselves, well, look, we don't really know what is better for a product. I mean, we have some idea, but it's not, you know, it's better than 50%. We're not much better than 50%. So, you know, at that point, we kind of started this rule that every change of Duolingo is an A B test. And so we built a pretty sophisticated infrastructure, which was way ahead of its time. By now, I think, you know, this was more than 10 years ago, it was way ahead of its time in terms of how good it was also for the size of company. Maybe Google at the time has such a good infrastructure, much probably better than ours. But for a small company, you know, it was kind of unheard of to have this infrastructure. But we built an infrastructure that really allowed us to make everything an A B test.

• 6:12Speaker 01

  And I think that, that really made a, made a big difference. The other thing that I think in terms of a culture. Yeah, we make, we make it clear that it's totally okay to run A B tests that don't succeed. And you know, we have the stats and you know, I don't know off the top of my head anymore, but I mean, we run something around 500 A B tests per quarter. And look, something like 60% of them succeed and the other 40% fail. That means hundreds of A B tests that we run per quarter fail. And it's okay. Everybody's, everybody's done that. And I also like to tell the story that some of the best features of Duolingo uh have been things that I've been vehemently against at first.

• 6:54Logan Bartlett?

  Most notably, the, the streak, I think, was one that you said you originally pushed back on. Is that right?

• 6:59Speaker 01

  No, the streak actually was not. The streak I was, you know, you don't really remember anymore, but I'm pretty sure.

• 7:06Logan Bartlett?

  Who came up with that originally?

• 7:07Speaker 01

  I'm pretty sure I was the one who brought it up, but there are, there's more than one person that will tell you that they're pretty sure they're the one who brought it up. So.

• 7:15Logan Bartlett?

  Success has many fathers, right? Or mothers.

• 7:17Speaker 01

  Like I'm not, the thing is, I'm not even lying to myself. Like I actually have a memory, remember, you better remember that. But there are other people who claim that too. So, and I believe them. So we don't really know who actually. So I was not against the streak, but I was against our leaderboards. We have this league system in our leaderboards. I was against that because we had tried leaderboards many times before and they had failed. And yeah, this was like the fourth attempt and somebody came and they said, we're going to do leaderboards. I'm like, no, don't do that. You've, we've tried it. We've tried it many times. It's just not going to work, but they were very um committed to it. And it turns out it was one of the best things we've ever done, the, the league system at Duolingo.

• 7:55Logan Bartlett?

  How do you think about like listening to people's feedback? I've heard you use the example before of like, people thought it was too bright. The green was too bright and they all complained about it. And then you said you changed it and they said, okay, wait, never mind. It's not too bright, even though nothing actually happened. How do you think about like trusting the data versus listening to the users and the balance between that and product design?

• 8:18Speaker 01

  A couple of things to say. The, the good news about Duolingo, what makes it a little easier about Duolingo was that we can be the users too. There are some products that you've assigned where you're not the user. I mean, for example, for a while I was, you know, we, we, we have a Duolingo A B C, which is an app to teach you how to read. I'm not the user. I know how to read. I just, I'm just not the user. I'm also not like four years old. Um, but for Duolingo, I am actually the user. And so. For that, we do take advantage of that. I mean, everybody that is in the product team at Duolingo, we ask them, you kind of have to really be a big user of it. And that makes a big difference. So that's one thing. You listen to yourself. When we talk to users, we have to take everything they say with a grain of salt.

• 9:04Speaker 01

  We just, we have found, that doesn't mean don't listen to them, but we have found, definitely do not build features based on the things that you've asked. I mean, usually there's a deeper reason for why they're asking for a feature. So what we're interested in is not, what's your idea for a feature? We're not very interested in that. We're interested in what, you know, where you're having, where you're confused, or where, you know, where you're having trouble or something like that. And if you have an idea, we want you to tell us why you have an idea. Why is it that you're suggesting that? Because that usually gets at what the actual user problem is.

• 9:42Speaker 01

  So, you know, we do, we do user research, we try to all of that, but ultimately, I think what works best for us is we have a group of people at Duolingo, which fortunately has grown over time. But it's a group of people that started with one or two people, who just, because we ran so many A B tests, we had a really good idea of what worked and what didn't. And by now, we have a group of, I don't know how many, call it 15 to 20 people in the company, who are just excellently

• 10:14Speaker 01

  attuned to what works and what doesn't with an A B test. They're very good. And I think that's what's worked the most. We just, when you have, under your belt, when you have hundreds of reps, where you're like, I tried that, and you know, did it work? No, it didn't work. I tried that, and you know, did it work? Didn't work. And you do that hundreds of times. You just get a really, really good nose. And I think that's the thing that has worked the most for us, having this group of people.

• 10:38Logan Bartlett?

  I've heard you say that when you find a metric that you want to optimize, you'll actually form a team that focuses on the optimization. And how does that, is that a cross-functional team, across product engineering, marketing? Like, what does a team look like? Maybe give an example of a metric that you thought about, like how it actually comes to be.

• 10:55Speaker 01

  Yeah, I'll give you some examples. I mean, one, for example, is

• 11:00Speaker 01

  a metric that we call time spent learning, P S L. A while ago, a few years ago, we decided, you know, we should have people spend more time in the app because the more time they spend, the more they learn. It's very simple.

• 11:15Speaker 01

  So let's form a team around it. And what a team usually is, is we have

• 11:22Speaker 01

  usually two team leads, sometimes three. Usually there's an engineering team lead and then there's a product manager team lead. Sometimes there's a design team lead and sometimes there's a learning scientist team lead.

• 11:38Speaker 01

  But usually it's two. And then we put in that team, we put designers, we put engineers. We may put one or two more program product managers, depending on how many, how big the team is.

• 11:51Speaker 01

  And then they work and they can touch any place in the app. After a few months, they may figure out what moves that metric. In the case of time spent learning, what moves the metric is the leaderboards. And so the time spent learning team mainly touches the leaderboards. That's what they do.

• 12:11Logan Bartlett?

  And as you think about the unintended consequences of setting these metrics, I think I've heard the example of the streak. People were just signing into the app just to keep their streak going. Maybe speak a little bit about that. Or the Groupon example of when a channel gets too saturated and the balance of local optimization versus long-term value creation.

• 12:39Speaker 01

  Yeah, this is the tyranny of metrics. I mean, but we have to watch out for that a lot. Our teams are so good at optimizing metrics that sometimes they lose sight of what's actually important. They're just, it's just a machine to optimize a metric. And they're very good at it. And by the way, the people that we hire. You know, usually these people that have optimized their entire life, I mean, they were probably the best student in their preschool. I just kind of optimize the entire absolutely massively good at optimizing and they'll optimize the hell out of something. And so we have to really watch out. And you know, the way we watch out for that is by, we have this process called product review, where, you know, every

• 13:21Speaker 01

  change of the app to the app gets reviewed by a senior group of people who are kind of overseeing the whole thing. So they have to keep in mind, you know, they're the ones who stop spamminess and stuff like that. But, you know, when you just let teams just fully optimize a metric, basically just family behavior. And, you know, I, I this, we have a, a story that we use inside the company a lot, um, which is comes from Groupon, from the founder of Groupon who told us this story, um, who said, look, originally with Groupon, we have this rule that we could only send one email per day per user. Because who wants to receive more than one, you know, kind of marketing email per day from a company?

• 14:03Speaker 01

  And, uh, at some point, some product manager said, well, can we try sending two? And, you know, the founder said, no, we only send one. And the product manager used this amazing argument, which is, are you against knowledge? We just, let's just, let's just see, let's just see, can we, and of course, that's a very hard argument to argue against. Then, no, of course, I'm not agreeing. So it's fine. You try, but there's no way in hell that we're going to launch two. Like, we're just going to see what it does. So they, they tried it. They saw what it does. It turned out the metrics were way too good not to launch. It would have been irresponsible not to launch it. Like, my God, this, we preach everything so much. And so, you know, of course, they, they, they launched it.

• 14:47Speaker 01

  Um, and, uh, then, you know, I was like, well, we tried two. What about three? They tried three. Three was better. They launched it and they had A B tested their way to a larger number. I don't remember the exact number, but like seven emails per day. And then immediately after that, the whole channel died.

• 15:06Speaker 01

  Because everybody started marking their emails as spam because it's too much. And then they just A B tested their way to killing the channel. And the reason for that is because they didn't have full information about what was going on. All the metrics they could see were good, but they actually couldn't see how many people were marking your emails as spam. And that's always true. You never have full information about what's going on in the users. You just have some metrics and it's from what you can see. It's good, but you're not seeing, for example, uh, that maybe you're you're getting people to spend more time in the app. But you're actually frustrating them. You're not seeing their frustration. You're just seeing the time and you're like, okay, well, that's good. And so, you know, we we combat that by basically. You know, it is okay at Duolingo. There's so many tests. We just don't run.

• 15:49Speaker 01

  Even if when people say, are you against knowledge? I'm like, yep, I actually am against knowledge. I don't want to know what happens when you send thirty notifications in one minute. I don't want to know. Sorry. And so because because in general, we do things. And that's why it's good that we're users of the app. It's like, would you want to receive seven emails per day? Uh, that are all marketing emails? Would you want to do that? No, I wouldn't. Okay, then don't do that. And I think that has really helped us um, do the right thing here and not. Drive our product to the ground. I think a lot of products, particularly games, drive themselves to the ground. When, you know, they're just like, well, we're just gonna monetize more and monetize more and monetize more.

• 16:28Speaker 01

  And it works for a while until at some point, it's just like, you know, I, by the way, I wouldn't know how it is with Facebook anymore. But at some point, Facebook, the Facebook app. Kind of just had so many ads that even I think they kind of ran it to the ground too. Um, I mean, I haven't used it in a long time, but this was, you know, if they started with like one ad per. Whatever ten posts at some point, it was, I don't know how many ads, you know.

• 16:52Logan Bartlett?

  Do you set constraints on, I mean, push notifications and emails is a very obvious one. But are there constraints about things that you won't change within the app? And those are non starters or is is push notifications really a uh, a one of one thing? Because it can actually be marked as spam.

• 17:12Speaker 01

  No, no, it's not just um. We have a lot of channels in the app. That our product managers get very good at. So for example, when you finish a lesson on Duolingo, there is a sequence of. Screens that we call session end. So there of course should be one that says you finished and here are your plans. But we start adding other ones. For example, you increased your streak. Good. You increased your streak. That's another screen. But we have other ones too, without I just like, hey, by the way, do you want to subscribe to Super Duolingo? Or we add another one. Oh, by the way, have you tried our podcast? It's basically an ad. We just have decided to start adding things in there. And it works every time you add another session and screen, it does what we want it.

• 17:56Speaker 01

  But you can't have thirty sessions in screens after you finish one lesson, right? So this is just like, at some point, if you finish one lesson, you have thirty screens that you have to click through. The channel dies. I mean, it just, it just makes the whole thing so, for example, we have limits, you know? We just cannot have more than a certain number of screens at the end of a lesson. Uh, and, and then there's a whole pretty sophisticated system that tries to, it's, it's kind of like an auction. It's not exactly an auction, but there's, there's a pretty sophisticated system that tries to figure out, okay, if I can only show three, which three should I show? Um, that's the type of thing that we've done. And there's a lot of channels like that in the app. And, and you know, I'm still going to say red dots. Turns out that if you put a red dot somewhere in the screen, people are going to tap on it.

• 18:38Speaker 01

  Well, after a while, when we, when that was discovered by a, uh, enterprising product managers, the app had like thirty red dots all over the place. And then the channel dies because if there are thirty red dots, you don't click on any of them. There's just thirty red dots. Um, so that, that's the type of thing that, that, you know, we use the Groupon story a lot for.

• 18:57Logan Bartlett?

  And then how do those things roll up to a decision? Like, is it, does it ultimately land on your desk or someone else's that like, hey, maybe it might be penny wise, pound foolish on, uh, on these types of tests? And, and this is a little hacky or gimmicky to approach it this way?

• 19:14Speaker 01

  Every Tuesday and Thursday for two hours, we do this thing called product review, which is, I am there. There's usually three reviewers. There's me, there is somebody from product manager and somebody from design. And every change to the app gets 15 minutes locked and we decide whether it's okay or not. And after product review, uh, we spend, uh, what we call debrief for that day. And we just go through everything that we approve or didn't approve. And we make decisions about like, hey, um, this seems like a, like a more general thing. Let's, let's add on, we, we call, we, we internally have been calling them fat was, which had like, don't do that anymore. So that's the type of stuff.

• 19:56Speaker 01

  There's a lot of feedback in, in this where we, we try to make ourselves a better product team that way. So we spend, yeah, I mean, every Tuesday and Thursday, multiple hours kind of discussing what is okay and what is not okay in terms of, um, features to add.

• 20:12Logan Bartlett?

  If you were, uh, an entrepreneur or, uh, an executive at a company and we're trying to copy or mimic elements of just how iterative of a culture you all have built. I sort of think like consumer apps, maybe gaming is the furthest extreme of just what a knife's edge it's on in which you need to iterate and do as much as you can to keep users. And then enterprise software is probably on the, the far other side.

• 20:39Speaker 01

  But you were trying once three years ago.

• 20:41Logan Bartlett?

  Yeah, yeah, exactly, exactly. If you were trying to inject some more of this DNA, uh, into a company, is there any tactics or anything that, uh, outside of what we've spoken about here or philosophies that you would espouse to a founder that's thinking about this?

• 20:59Speaker 01

  You know, one thing you did say right is I don't believe that this iterative approach, it's what's best for every single product.

• 21:09Speaker 01

  It happens that with Duolingo, a lot of things came together that made it so that it works really well. So for example, We don't have to pay licensing fees for stuff. We don't license music that doesn't allow us to play this song in that country or whatever. We don't have to worry about that. So we don't have to worry. We don't have to worry about that type of stuff.

• 21:32Speaker 01

  We don't, we don't have, we also don't run something where it's like life or death, where something changes. Somebody's gonna, you know, they're gonna lose their job or whatever. We can easily just change the app at any point in time. So there's just a lot of things that, that came together so that it makes it so that we really can just continually improve the product. And with the other thing is that we can deliver the product, you know, we can change the app often. I mean, we, we change the app once a week. There's a new version of the app once a week. So we can change the app often. There's a lot of this that, that makes it so that we really can change very fast and can iterate. And because we can, then we do it. And, and if what's what, that's what you want to do.

• 22:17Speaker 01

  And I would say the most important things are make it so that

• 22:23Speaker 01

  there's no shame in failing for the A B tests. And that, that's very important. Make it so that you can

• 22:32Speaker 01

  iterate pretty quickly. So

• 22:36Speaker 01

  I mean, that's, those to me are probably the biggest things.

• 22:40Logan Bartlett?

  Now shifting gears a little bit to marketing, I'm not sure I would have guessed that a professor at Carnegie Mellon, starting a company would be the, the one, especially a computer science background, right? It would be the one to build a world-class marketing organization and brand. And it's, it was not intuitive to, to me. Can we talk a little bit about how you think about marketing and where that fits in to Duolingo's strategy, including maybe brand and paid user acquisition, influencer, organic, all that stuff?

• 23:16Speaker 01

  Yeah, marketing has had a long history at Duolingo. I think

• 23:21Speaker 01

  you know, the first several years

• 23:26Speaker 01

  we had a marketing budget of zero. So can't spend any money. Just can't.

• 23:34Speaker 01

  The reason for that was because we were making no money and I thought it was silly to spend money acquiring users. And by the way, you know, when we were launching, everybody was, that's kind of when performance marketing was getting like, you know, really growing. Where you're like, oh, my God, you can just go to Google and pay them five bucks and they give you a user. It's amazing. Just do that. And I thought it was silly to do that, you know, pay Google five bucks or however many dollars for a user when we were going to make no money from a user. So that just seemed like throwing money away. So this is why we never got into the performance marketing boat. We just didn't. But what we did, you know, at first, marketing, for us meant PR.

• 24:12Speaker 01

  And it actually, when you were small, first, when you were small and be ten years ago, PR was actually quite a strong channel. I think it's a much less strong channel now because, you know, it's kind of social media has kind of taken over a lot of that. Generally, news outlets don't have as much traffic as they used to. But that was quite a big channel for us. And so we had a good story for that, and that helped. The other thing that helped us is that we had this green owl that was kind of weird. And I think people started taking to it. And so you would see like posts on Reddit or posts that would go viral, but it would be, you know, kind of once every few months.

• 24:52Speaker 01

  A post on Twitter or a post on Reddit or one of these would go viral. And I had to do something with the owl or something like that.

• 25:01Speaker 01

  We became significantly more professional with marketing when we hired our first CMO, who was Kami Dunaway. Kami had been the CMO for, you know, very fancy company. She was in the heyday of Yahoo. She was the CMO for Yahoo. Then she was the head of all marketing at Nintendo, which is this crazy, crazy marketing budget. And then she came to us and then, you know, I said, well, we have no marketing budget. And then she said, obviously, I can't operate that way. You gotta give me something. And she convinced me, and I don't know, I don't remember anymore, the first year. She may have convinced me for something like thirty million dollars of marketing, which to me seemed like a.

• 25:45Speaker 01

  My God, what are you doing? And for her, it must have been the smallest marketing budget she ever had in her entire life. But she was really quite a trooper for it because she was like, okay, well, I'll operate with this. And she started trying a lot of stuff and she hired a really good team. And she started trying a lot of stuff. The majority of it didn't work. We tried stuff like, oh, you know what? We're gonna try a radio ad for Latinos in the US who wanna learn English. Stuff like that. Kind of, a lot of it wasn't working, but what we did start noticing was that more and more, there were these memes kept popping up relating to our owl. And we weren't doing much for that.

• 26:29Speaker 01

  We were just, you know, we were trying our own marketing stuff, whatever it is we were trying. It's not that it was working zero, but it was not home runs. But more and more, we started noticing that like, oh my God, there's another, another meme of some person that decided to say that they're scared of the Duolingo Well because it's gonna kidnap their family. And that's when, you know, Kami and a few people in the team and me and all of us decided, you know what? Why don't we lean in on this? That works. Like, a lot of the stuff we're working on, like we're doing, you know, it's, it's half-assed working. It's not, not working like that. So we started leaning in on it.

• 27:11Speaker 01

  And we had a group of people that she had hired that were just really creative and really had their finger on the pulse. And then we started, you know, I remember we did one video, which was an April Fool's campaign, where we hired, the idea for the April Fool's campaign was that we were gonna make a video of one of these really cheap-looking lawyers, lawyer videos, where they're like, you know, "I will get you millions of dollars." And, and we, we saw, we thought, okay, let's make a video just like that, like the crappy video, saying, uh, for a lawyer that is looking for people who had been kidnapped by the Duolingo Well. And you know, there were so many good things. By the way, a really interesting thing is we actually went to try to get one of the companies that make these crappy videos.

• 27:56Speaker 01

  And we talked to them and we're like, hey, but we want to like crappy like the videos you make. And they're like, what do you mean crappy? Like we make excellent work. Okay. And they kind of didn't. And in the end, we didn't hire any of them. We actually hired a very, very fancy company. And told them to make it crappy. And they actually understood how to do that. And it was very good as opposed to telling.

• 28:16Logan Bartlett?

  Just do your normal thing. Just make that.

• 28:19Speaker 01

  No, that wasn't working so well. We ended up hiring a very fancy company to make a very crappy looking video.

• 28:25Logan Bartlett?

  What year is this that you guys are doing this?

• 28:27Speaker 01

  2019, maybe that was 2019, I think.

• 28:30Logan Bartlett?

  So there were some precedent, like I sort of think of Dollar Shave Club and their video back in whatever 2013, 14, that's kind of the first satirical, brandish video out there. So you guys had some precedent. They did a great job with it.

• 28:47Speaker 01

  We did, but we weren't thinking about that. We, I mean, I do know about that video and everything. We just went, we were just like, look, this is what, this is what people are saying anyways. So we put out that video and it went really quite viral.

• 29:00Logan Bartlett?

  And the video, by the way, to be clear, I mean, it's a very inside joke of, okay, so, so the push notifications you have, which I want to talk about in a second. People made memes that they are aggressive in nature or passive aggressive in nature. And so that the owl is going to come find you. And then you made a derivative joke about that, which is if you're a lawyer, you're a lawyer looking for the people that the owl actually stole. So it's like a very inside joke, right?

• 29:35Speaker 01

  We actually made multiple, we made multiple derivative jokes for it. I mean, we made another one where we also, as a joke product, we said, you can, you can actually pay us to send the owl to like, stand in front of you until you do your lesson and stuff like that. So yeah, we leaned in on that and that started working really well. And, and, and that's

• 29:56Logan Bartlett?

  And by the way, working really well as a percentage of, how did you even think about the, you guys are so data-driven. How did you think about the ROI or the, could you just see sign-ups?

• 30:05Speaker 01

  We didn't have much ROI at first. All we could tell was that the videos or the things we were posting were being watched a lot. That's the only thing we could tell. Eventually, we ended up getting data because one of the marketing people decided that they were gonna, that they were gonna do a, you know, we added a screen at the beginning. When you sign up to do a lingo, how did you hear about us? And we did a lot of number pension there and we figured out there's really a very high correlation. People actually answer pretty truthfully there. It's not a hundred percent, but it's pretty truthfully. And we are able, we use that data and we're now able to really figure out where things come from and that really works. But at first, all we knew is that a lot of people were watching our stuff.

• 30:47Speaker 01

  And we weren't paying for it, which, which was great. I mean, people were watching our stuff and we weren't paying for it.

• 30:53Logan Bartlett?

  How did you, I mean, this is something innately a founder can do, I guess, but you're, you're taking a chance by making your brand and icon of the owl into a meme even more than just the internet's already doing, right? And so how did you think about like, clearly you must have a good sense of humor to find this funny yourself. But there's a risk associated with that, right? You're a learning app. You don't want to be taken as a joke necessarily.

• 31:24Speaker 01

  And we discussed it. There was a risk and we discussed it, but in the end, you know, it, it was a risk, but it wasn't. It didn't seem like a existential risk at first. It's like, wow, I just published that video. What's the problem? And, you know, it kept working and we kept doing it more and more. I think that was, we kind of boiled the frog into it. I mean, it just didn't seem like that big of a risk at first. And I did learn something pretty big here. We hired

• 31:55Speaker 01

  a person who was straight out of college and Kami hired this person, a woman named Zaria. It was amazing. Who, you know, I remember, I remember her hiring meeting. You know, Kami was trying to explain that we should hire Zaria and, you know, I saw her resume. I saw the whole thing. I'm like, Why are we hiring this person? I don't understand. Cammy's like, trust me. Look, you have to hire some young people because they understand young people. You don't anymore, Louis. And um, just trust me on this one. I'm like, okay, Cammy. And Zaria came on and pretty quickly she said, you know what? You know where young people are on TikTok? I'm like, my God, TikTok. Fucking people dancing, they had, I don't understand it.

• 32:41Speaker 01

  And uh, it's like, look, let, let me make this video. They made a video. They showed it to me. I'm like, I, I don't even understand what this is. And I have no idea what this is. In fact, it's not gonna work. Cause nobody's gonna understand this, is what I told them. But they said, well, will you let us? I'm sure. Carry on. I don't think it's gonna work. Anyways, they posted a couple of videos and they went crazy viral. Like, yeah, I don't know how many. And, you know, I lost track. Twenty million views on crazy viral video. And I still didn't understand why, why it was even funny. I just did not get it. And one of the things that I understood, which I'd be a major shift, you know, most of my time at Duolingo, and even before Duolingo, I spent designing products.

• 33:26Speaker 01

  When you design a product, it is important that every single one of your users understand what you're doing. It needs to be understandable. And I was applying that lengths to our marketing. I thought, well, everything we put out has to be under, at least understandable by the world. This thing you put out makes no sense to me. How? In fact, most people won't understand it. And what I learned that with marketing, that's okay. Especially with things like social media that targeted to the people, you know, somehow TikTok targeted it to the people that understood it. And that was fine. And so, you know, that, that worked out. And we started really leaning in on TikTok quite a bit. And then we started getting worried that we were too reliant on TikTok, at least for our marketing. Um, uh, so we started, uh,

• 34:13Speaker 01

  Figure out how to get better at that. We started, we tried the same thing on YouTube Shorts and it works on YouTube as well. So by now, our TikTok and our YouTube provide about the same number of new users to Duolingo, which is awesome. The other thing is, we started replicating the same formula in multiple countries. So now we have accounts, TikTok accounts and YouTube accounts on, you know, in Germany, in Japan, in Brazil, in Spanish-speaking world, and in a bunch of different countries, in Vietnam, and they've all succeeded. And they all use the same formula. And, and so we're, you know, it's worked out really well, but, but the formula, you know,

• 34:58Speaker 01

  there's a number of things, but I think the biggest thing about the formula is, if you look at our TikTok stuff and our marketing stuff, it does, it's not selling to you. It's just funny crap. And like, there's no button that says, "Download now," or "Buy now," or anything like that. It's just, we now have built a personality for the owl and people love it. And I think that, that really gets a lot of people, you know, it's basically free marketing. We don't, we don't pay for that. It's not true that we don't pay for any marketing, but that, that we don't pay.

• 35:29Logan Bartlett?

  Was there a session in which you needed to come up with what the personality of, of the owl was? It's just, it's iterative.

• 35:37Speaker 01

  It's, it's evolved and it really didn't mean, I would say the first glimmer of a personality came, this was many years ago, from the passive-aggressive notification. And this one was one I added. And it was, it was as follows, but I didn't think it was gonna be this, this big or anything. It was, it was because, you know, we've always stopped sending notifications. We stopped using Duolingo. So we, you know, if you don't use Duolingo, we stop after like, it used to be about five days. We're like, okay, after five days, you know, whatever, you've given up. We're not gonna spam you forever. And it occurred to me, you know what? If we're gonna stop, we may as well tell people that we're stopping. Why not? So the last notification on the fifth day that we started sending was,

• 36:21Speaker 01

  "These reminders don't seem to be working. We're gonna stop sending them for now," or something to that effect. And it turned out this, we didn't expect. It turned out that really got people to come back because they, you know, it's this passive aggressive thing that they, you know, they felt bad that Duolingo was giving up on them. And that is, that is the first glimpse of personality that was just like, oh, this owl really wants me to be there and he's gonna be passive aggressive. And that's, I think that's kind of what gave rise to a lot of the memes, but I mean, you know, we didn't. And, and so the personality of the owl has really evolved over time. And it's been kind of co-created by a lot of different Duolingo employees, but also by the community. Because they've made a lot of this stuff. And so

• 37:05Speaker 01

  there, there hasn't been, no, what we have had meetings about, especially for some of the videos. You know, with a lot of the videos, we, you know, we are learning brand after all, and so we don't want to do things that are lewd. But with a lot of the videos, we are, um, we're unhinged.

• 37:22Speaker 01

  And so, um, we do have a system in place for not

• 37:29Speaker 01

  posting things that we're gonna regret later. The system has failed a few times. Um, but we do have a whole approval mechanism, uh, for not posting really crazy crap.

• 37:40Logan Bartlett?

  What, what do you do when you, uh, when it feels it's crossed the line and, uh, and something just offended people in a way that you didn't expect? You just take it down and, and move on?

• 37:53Speaker 01

  We usually take, I mean, we haven't, that hasn't happened too many times. Um, we do take it down, by the way. I don't know, good or bad, usually the most offended are the Duolingo employees. Um, so we, the, the times we've taken stuff down is more often than not is because somebody in the company or a group of people in the company are like, how could you do that? And so that's, that's what has happened. But you know, but there was only one time that where there was a major, major backlash publicly, which was when we commented on the, you know, I couldn't have told you this was going to be that bad. There was a, our account, our TikTok

• 38:36Speaker 01

  account left a comment on a video about the Johnny Depp Amber Heard thing. And the comment said something to the effect of like, you guys think Amber Heard is on TikTok or something like, I think that was the comment. And there was a huge backlash because somebody posted saying that Duolingo does not care about, um, uh, like spousal violence or something like that. And there was a huge backlash to the point where our social media team had to make their, their personal accounts private because they were getting death threats.

• 39:16Logan Bartlett?

  Wow.

• 39:16Speaker 01

  Um, so that's when we learned to stay away from, you know, there are certain things we just stay away from. Even, even if our comment, even if our comment is, you know, I don't think the comment was particularly incendiary, but you know, there are some things we just stay away.

• 39:32Logan Bartlett?

  We had one, one time, we have a, a, we'll joke on TikTok and Instagram reels and YouTube shorts. And we had one, one time that was, um, it was when the market had corrected and, uh, it was VCs six months ago versus VCs today. And the joke was VC six months ago were, spend as much as you want. Like, are you, are you really burning as much cash as you can? We can give you more money, blah, blah, blah. Then VCs today were, are you sure we need a marketing team? Is that something? And the joke was that the VCs were flippant in their advice, right?

• 40:15Logan Bartlett?

  Uh, people took it as we were making a joke about layoffs of a marketing department, right? And it was, The joke was about us and VCs, but I understood where they were coming from. So we got a big, that was a big outrage and you know, uh, it was something that we ended up, I, I think we apologized or something, but you just never, if you're going to take chances at these lines, inevitably you're going to touch one and you're not sure. I stand by that was just a good joke and we were making fun of VCs and not founders.

• 40:47Speaker 01

  It's impossible to tell. It's impossible to tell what somebody's gonna, you know, I'm herd mentality takes it, you know, they, they kind of go against you. We haven't had a, that was the last, this was maybe like a year and a half ago or whatever. We haven't had a big problem since then. Um, yeah. And, and I think it's important, by the way, if you're running a marketing team like ours, which I think this is a major home run machine.

• 41:12Speaker 01

  To be okay with it. So you know what I did when that happened and they, they thought I was going to fire a few of them and I actually walked to the desk of, in fact, the person that posted it and I said, it's completely okay.

• 41:21Logan Bartlett?

  To stand out is the risk being polarizing in some way. And so inevitably, there's going to be some people that don't find it funny and inevitably there'll be some percentage of people that are offended. And it's hard to say where it's going to come from. So that's, that's great that you went over it and said that that's, yeah. The, the account, the checks and balances are important in trying to get enough diverse eyes on the things. So that you could understand where the totality of your audience maybe is going to react to it. But it, it's the cost of doing business in some ways that inevitably you're going to get hit with some of these things.

• 41:56Speaker 01

  And yeah, we're going to continue. I mean, and, and by the way, we've had a number of things that we've had to pull. I mean, the most recent one, we had a hilarious video that the marketing team just made. It was hilarious.

• 42:08Speaker 01

  That we were going to post close to Halloween. We had gotten, we had gotten a famous person like in there. It was, it was great.

• 42:18Speaker 01

  But it had kidnappings. And the Israel Palestine conflict erupted right around that time. And so we didn't post it. We were so sad that we couldn't. I mean, obviously, there was no way we should have posted that. But in other times, this would have been an amazing video.

• 42:36Logan Bartlett?

  What about being scrappy on the brand marketing side? I'd heard a story about like you guys just hanging around, waiting for billboard inventory to pop up and then being able to take advantage of it. Can you can you tell that story as well as like other examples of just sort of being scrappy about opportunities that present themselves?

• 42:56Speaker 01

  You know, because historically, our marketing budget has been pretty small. I mean, it's still pretty small. Our marketing budget now is about 50 million a year, which for a company with a revenue is this, it's about, it was about a fifth of what it should be. Um, you know, we don't, we are very scrappy. I mean, there's this story with, you know, with the billboards we, and the billboard was going to be a funny billboard. It was, um, this was a billboard, not necessarily so much for marketing Duolingo, but it was for getting employees.

• 43:26Speaker 01

  So we just put a billboard in the 101 in San Francisco that said something or other like,

• 43:33Speaker 01

  own a home, work in tech, move to Pittsburgh. And it just said Duolingo or something like that. Um, yeah, I mean, we only got that because we got a really good deal. We just hung around for a long time until we got a really good deal. Um, and what we used that, we used that same idea in a lot of ways. By the way, that is exactly what we do with our performance marketing. So we do performance marketing. We don't do very much. We do. You know, I know, um, on the order of 10 million bucks a year in performance marketing, a little more than that, but not much more than that. And that's what we do. Um, and the, what we do with performance marketing is we, we noticed we were in a really good position,

• 44:15Speaker 01

  which is we don't particularly care which country our users are coming from. We don't particularly care at what time we acquire them. We don't, we don't need to acquire them. You know, there are some, some companies that need to acquire them, you know, for a specific date or for something, we just don't care. And so one of the things we do with our performance marketing is we just have a system that basically looks at the whole world. And so if, for example, right now ads are too expensive in Italy, we just don't do anything in Italy. We move all our budget to France or, or, or whatever. If it's too expensive in France, we move it to Vietnam or something. So it just, we're always shifting budget a lot between countries because we are, we're in this great position where we, you know, operate in every single, single country in

• 44:59Speaker 01

  the world or have users in every single country in the world. And we are agnostic as to where they come from. Um, and so that, that's the type of stuff we do. We're, we're, we're very, um, very scrappy with our budget.

• 45:10Logan Bartlett?

  I heard when you were starting Duolingo, uh, you had no experience in teaching languages. And so you read a bunch of books and they all contradicted each other or they, they had mixed messages on how to learn, how to teach. I, I don't know if that's true, but if so, like, how did you go about figuring out what the core nucleus is of teaching language and how to start?

• 45:35Speaker 01

  We didn't, I didn't know how to teach languages. I mean, I knew, I knew Spanish because that's my native language, but I had never taught Spanish. Um, Severin, my co-founder, knew German because that was his native language and he'd never taught German. And so, yeah, we, we read a bunch of books. We, you know, there were some commonalities about what all the books said, how to best teach a language, and we kind of took those. Unfortunately, the books didn't agree with each other. They all had their own method and they're all, you know. Um, so we took the commonalities and then we did what we thought made a lot of sense. By the way, we made a bunch of dumb mistakes that only computer scientists would make. So for example, this was, this was dumb. Um, and we thought, hey, yeah, this, this, this kind of approach to deconstruct everything. Language is just words.

• 46:20Speaker 01

  So all we need to do is teach you words and how to use them. And if you know all the words and how to use them, you learn the language. Done. And so we're lucky though, so that's what we're gonna do. And so the first, we are one of the first versions of Duolingo had just, we just were teaching you a list of words ordered by frequency. So you know what? The first word, the most frequent word is "the." Teach you "the" first. Then the second most common word, whatever the hell that is, teach you "that" word, etc. So that's a very bad way of teaching. Um, and uh, you know, it's also not true that because you've learned enough words to cover, the first like thirty words or whatever cover something like twenty percent of the language. It's not true, you know, twenty percent of the language by knowing the first thirty words.

• 47:04Speaker 01

  That is just absolutely not true. Um, but that's kind of how we thought about it. So this is the type of stuff that we were doing. Um, over time, we learned that we could run A B tests to figure out how to teach better. And so that, we started getting better at that. So we started kind of figuring some stuff out ourselves. Um, and then, but then we got significantly more pro. Um, when we hired our first person with a PhD in second language acquisition. Um, right, that was a, we should have done that earlier. Um, yeah, we got a lot more pro. Uh, and, and that, that has really helped. And now what's interesting is when we got these people with PhDs from second language acquisition, it's not like they know how to teach with an app.

• 47:47Speaker 01

  They didn't, because they know how to teach in a classroom setting. Um, but we knew how to teach with an app. Now, we had our own wonky ways of teaching with an app. And they had their own very professional ways of teaching through classroom settings. Um, what has been really amazing is kind of, you know, they've learned to how to take everything they know, and some of it applies to, to an app, some of it doesn't. They learn how to make that kind of closer to the app. And we with the app and with the A B testing and everything have learned how to go more closer to them. And so at this point, you know, I can, I can say that compared to ten years ago, Duolingo teaches, I know, way better. Um, but we've understood that actually there's just all these skills that you need to learn

• 48:31Speaker 01

  and that they don't carry to each other. Like, for example, you can be amazing at reading a language and have no idea how to speak it. And we just didn't understand that at first. So now, you know, we try to teach you all the different skills that are required to learn a language. And, you know, we understand that the way to teach better is at first, you really need to teach things about how to, you know, how to talk about completely about things and greetings and stuff like that. These are not the most popular or the most kind of high frequency words at all. But they are very useful to get started. And so that's, um, we've learned that.

• 49:06Logan Bartlett?

  In the, in the early days, when it was just you and Severin, you know, I think we talked about this a little bit before we started recording, but neither of you actually retained as users of the app necessarily in trying to learn new languages. Was there a, was there a single product unlock or an aha that you realized about how to retain users or teach people through it? Or was it just a continuous process of these little tests to get incrementally better every day?

• 49:38Speaker 01

  So what happened with, this is before we launched Duolingo.

• 49:43Speaker 01

  You know, I made the first version of the Spanish course. And Severin made the first version of the German course, and we were going to learn each other's language. I didn't know any German and he didn't know any Spanish, and we were going to learn each other's language. And we ran into this problem that neither of us wanted to do it. I mean, we would come into the office and we would say, hey, did you do your Spanish lesson? And you'd be like, no, man. So, so boring. And I thought the same thing. It was so boring to learn German. And that's when, that's when we realized that the hardest thing about learning a language by yourself is staying motivated. And so that's when we started like adding, um, you know, adding all kinds of game mechanics. I like progress bar at the top. At the end of a lesson, we give you points. The lessons should be short.

• 50:29Speaker 01

  There's all this stuff. And this is before we launched, so we weren't A B testing anything. But we added a bunch of stuff so that at least we could swallow taking a lesson. I mean, it's like, okay, well, at least it doesn't feel like a complete chore. It feels, yeah, kind of fun. And that's, you know, we launched with that. And so by the time we launched, it was kind of fun. It wasn't as fun as it is today, but it was kind of fun. We had done no A B testing. It was just, at least with us, we had made it, you know, kind of engaging. And but, but the, the learning remained, which was.

• 51:05Speaker 01

  Just making us engaging as possible. And so that continued.

• 51:11Logan Bartlett?

  What did you think this could be when you were starting the company? You had been independently successful selling recaptcha to Google. And so you were able to be a little bit more mission driven in your pursuits. I think, which my understanding was that was an important consideration in the early days. Did you think that this would be a commercial, potential public company? Or was it, hey, this is just a good service and maybe it's a good business along the way?

• 51:39Speaker 01

  I never thought this could be a public company. I thought, you know, I thought we could do something that would help teach people mainly English. I thought that that was going to be there. I didn't know how big it was going to get.

• 51:55Speaker 01

  When we were launching, Rosetta Stone was like huge. We thought at the time was huge.

• 52:03Speaker 01

  It's not even clear that we thought we were going to get bigger than them. We just thought, well, we're free and we're at least going to let people, you know, learn that can't afford to pay. And any recession was super expensive. It was like 500 bucks or something like, okay, well, we're going to let people learn that. You know, when they don't have to pay us 500 bucks.

• 52:26Speaker 01

  Before we launched, I thought if we ever get to about 100,000 monthly active users, that would be success. That's what I thought. And, um, you know, that's funny. I mean, we get, we get way more than that new users per day now.

• 52:41Logan Bartlett?

  There's a great YouTube video. I think it's around the launch of Duolingo, whatever, 14 years ago or 12 years ago, or whatever it is. But I believe it's on that video. You make reference to this, this concept of the cap on human coordination that historically had been around 100,000 people. I think that was true of the pyramids, Panama Canal, getting a man on the moon. That like, that's been the, when, when, when physical objects were involved, that was the most we could scale to as, as humanity. Can you talk a little bit about that and how you, how you thought that as an influence of starting Duolingo?

• 53:21Speaker 01

  So we always wanted Duolingo to be free. And it was one potential way to monetize it. We ended up not monetizing this way, but there was one potential way of monetizing, which was, look, maybe we can get a lot of people learning a language on Duolingo. And at the same time, while they're learning, they're helping us translate stuff. Because at the time, computers were just not as good. Today, they're essentially perfect, but at the time, they weren't as good. And they are today are translating stuff. So we thought we could, rather than charging people to learn a language or using ads, we thought we could use their work to help translate stuff on the web. And then we could make money from that. So for example, the idea was going to be something like, well,

• 54:02Speaker 01

  if you write a news piece in English, like if you're CNN and you write a news piece in English, you can send it to us. And our users, while they're learning English, they could help translate it into Spanish because they're native Spanish speakers and that's part of their learning. And then, once it gets translated by our users, we send it back to CNN. And then we charge them for the translation or something like that. We thought that was a potentially a good business model because it, you know, no money was involved, at least for the users. Turns out, this is a very bad business model. So we ended up not doing that. But that was, that was one of the original ideas with Duolingo to try to make money while keeping it free. It turns out a much better way to make money while keeping it free is to use a free me model, which is you can learn as much as you want, but you may have to see some ads. You don't want to see ads, you can pay us.

• 54:47Speaker 01

  Turns out that works really well. That was not clear back then, that this would, that this would work so well.

• 54:52Logan Bartlett?

  Yeah, so what year was it when, so Leila Sturdy from CAPG invested and told you, hey, you got to figure out how to make some money here at some point, right? What year was that?

• 55:05Speaker 01

  She invested in 2015, I believe. We, at that point, Duolingo, she invested a valuation of about half a billion dollars.

• 55:12Logan Bartlett?

  With roughly how many users? Zero revenue, plus or minus?

• 55:18Speaker 01

  Revenue, we must have had, yeah, I don't remember anymore. Call it. I'm gonna make up a number, call it a single digit, maybe one, two million daily active users.

• 55:30Logan Bartlett?

  One, two million daily.

• 55:31Speaker 01

  Active users. And if you looked at monthly active users, probably, uh, call it, uh, I don't know, six, seven million monthly active, something like that.

• 55:41Logan Bartlett?

  So, so a rich valuation, uh, if we're certainly for, uh, uh, those times, uh, maybe.

• 55:48Speaker 01

  Yeah, we were, the good news is we're growing a lot. So we had just, you know, we, the previous year, we had like 10X. So it looked like we're growing a lot, which was good. And, but, you know, she said to me, look, um, great product, great team, um, yummy money. And, uh, yeah, she, she took me out to drinks. After I did my third drink, she said, look, you're not gonna find, uh, she's used exactly this word. You're not gonna find a bigger fool than Google. Like, but this is it. Like, you, you know, we invested, we like you. This is great. We believe in you, but you're not gonna find a, you know, you're not gonna go get a valuation hype from anybody else.

• 56:29Speaker 01

  You're not gonna do that unless you make some money. So get it going. And then, um, I got to start some other types.

• 56:36Logan Bartlett?

  Yeah. Well, and so now at this point, you have a very mission driven company, right? That are very aligned with your democratizing access to, to learning languages and all that. And so you come back to them and say, hey, you're a little hungover after a couple of drinks with Leila. You start noodling it and you say, hey, we, we've got to figure this out. I assume not all of the company was, uh, was excited to start showing ads or charging subscriptions. Is that a fair, fair assumption?

• 57:07Speaker 01

  Yeah, no, they really were not. They were in fact, um, entirely against it. Um, most everybody that joined Duolingo joined because of the mission to really deliver education to everyone. Um, and so, you know, I said to everyone, we gotta, we gotta, you know, somehow pay for this whole operation. And what everybody, you know, first question everybody said is, why? They're, well, you gotta pay for your salary. And I was, you've been paying our salary this whole time. What's the, what's the problem? You know, that's when I kind of told them kind of what Leila had said. You, you can't raise venture capital forever. Um, and, uh,

• 57:45Speaker 01

  You know, people finally were like, okay, okay, fine. I thought this took like months for people were like, fine. You gotta make money. How are we gonna?

• 57:53Logan Bartlett?

  By the way, when you have that, I mean, a lot of our companies have dealt with different versions of this, in particular now trying to get people back into office or whatever it is, right? Different social issues or the mission-driven nature of a company and trying to corral it in some way, shape or form. I think as we're recording this, there's this, there's this large business called OpenAI that's maybe going through some, some elements of, of this. Like, how did you actually go about building consensus around getting people on board with this? It sounds like you didn't do it by edict and say, hey, listen, this is the way.

• 58:29Speaker 01

  No, no, it had to be consensus. It, it, Duolingo was generally a consensus-driven company. It had to. I mean, the way that works, we were, I don't know how many employees we were in that time, call it 150, maybe, keep or take.

• 58:42Speaker 01

  You know, there was clearly a spectrum of how much people cared and also a spectrum of how influential they were. And, um, you know, I spent time with each, with each individual who either cared a lot or was very influential, like one-on-one. And some of them were easy to convince. There were a few stragglers who were just like, really resisted the idea.

• 59:06Speaker 01

  But, you know, it took, it took a period of, I don't know how long it was, a couple of months of, you know, people being, you know,

• 59:15Speaker 01

  upset at, at this. And then when they started seeing the mox of how, they became even more upset when they started, the first time somebody saw a build of Duolingo with ads in it. People were like, what is this? That's awful. And it's because, yeah, it's true. I mean, the ads, the truth is the Duolingo product experience is quite polished. And then you put an ad which is looked like, sorry, but most of these ads that you play on apps look like crap.

• 59:40Logan Bartlett?

  Yeah.

• 59:41Speaker 01

  So, yeah, it, it, it took, it took quite a bit to, to figure that out, to, to, sorry, for people to get used to it.

• 59:50Speaker 01

  But, you know, the explanation that really

• 59:54Speaker 01

  Resonated most with everybody is this, if we make money, we can use it to really speed up our own mission. And it was 100% true. As soon as we started making money, we started growing the employees a lot instead of having like four separate product pipelines. Now we have, I don't know how many product pipelines we have. We have like 50 separate, you know, kind of lanes where people are just improving the product. And that's all because we, you know, we make revenue now.

• 1:00:23Logan Bartlett?

  How did you think about the line between subscription, which my understanding is that drives is a very small percentage of the users, but a high percentage of revenue and then advertising, which is the opposite. When you were thinking about this philosophically, how did you come up with where the lines of demarcation were?

• 1:00:41Speaker 01

  We didn't have too much philosophical. It was a lot of testing. The only thing that we decided to do is we're going to put an ad on the free pop that we're going to do. At the end of our lesson, we would put one ad that, you know, after five seconds is capable. We're going to do that. And we decided we were going to do. And then we added an edict that said, we're not going to degrade the free user experience further than that. That is it. We're going to degrade it by putting one ad at the end of a lesson. Don't degrade more.

• 1:01:13Speaker 01

  We didn't know whether ads or subscription was going to give us, you know, that much more money. In retrospect, I think, I think subscription businesses are a lot more well understood now than they were. At least digital subscriptions like that for apps than they were a while ago. In retrospect, this would have been obvious, but at the time it was not clear to us that ads or subscriptions, you know, which one was going to be the killer one. But after running it for about a, you know, a few months, it became clear that subscription was so much of a better business.

• 1:01:44Speaker 01

  And so, yes, to a, a give or take, you know, 8% of our monthly active users are paid subscribers. And 92% to use it for free, yet for a revenue, give or take 80% comes from the subscription.

• 1:02:00Logan Bartlett?

  And you guys philosophically will not charge people for content, right? Like that's, you don't want to become Rosetta Stone and gatekeep content too much?

• 1:02:10Speaker 01

  Correct. We don't charge people for content. The only things that are okay for us to charge for are things that we have to pay for. So for example, we now have a tier in Duolingo, which is Duolingo Max, which has some features that have large language model usage. We have to pay for that because we don't have our own large language model. So I have to pay for that. And because we have a variable cost on it, we do put it behind.

• 1:02:37Speaker 01

  You know, you have to pay for it because otherwise that would be a major money loser for us.

• 1:02:42Logan Bartlett?

  There's a rule I've heard you guys have that if if five people or less can build something as well, you won't charge for it. Is that is that still true? And how philosophically do you think about that?

• 1:02:55Speaker 01

  This is, you know, it's one of those internal, she called the five Germans and a dog. It's five Germans and a dog can build it.

• 1:03:04Speaker 01

  Why are what why can we charge it? Because, you know, the reason we decided not to do it is because that, you know, that's how you that's how, honestly, that's just how we beat Rosetta Stone. I mean, we weren't German, but it just made a thing that was free. And it was not made by 300 people. It was made by a very small number of people. And very quickly, we took them over in terms of usage. Not in terms of revenue, because we're making any revenue, but in terms of usage, we took them over very quickly.

• 1:03:33Speaker 01

  And so, yeah, we're going to get disrupted. If we build something and start charging for it, that can be built, you know, so easily. Some company that has a small amount of funding, I mean, for, you know, five people, you need five million bucks of funding or less than that. They can build it and eat your business. So that that's been the rule. Um, we're a little less worried about that these days. Just, you know, that was more of a worry a few years ago when when our dominance was not as big. I mean, at this point, it's just we have a lot of moats and it's it's become pretty hard to compete with us. So a little less worried about that. But that was a worry. That was a pretty major worry, I'd say, you know, six years ago.

• 1:04:14Logan Bartlett?

  So now you're actually branching outside of languages. How do you think about doing that? And then what's the hardest part? Is it, is it getting the brand? Is it the content? Is it rebuilding elements of the product? What's the hardest part of doing that?

• 1:04:29Speaker 01

  It's creating the whole learning experience. You know, we're branching out to math and music. There are some differences in how you teach math and music to languages. And so we're creating the learning experience. That's, to me, that's the hardest part.

• 1:04:42Logan Bartlett?

  What's the most interesting difference of how someone would learn a language versus math or music?

• 1:04:47Speaker 01

  Well, math needs more explanation than, than languages.

• 1:04:52Speaker 01

  It's also the exercises and there's a lot more graphical elements to it. It's just a lot of math. I mean, some of it is just numbers, but a lot of it is just, especially if you're going to teach geometry and stuff like that. It's a lot more graphical stuff. And, and you have to start really, you know, you have, there's a lot of things you have to kind of figure out that we've now just figured out with languages. I mean, we figure out, for example, with languages, I'll tell you, it took us many years to figure this out. And in retrospect, it's obvious, but look, if you're going to teach a language, you should, you should align yourself with this thing called the Common European Framework of Reference. It's called the CEFR. Just do that. Don't deviate.

• 1:05:28Speaker 01

  There's no such thing for math. For math, there's something like, it's called like the Common Core, but it's not clear to us that, that the Common Core is as good as CEFR. And so that's the type of stuff that we just don't know. And, and also for math, we're not entirely sure who our audience is. There's our audience we want, but there's an audience that we may end up getting that is not what we want. What we want is everyone. We want to do for math what we did for languages, which is, you know, 80% of our users were not learning a language before Duolingo. We want it to be the case that anybody who's 25 years old or 30 years old or however, or they're eight years old, they just use Duolingo to learn math and they get better at it. We want that to be true.

• 1:06:12Speaker 01

  It may end up being that the only people we get are the people who are forced to learn math because we just can't make math fun enough. But what we would like is, is to get to everyone. And with math, this is a real open question. Can we get it so that your average 35-year-old decides to spend five to 10 minutes a day getting better at math?

• 1:06:33Speaker 01

  Can we do that? I don't know. That is what the product team is tasked with. And we have a really good product team, but I don't know if they'll succeed.

• 1:06:41Logan Bartlett?

  Because it's interesting, and you alluded to this of getting to spend five to 10 minutes a day. But you don't view your competition as Rosetta Stone or a private language tutor or whoever. It's like, how do you think about your competition and what you're going up against for that individual?

• 1:07:01Speaker 01

  Our competition is time. And, you know, if you want to be more precise, it's time on the phone. So a real, you know, when we talk to our users and we ask them, you know, the ones who like stop using Duolingo. What did you stop?

• 1:07:16Speaker 01

  I don't know what we've done. I'm sure they exist, but I don't know we've done any user research where they've told us they've gone to a competitor. Like a, like a language learning competitor. That's not what people say. The, the most common thing that people say, well, the first thing they say is I haven't stopped, which they're just lying about. But once they, once you convince them that yes, you know that they have stopped. They'll tell you, well, to be honest, I'm just spending more time on TikTok. Or Instagram or whatever the hell it is. So our real competition is, is, you know, the apps, games, whatever it is, the, the things that take up your time. On your phone.

• 1:07:55Logan Bartlett?

  Is there an optimistic point about humans and their desire to learn embedded in that? That they're willing to waste time on TikTok or Instagram, which presumably I think most of that is. Or spend it learning a language if the language experience is comparable or enjoyable or easy. Like, how do you, how do you think about the?

• 1:08:19Speaker 01

  I believe that. Obviously, we haven't gotten there. I mean, obviously TikTok has more users than Duolingo. But I do believe that.

• 1:08:27Speaker 01

  As long as you're close to as fun as TikTok or as engaging or however you want to call it, addictive, engaging, however you want to call it. As long as you're close, I think we have a good, good shot. Because afterwards, the good news about Duolingo is afterwards, you don't feel like you wasted your time. You at the very least got somewhat better at Spanish.

• 1:08:47Speaker 01

  Whereas if you spend 30 minutes, you know, scrolling on Instagram.

• 1:08:53Speaker 01

  No, you may learn a couple of things, but really mostly time wasted. Um, and so I, we think that as long as we're as close, as long as we're close to how engaging that is, we're in a good spot. And, and again, I think we've gotten somewhat close, but we're not there. Uh, you know, we have, I don't know, the latest numbers we released is something like 24 million daily active users. We don't have 500 million daily active users. So we gotta get there.

• 1:09:22Logan Bartlett?

  Yep, there's room to grow. Uh, in terms of artificial intelligence, you all have been fairly early in terms of using this. Can you, can you talk about your original desire to maybe build some of this stuff internally? And then, uh, where you all are in the evolution of, of using tooling and what that's going to mean from a user experience standpoint?

• 1:09:44Speaker 01

  Yeah, for artificial intelligence, um, you know, we've always, from the beginning for Duolingo, the idea was we're not going to use humans to teach you.

• 1:09:54Speaker 01

  Uh, there's a reason for that, uh, because humans are expensive.

• 1:09:59Speaker 01

  Um, you know, if we want to teach literally, whatever, hundreds of millions of people, you gotta find a lot of teachers and they're expensive and you gotta pay for them. And we're just not going to use them. Basically, we're going to teach you with, with a computer. Um, so that has always been the idea. And if you're going to do it with a computer, you know, you gotta make it so the computer can teach as well as a one-on-one human tutor. And so we've been working really hard to use AI to teach as well as a one-on-one human tutor. Um, there was a major shift, of course, in the last year, call it, because of large language models. I, and I don't think almost anybody else could have told you five years ago that large language models were going to be this good. Um, turns out they're extremely good.

• 1:10:44Speaker 01

  So, you know, immediately when they came out, um, we started, you know, applying some of our AI efforts into that. That's not the only place where we've used AI. We use AI for a lot of different things. Um, but at least for conversational practice, this is the thing we were never able to do. Uh, without large language models, we just could not build a very good conversational partner. But today, we can build in a really good conversational partner with a large language model.

• 1:11:13Logan Bartlett?

  How has going public made you all operate differently? Has it been a good experience? What would you recommend for for people maybe thinking about it?

• 1:11:24Speaker 01

  For for us, I think it's been an excellent experience. For me, it has been a really good experience. I think there's a couple of things. I think we, you know, I heard this from Rich Parton, who started Zillow and a number of other, you know, amazing companies. He said, look, going public is like joining the major leagues. Like, everything just operates at a higher level. And I believe that. I mean, just everything started operating on a higher level. You just are forced to do all these things that you didn't have to do before, so you don't do them. It's like, you know, when you get no visitors in your house, you're not really forced to clean. But if you're going to have visitors every day, you keep a pretty clean house. It's the same thing, like everything just, we operate a lot better, I think.

• 1:12:05Speaker 01

  For me in particular, I've been very lucky that we have a really good CFO and finance team. So I haven't had, my job hasn't changed all that much because a lot of the public company stuff is done by our finance team. And I'm, and other than just regular, doing regular earnings calls, I haven't changed all that much for me in particular. But I think I've had a fortunate experience for a number of reasons. But I think the part of the biggest reasons, reason why we've had a fortunate experience is that

• 1:12:38Speaker 01

  we got very good at forecasting and we don't over promise. And I think we've historically haven't, we haven't just been one of these companies that completely over promises. And so I think that that has helped.

• 1:12:55Logan Bartlett?

  Why do you think there there haven't been more successful consumer companies? As we look today, you all are at your all time high despite having gone public. You know, everyone else that sort of has gone public in the 2021 vintage or whatever are definitely below their all time high. I would say there, there aren't that many businesses that have been created between five and $20 billion, $25 billion in equity value and within consumer. Do you think? There's some innate cap that you either get super big like a Facebook or a Google or a Snapchat, or it feels like there's just not a lot that exists at that level versus

• 1:13:40Logan Bartlett?

  enterprise software. There's a ton. Do you have a perspective on that?

• 1:13:44Speaker 01

  I mean, we ask ourselves this question. There does seem to be something to, once you get kind of big enough, you just immediately turn into like a Facebook. There does seem to be a little bit of that. I think another thing that has happened, certainly for a lot of kind of the smaller consumer things, I think, I think a lot of them were fooled by performance marketing. I think we were very fortunate, not because we were so smart. It's not that we were so smart. It's just we just, because we were not monetizing, we didn't do performance marketing. But performance marketing is such a drug. It's such an addiction. And, and once you're in it, you can't stop it. And the problem with

• 1:14:28Speaker 01

  it is that really cuts into your margins to the point where it's very hard to, it's very hard to ever become, you know, very profitable when you're, when you're relying on a lot of these consumer apps that are not Instagrams of the world. If you look at where their users are coming from, it's overwhelming with performance marketing. When you look at like 80% of their users are acquired or something like that, you're going to have a shitty margin. But that's that. And so, and then, then yeah, investors are driving your shitty business too, because they have a shitty margin. Fortunately for us, we never did rely on performance marketing. And so our margins are really good because we don't do that. And I think that's it. I think, I think in retrospect, I can understand that. Again, it's not because we were so smart back then. But in retrospect, I can

• 1:15:13Speaker 01

  understand, look, the truth of the matter with performance marketing is that companies like Facebook, like Meta or Google are excellent at extracting all the value for that.

• 1:15:25Speaker 01

  You know, if there is more value to extract, if there's more margin to extract, they'll take it. Uh, cause that's what they're good at. And so I, I think that's another big reason, I think.

• 1:15:38Logan Bartlett?

  Interesting. So Duolingo today has approximately how many people? Uh, LinkedIn says seventeen hundred. Does that sound about right?

• 1:15:45Speaker 01

  No, that's a lot of, um, I don't know why that is. There's some contractors there. Who knows? Who knows what else randomness there is? Uh, we have about seven fifty employees.

• 1:15:56Logan Bartlett?

  Seven fifty, okay. Uh, I had heard you or Severin, your co-founder have been or were in every hiring meeting. Is that still the case?

• 1:16:07Speaker 01

  Both of us.

• 1:16:08Logan Bartlett?

  Both of you? All seven hundred and fifty people you've met with?

• 1:16:13Speaker 01

  Well, we've hired more than that because we've also lost some people.

• 1:16:15Logan Bartlett?

  Uh, yeah, I guess, I guess.

• 1:16:16Speaker 01

  Everything. And by the way, not only that, we've also turned down a lot of people. So yes, we've been in the hiring meetings of thousands of people.

• 1:16:24Logan Bartlett?

  How do, how do you make that work time wise? Uh, and how is that a principle?

• 1:16:28Speaker 01

  Every week, we just do it every week. There's a meeting every week that lasts about half an hour where they present us. Um, you know, the hiring manager comes and presents the case for the person that they're uh, getting the catch for send roles, you know, this goes super fast, particularly for more kind of junior roles. Sometimes these meetings take a long time and, and you know, we may go through a lot of, um, back and forth. But it, you know, it works. It's not like, I mean, again, it's not like the majority of the people that apply for Duolingo get rejected much earlier than that. The only ones that get to us are the ones that were everything looks pretty good until they get to us. So it's not that many per week. It's, um,

• 1:17:09Logan Bartlett?

  Got it. So it's not that you don't meet every employee before they join, but you sign off on it?

• 1:17:14Speaker 01

  Not, we used to meet every employee. That was, that, that was, that was a while ago. I mean, and that probably stopped five years ago. Um, at this point, it's, it's really just the case get to make to us.

• 1:17:26Logan Bartlett?

  Yeah.

• 1:17:26Speaker 01

  By the way, this is very important. Um, and it's because of this. Everybody else but us, and it's not because we're special people. I'll just tell you what it is. Everybody else but us has a true incentive to fill the role. The the person from the talent acquisition team just wants to fill the role because they're being paid to fill the role. The hiring manager has a hole in their organization, so they need to fill the role. So everybody just has an incentive. So what happens is they're just like, ah, this this person is good enough. And so they come in with an incentive to hire, whereas we have an incentive to hold the line. And by the way, yeah, again, it's not because we're such great individuals with such great. It's just a matter of incentives because we do the same thing because I've seen when Severin wants to hire somebody. He also has this problem. And when I want to hire somebody for my team, I also lower the bar.

• 1:18:11Speaker 01

  And so we try to, you know, it's it's simply, it's not because we are such great judges of talent or whatever, simply. I mean, it's it's a matter of incentives and that's why we we we.

• 1:18:23Logan Bartlett?

  It's interesting. So it's kind of human nature. If it's yours, then you see the pain of having to restart it. And so.

• 1:18:29Speaker 01

  That's it. And that's exactly right. I mean, when I'm hiring for my executive team, I'm just like, my God, I've been looking for six months. Look, this person is pretty good. They're pretty good. They're pretty good. And then if I show it to somebody who's way more impartial, they're like, really?

• 1:18:43Speaker 01

  Come on. And so that I think that's what it is. It's it's simply that.

• 1:18:47Logan Bartlett?

  Beyond just being a good person, what's a what's a non-negotiable for someone joining Duolingo? And maybe maybe to the good person example, if you could also tell the the car ride. Of what what you all do. If people hear this, maybe they'll know now to be nice to.

• 1:19:03Speaker 01

  Well, we do a bunch of things and you know, um, we do that. You know, for us, it's important that people are kind. Um, and one of the places where that shows up a lot is how you treat people who are

• 1:19:20Speaker 01

  lower in the organizational structure than you are. Yeah, everybody's pretty nice to me when they meet me and they want to work for Duolingo. But, um, you know, we definitely see how people, uh, treat. Um, some, and and this is usually for for, um, more senior roles. Um, where we look at, you know, how they treat much more junior people. And, you know, I mean, this is one, you know, we've I've told this story before where, um, you know, we were looking for a CFO for a long time. And, um.

• 1:19:53Speaker 01

  One person had everything look good, everything look good. But um they did not treat their driver well. And the driver is a person that has worked with us, you know, not, he's not a dual income employee, but it's the same. You know, we have a few drivers that we work with to send people to the airport, you know, to to pick them up. And, you know, we asked them, was this person good? And like, actually, no, they were pretty, pretty nasty to me. And we didn't hire them. And that was, that was painful for me not to hire them because I wanted to hire this person because I had been looking. This is an example, I had been looking for a long time. At that time, I probably had been looking for like nine months. And this was a very smart person, uh, very competent, but I guess they were not nice to our client.

• 1:20:37Speaker 01

  And and my sense is if that happens, you know, they're probably not going to be very nice to a level one, you know, finance associate or whatever.

• 1:20:45Logan Bartlett?

  Beyond that, beyond being kind to people, are there certain traits that you look for uh that are non-negotiables? Or is every role kind of specific beyond that?

• 1:20:55Speaker 01

  It depends a lot on the role. No, it's not the role, but more the function. So, you know, in the case of um uh design, you know, we really look at people's portfolio and we really care about polish. You know, some design organizations are okay that some of the designers don't have as much polish as others. For us, every designer that we hire, we want to have polish. So we look at their portfolio a lot. It just kind of depends on the function. Uh, but I would say company wide, you know, we want people to be kind and obviously really good at their craft. That that's just, you know, kind of, obviously. So there's one other thing that we do look for, it's it's it's which which we try.

• 1:21:37Speaker 01

  We really want people who are um in one way or another exceptional. And and we we look for that quite a bit. And exceptional can mean different things. I mean, in some cases, you know, exceptional is the usual kind of tech company definition of exceptional, which is like went to Stanford, got a four point O, um, et cetera, et cetera. And that's pretty exceptional. Honestly, very few people were able to get a four point O from Stanford. So that's pretty exceptional, but we look for a broader definition of exceptional. I mean, first first person in the family to go to college, it's pretty exceptional.

• 1:22:11Speaker 01

  Comes from a country, like from a very poor country, actually grew up in a very poor, poor country, and somehow managed to, you know, go to the university in France. That's pretty exceptional. So, stuff like that, you know, we try to make it so that everybody we hire has something exceptional.

• 1:22:30Logan Bartlett?

  What have you learned about, I guess, both leading and managing over the course of running Duolingo?

• 1:22:39Speaker 01

  Managing, definitely I've learned how to not become, how to not be a micromanager. I actually believe that if you're a small company, you know, under 20 employees, it really is actually a good thing for you to be a micromanager. Like, why not?

• 1:22:55Speaker 01

  After a certain number, you just can't be. And then after you start hiring very senior people, you piss them off. So I've learned not to be a micromanager. You know, there's a number of other things I've learned that ultimately, when there's a fuck-up, not to point fingers to anybody but yourself. And we actually have made that a pretty good part of the culture, especially in some functions where, you know, there's fuck-ups happen every day.

• 1:23:24Speaker 01

  And we've gotten pretty good at, you know, it's just like, okay, well, keep going. And as a leader, I think it's really important to be very, very ready to blame yourself. I think a lot of leaders, I've seen it with, you know, even with people who have, over the years, reported to me, where they're really trying not to get blamed for something, like, no, no, somebody else did it.

• 1:23:50Speaker 01

  I think people respect it a lot more when you say, look, ultimately, I'm responsible for it. And, you know, by that, I'm responsible for all the Duolingo fuck-ups. And we've had hundreds, maybe thousands.

• 1:24:05Logan Bartlett?

  It's probably safe to describe yourself as a accidental entrepreneur. Is that, is that fair?

• 1:24:12Speaker 01

  I did not grow up thinking I would be an entrepreneur.

• 1:24:15Logan Bartlett?

  And then captcha, then recaptcha, then Duolingo. I guess, when recaptcha sold, how many people did you have?

• 1:24:25Speaker 01

  Oh, not a lot. That doesn't.

• 1:24:27Logan Bartlett?

  Got it. What, what, what part of the entrepreneurial journey do you innately enjoy the most?

• 1:24:37Speaker 01

  I don't know if it's the entrepreneurial journey that I enjoy it more, and I enjoy doing things that have impact. That part I enjoy. So I enjoy, I'm enjoying a lot what I'm doing. I know with Duolingo, but it's because I'm still allowed to touch the product. On the day I'm no longer allowed to touch the product, I, my job satisfaction will low down quite significantly. And so I like that. I mean, I like tinkering and I like being able to have a lot of say in what, in, in, in what goes into the end product of a, of a consumer product.

• 1:25:10Logan Bartlett?

  I've heard you say that working out has been a life-changing thing for you. When did that start? And, and how, how has it actually changed your life?

• 1:25:22Speaker 01

  I mean, it started, I started about right around when I started Duolingo. Um, I just think it was a streak. I decided to have a workout streak. And um, I haven't, I haven't lost it actually. It's been, you know,

• 1:25:36Speaker 01

  however many years, thirteen give or take years.

• 1:25:39Logan Bartlett?

  Every, every day or every?

• 1:25:40Speaker 01

  Every day, every day, every day. I've not lost it. I did cheat a couple of times as follows at these trends, these, these flights to, to Japan. There's a problem with the date there.

• 1:25:51Logan Bartlett?

  Yeah.

• 1:25:52Speaker 01

  By working out twice the day before. And uh, you know, something like that. But, but yeah, I haven't lost my streak of working out every single day.

• 1:26:02Logan Bartlett?

  What's the most bare bone thing you've done to, uh, to, to make it work?

• 1:26:07Speaker 01

  Uh, yeah, the, the New York Times seven-minute workout.

• 1:26:10Logan Bartlett?

  Yeah, got it.

• 1:26:12Speaker 01

  Look, because, because you can always say you can go running, but sometimes it's like horrible weather. You're in some random hotel that has no gym or whatever, and all you got is you. And yeah, you do that seven-minute workout a few times and that's all you do.

• 1:26:29Logan Bartlett?

  How has it changed you? Is, is it, is it the energy from it? Is it the?

• 1:26:34Speaker 01

  It's the energy. You know, I used to get sick quite often, which is funny, which I know I am sick, but I, I get sick once every like two years. Um, I used to get sick very often before that. That's one thing. I think the other thing is it really allows me to regulate my own emotions quite a bit. But I think I, you know, I'm generally a Pretty emotional guy, but I think that um my wife says I'm Latin American. Latin American men. But I think uh um you know, it allows me to regulate my emotions, I think.

• 1:27:08Logan Bartlett?

  Someone that uh maybe struggles with that or or wants to get going, is there is there any advice you would have to them on like how to just start making progress? Does it just get a streak going uh in a similar way?

• 1:27:21Speaker 01

  Yeah, yeah, yeah, yeah, yeah. And by the way, what that's what matters the most, just get a streak going with anything. I mean, this is this, by the way, is the same with Duolingo. Look, a lot of times people are like, no, no, no, there's a there's a better method for learning a language. And it's like, no, ultimately, that's like saying the treadmill is better than the elliptical. It probably is, but you know what matters 95% is that you do something. That's, you know, did you do it with a treadmill, with an elliptical, or one of those weird climbing machines or whatever? Sure, probably there are some differences. But between not having done anything and having done even the dumbest one of these, it's better to do the dumbest one of these. I mean, and that's the thing with Duolingo, we are optimized for keeping you going.

• 1:28:00Logan Bartlett?

  Is there is there another habit or anything you've picked up on maximizing productivity and balancing your your, you know, work-life balance and all of that stuff?

• 1:28:10Speaker 01

  I mean, I'm pretty efficient with most everything I do, and I think that's, you know, a lot of people comment on that. I'm I do most everything very fast. I eat very fast, so I I don't know if it's a habit that I think for some reason, I'm just very fast at most everything I do.

• 1:28:23Logan Bartlett?

  I'm the same way eating, by the way, I it's always been a problem.

• 1:28:27Speaker 01

  People think that I've been to jail.

• 1:28:29Logan Bartlett?

  Yeah.

• 1:28:29Speaker 01

  People think that I've been to jail because I like eat so fast. I eat so fast. But the truth is, I you know, I love food, but I I don't savor it in my mouth for five hours.

• 1:28:39Logan Bartlett?

  Yeah, that's funny.

• 1:28:42Logan Bartlett?

  Um, one of the uh, one of the things I'd be curious about is um. Recapture was sold in, what was it, 2008, nine, 10?

• 1:28:52Speaker 01

  Nine.

• 1:28:53Logan Bartlett?

  Nine, okay. Uh, and at that point in time, I I don't think it's ever been disclosed, but you didn't raise any outside money and sold for I I've heard you said tens of millions of dollars. Uh, and so. And now Duolingo is an enormous success. I'm sure we can look up in the S1 how you know, your ownership at time of IPO, but for safe to say you you you've made more money than you can probably spend between both of of these things, unless you want to go buy a sports team or something. But I assume I don't know if that's in the cards for you.

• 1:29:26Speaker 01

  But Depending on the sports team, depending on the sports team.

• 1:29:30Logan Bartlett?

  Yeah. Well, in that case, then then there's a lot of stuff. There's a lot more that you can keep going to buy the sports team. What would be the sports team of choice if you could?

• 1:29:40Speaker 01

  I mean, I don't know. I like soccer.

• 1:29:43Logan Bartlett?

  Yeah, okay.

• 1:29:44Speaker 01

  I'm not that crazy. I mean, I like soccer, but I'm not.

• 1:29:47Logan Bartlett?

  You're yeah, it's not gonna be the Steelers or the yeah, one of the Pittsburgh teams. Okay.

• 1:29:51Speaker 01

  Well, the Pirates would be cheap, but.

• 1:29:53Logan Bartlett?

  Yeah, that's right. That's right. How um, how has that most changed your life for how you view the the the world having that level of of success? It seems like you you still live in Pittsburgh, presumably a similar ish life. But is there any way that it's changed to the positive versus the negative on stuff?

• 1:30:16Speaker 01

  Oh, yeah. I mean, you don't worry about some of the stuff that you used to worry. I mean, I remember when I was a graduate student, you worry about like, are you going to have enough money to pay the rent and stuff like that? You know, those those worries entirely go away. And that I think, you know, that's, that's quite a nice thing. I mean, I just don't. And not only that, the other thing is just not having to spend any mental energy on like, you know, to the this toothpaste cost more or less than that other toothpaste, etc. I just, just, I, you know, I basically learn, just don't think about money unless, again, unless you're going to go doing some crazy thing like buy a sports team, kind of just doesn't matter. And, you know, at least I don't spend the mental energy on that. Now, for me, I'm, you know, it's not like I go around spending crazy amounts of

• 1:31:00Speaker 01

  money. So it hasn't changed that much how I live my life. I mean, I have, I have like three pairs of jeans. That's it. It's kind of what I have.

• 1:31:13Speaker 01

  Um, yeah, it hasn't changed that much. I mean, I did, you know, I, right now, I'm in New York. I'm in New York and, you know, I did buy a pretty expensive house here, but other than that, I haven't done all that much.

• 1:31:24Logan Bartlett?

  I, I've heard you say, uh, something about like to truly understand something is to be able to explain it simply or, or some philosophy like that. Can you espouse or expound on that a little bit, uh, for, for why?

• 1:31:39Speaker 01

  Yeah, that's from my PhD advisor, Manuel Blum. I mean, it is an amazing thing. And it's hard to explain unless you've actually felt it.

• 1:31:51Speaker 01

  Once you actually understand something, you can give an amazing, clear, simple explanation of it. Um, and I used to think about that. I just didn't know that. Um, and you know, I learned that.

• 1:32:06Speaker 01

  From my advisor and it was as follows, it was early on in my PhD, you know, I had to explain a scientific paper to him. And he said, go read this paper, explain it to me, because I haven't read it. I would come, you know, I came in for a one hour meeting and I said like the first statement. And he said, I don't understand. And I'm like, how can you understand? It's just a simple, a simple statement, but I could, I just don't get it. Why did you say that? Why did you start with that? Why? I don't get it. And I spent the whole hour and I couldn't even go after, past the first line of the paper. Um, I came back the next week and it was the same thing. It took me six months to finally get to a point where he said, okay, I understand.

• 1:32:49Speaker 01

  Meeting him weekly. It turned out he was, he was pretending the whole time, actually. That paper, he was a real expert in. Um, but it took me really six months and it was true. I mean, the explanation that I had at the end of the six months was just because I really understood it. It was just crystal clear and I could explain something that at first I was planning on taking like 40 minutes to explain it to him. I was able to explain the idea in like a minute. There's like, no, this is the main idea of the paper, blah. And, um, yeah, but that only happens once you really understand something and that's, I mean, that's the genius of my PhD advisor. He's, he, he really understands things well.

• 1:33:27Logan Bartlett?

  I think today, there's more people learning languages in the US through Duolingo than the education system combined. Do I have that fact right?

• 1:33:37Speaker 01

  Correct. There are more people learning languages on Duolingo in the US than there are people learning languages across all US high schools combined.

• 1:33:43Logan Bartlett?

  But I heard you say recently that you're not anti the US education system, and in fact, maybe pro of it. Having learned so much about how people learn and built a business around it, at least for languages and now increasingly music and math, if you could wave a magic wand, is there anything from a teaching standpoint or how we go about the educational system in the US that you would change if there was one thing that stuck out?

• 1:34:21Speaker 01

  It's very different from what we do at Duolingo. But I mean, what matters the most in the educational system are the teachers. This is by far what matters the most. And there are some excellent teachers. But there's also quite a high variance. And, you know, what I would wave the most is, or what I would want to is make sure that every teacher that is out there is amazing. But it's pretty hard to do, given a system that doesn't pay teachers very well, given a system where, you know, there's essentially the equivalent of tenure in some of these places where, you know, you basically can work and not do all that much because of teacher unions. So there's a lot of stuff that is working

• 1:35:03Speaker 01

  against that. But the thing, you know, you want better education, better teachers. That's, you know, again, very different from Duolingo. Duolingo, we don't use humans. But the truth of the matter is that for the foreseeable future, humans are going to be needed for, you know, the educational system.

• 1:35:19Logan Bartlett?

  Can you tell the story of Bill Gates? So Bill Gates called you, wants to try to personally recruit you to join Microsoft. Can you give that backstory? I picked it up in something I was researching, and I was like, well, I have to get that story or a little bit more detail into it. Clearly, it didn't work. You never went to go work at Microsoft, right?

• 1:35:36Speaker 01

  Yeah. And so when I graduated from my PhD, I, there's this one thing that happens when you, kind of the winner takes all kind of thing. I was the kind of most sought after candidate after I graduated from my PhD. Every university wanted me to become a professor there. And usually when you graduate from PhD in computer science, you go become a professor in computer science at our university. But there's another path, which is you could also join Microsoft Research. And I was the most sought after candidate, you know, every university, every computer science department wanted to hire me. Microsoft Research wanted to hire me too. And Microsoft Research, I guess, at the time, I don't know, they had a thing where the person they wanted

• 1:36:18Speaker 01

  to hire the most that year, they would get Bill Gates to call you and personally pitch you for as long as you wanted to. So I ended up talking to him for like an hour, mainly not because I had, you know, I love a lot of the stuff that they've done at Microsoft, and certainly Microsoft Research, but I wanted to become a professor, so I had no, no desire, but I wanted to talk to Bill Gates for an hour. So I was happy to do that.

• 1:36:43Logan Bartlett?

  Yeah, well, that's good. Listen, if Bill Gates ever calls me, I will gladly take his call and I'll spend an hour talking to him.

• 1:36:51Speaker 01

  He's such a smart guy. He's crazy. When you enter a room and you're like, he's clearly smarter than me. Yeah, you think that with him. Because I've been in a room multiple times with him for different things, different things. Every time you're like, yeah, that guy's smarter than me.

• 1:37:03Logan Bartlett?

  Do you remember what you talked to him about or what questions you asked him?

• 1:37:07Speaker 01

  At that time, no, I don't, too much. But I've talked to him about a number of things. I mean, he's, you know, he's a, we talked about Duolingo and stuff a number of times. Again, he's such a smart guy.

• 1:37:16Logan Bartlett?

  Yeah. Now, last one, I'll let you hop. What's the, what's the story of you setting up a honeypot website to catch people that were cheating on your, on your tests when you were a professor?

• 1:37:29Speaker 01

  I was a professor. Yes, I was a professor of computer science. And it wasn't tests. I, you know, we were, I was assigning, you know, math problems. It was basically discrete math, but it was math problems. And we had this rule, which was a dumb rule, but it was a rule that I inherited from the guy who taught the class before me, that was you can't use, you can't Google for solutions. And so we tried every time that we never, because there's only so many really awesome math problems. And you know, we tried to disguise them a little bit, but you know, we thought, well, probably people are Googling for solutions. And then it occurred to me, well, I'll teach them not to do that. And so I made up a problem, I specifically made up a problem.

• 1:38:13Speaker 01

  And then I made a website for that problem. That this is a fully unique problem in a website that had the problem, etc. And I own the website. And what happened is I assigned the problem and I said, of course, as usual, you can't Google or anything. But if you Googled that problem, you got to my website and I could track who you were. On the website and so um, turned out thirty percent of the class, this was a class with three hundred kids, so about a hundred of them. Fell on the little website. And um, I found them all and then um, the next lecture. You know, I didn't say anything, I didn't contact them the next lecture, I just went in, I stood in front of the class, I.

• 1:38:56Speaker 01

  Projected my screen, I typed, you know, the query to get to that problem. I got to the website and everybody's just like, and then I said, I did a, you know, to to know who owns a website, you just do who is. And the name of the website, I typed in in the prompt there, who is the name of that website? And then it said that it's owned by me. And then you see everybody's jaw just drop and then I'm then I then I then I, you know, I'm using this is using Linux and I'm showing them. You know, the the server logs and just everybody's IP address and I'm like, woo. And everybody was just, they were um, they were peeing the pants. Um, and so.

• 1:39:37Logan Bartlett?

  What'd you do with it? Did you, did they confess or or what happened?

• 1:39:42Speaker 01

  I mean, I knew who they were, um, so I just gave them a zero all of them in that assignment. I said, look, I'm not going to ruin your life. I'm just going to give you a zero in that assignment. You can still get a name in the class, but. I was not interested in ruining anybody's life. I just wanted to have them not do that anymore.

• 1:39:56Logan Bartlett?

  Yeah, that's a good lesson. Well, thank you for doing this. This is fun. Hopefully you feel better powering through here. Yeah, so thanks for doing this. It's a pleasure. And yeah, this is great.

• 1:40:08Speaker 01

  Thank you, same. Blessings mine. Thank you.