ACQ2: Why Duolingo Worked (with Luis von Ahn, CEO)

(Episode originally released Nov 10, 2024) Duolingo has fundamentally changed the landscape of self-guided education, starting with language learning. It is now a $9B publicly traded company in a space where everyone thought you could never build a large and exciting company. We’re joined by Duolingo founder and CEO, Luis von Ahn. Luis dives into how learning English was transformative in his personal trajectory and opportunities in his life, inspiring him to create the most successful EdTech product of all time. *More Acquired:* - Get email updates https://www.acquired.fm/email and vote on future episodes! - Join the Slack http://acquired.fm/slack - Check out the latest swag in the ACQ Merch Store https://www.acquired.fm/store! _Note: Acquired hosts and guests may hold assets discussed in this episode. This podcast is not investment advice, and is intended for informational and entertainment purposes only. You should do your own research and make your own independent decisions when considering any financial transactions._

• 0:00Speaker 1

  Hello, acquired listeners. We have an excellent ACQ2 episode for you today. We are here with Luis Von Ahn, the co-founder and CEO of Duolingo. Luis was a math professor at Carnegie Mellon in Pittsburgh from 2005 to 2012. He previously founded Recapcha, which I'm sure many of you have filled out hundreds or thousands of times that sold to Google in 2009. He is Guatemalan and immigrated to the United States studying mathematics at Duke and later earning his PhD in computer science from Carnegie Mellon. Welcome to the show, Luis.

• 0:35Speaker 0

  Thank you. Thank you for having me.

• 0:36Speaker 1

  We're very excited to do this together. I think Duolingo is a completely fascinating company. And rather than me saying Duolingo is a why don't we ask you, what is Duolingo?

• 0:47Speaker 0

  Well, we're most well known for an app that teaches languages. It is the most popular way to learn languages in the world. It is the most popular education app in the world. We have about a hundred million monthly active users. We're also known for unhinged yet wholesome green owl that does weird stuff on TikTok. It's the mascot of the company. You know, that's basically it. We're doing what we can to teach people languages. It started because we wanted to teach people English. It turns out English is transformational. In the US, I guess too, if you don't know English in the US, you really should learn English. But in non-English speaking countries, English is pretty transformational. People who learn English can just make a lot more money. So we started with this very mission-driven idea that we could teach people English all over the world. But if we were going to teach English, may as well teach other languages.

• 1:34Speaker 0

  We started teaching other languages. And at this point, we're the most popular way to learn languages in the world.

• 1:41Speaker 1

  It's amazing. So my first question that I've been listening to you on other podcasts and doing research on the company and preparing for this. Why has Duolingo worked to such an extreme scale? You're a nine billion dollar publicly traded company. Why has it worked when so many companies in the language category in the past have been these small fragmented sort of sub-scale companies?

• 2:08Speaker 0

  There's a number of reasons, but if you ask me for one single reason, is that we understood something early on. The insight is the hardest thing about learning something by yourself is staying motivated by a margin. And so what we've done with Duolingo is we really have made it so that you want to learn. And I think that's what has made us very popular. I mean, most education apps that try to teach you something concentrate mostly on the learning outcomes. They try to actually teach you the thing. And that makes sense. However, that's not the hard part. The reality is you can learn anything with a book. You want to learn quantum physics, you can do so with a book. It's just nobody's doing it because it requires reading a quantum physics book. But if we can turn it into something that people actually want to do, that can be really powerful. And I think that's the biggest insight. I mean, there's other things that we've done.

• 2:55Speaker 0

  We became really good at product, we have really good design. There's a lot of things, but ultimately it's just that we understand that we need to get people to want to do it.

• 3:04Speaker 1

  All right. So what are some of your key levers in making people want to engage?

• 3:09Speaker 0

  When we started Duolingo, which was like in 2011 or so, we started working on it. We were building a program that taught you languages. And it was with my co-founder. My co-founder is a native Swiss German speaker, and I'm a native Spanish speaker. And what we decided we were going to do is I was going to make the first Spanish course. Even though I don't know how to teach Spanish, but I do know Spanish. And he was going to make the first German course. That was really stupid. I thought German and Swiss German were the same language. Turns out they're not. I thought they were like British English and American English. They're not. It's like a different language. Anyways, he does happen to also know German, like actual hoch German. So, anyways, he made the German course. I made the Spanish course. The goal was that we were going to learn each other's language. But we ran into this problem that we would go home and then the next day we would come to the office at 9 a.m.

• 3:58Speaker 0

  and I would ask him, you know, did you do your Spanish lesson? And he would say, No, man, so boring. And I had the same problem. It's because when we started, we were making these lessons that were like 30 minutes long,

• 4:09Speaker 2

  first of

• 4:09Speaker 0

  all. And there was no gamification, et cetera. And we couldn't even get ourselves to do it. So this is when we started really trying to add all the mechanics to get people to do it. Aaron Powell

• 4:18Speaker 2

  Was this in any way connected to your research or was this a complete side project?

• 4:22Speaker 0

  It was. Well, it kind of was. He was my PhD student, that was a professor, and we were trying to find a PhD thesis topic for him. That's what we were doing. We didn't even know this was going to be a company. I mean, we kind of in the back of our mind thought maybe this could turn into a company. We weren't super sure. That was the idea. And it was kind of connected to my research, but ultimately was just trying to teach you something with a computer. Aaron Powell

• 4:41Speaker 1

  Your research was around crowdsourcing, right? Aaron Powell

• 4:43Speaker 0

  My research was around crowdsourcing, yes. And this was just more about trying to teach you something with a computer. So we were doing that. But this is when we learned that we needed to make it more fun. And so we started doing a number of things. I mean, the first thing is 30-minute lessons, no good. You want to do three-minute lessons. So that's the first thing we did. We made in three minutes. That's very important. And even if ultimately people do spend 30 minutes on your app, making each chunk short matters a lot because that means you can do them while you're waiting for the bus or while you're literally going to the bathroom. 30 minutes just too long. So you have to be able to chunk them. So we started chunking them. That's important. So we made the lesson experience be something that was quite palatable. Three minutes, there's a progress bar that you get to fill, and then we started adding all kinds of little dopamine hits where like when you get something right, there's like a little explosion.

• 5:32Speaker 0

  That's kind of one thing. Then over time, we discovered a number of other really important levers. One super important lever has been the streak. So we have a streak. We've had a streak for like 12 years, basically. And the idea is you know the standard with a streak. If you use it for seven days in a row, you have a streak of length seven. And if you don't use it on the eighth day, it goes down to zero. So it's just a counter of how many days you've done it in a row. This is super powerful. At this point, we have I think I'm gonna get this number right. I think we have eight million people who are daily active users who have a streak longer than 365. So we have 8 million daily active users that haven't missed a day in the last year. Or more. So more than 365. So it's very powerful. Streak is very powerful.

• 6:12Speaker 1

  So that's around 8% of your users.

• 6:15Speaker 0

  8% of our active users have not missed the day in at least a year. That's very powerful. It really is. Another thing that has been really powerful is notifications. You know, many apps abuse notifications where they just notify you like crazy. It turns out that if you're teaching something to somebody, they're a lot more okay with you sending them notifications because people think, well, this is good for me. So we've taken advantage of that. While we're not spammy, we do send notifications probably at a higher rate than we could if we were not teaching you something. We have a very sophisticated system that tries to figure out it's an AI system that we've trained over the years that basically tries to figure out when to send you a notification and what to say in it to get you to come back that has been really successful. And then the other thing that has been pretty successful that we've gotten just a lot better over the last few years is the social component where we have leaderboards, leaderboards, by the way.

• 7:04Speaker 0

  There's a whole science of how to do them well. We got them very wrong at the beginning, but we're much better now. So you know, we have leaderboards, you can have streaks with friends now, you can have quests with friends, et cetera. So that's helped out a lot. Aaron Powell

• 7:18Speaker 1

  And we should say this notion of gamification. I mean, this was like a real hot word 20 call it 12 to 15. Yeah. And you started the company in 2011. So you were kind of blazing the trail on a lot of these things. The the idea that you could bring game mechanics to non-game experiences. It's pretty interesting reflecting back on that time, how that was the buzzword in every VC pitch that we were seeing that you really did build a public company on that concept.

• 7:49Speaker 0

  Yeah, for some stuff, gamification is not all that useful. Particularly things that you anyways have to do. So, for example, a smaller part of our business for Duolingo, we have this thing called a Duolingo English test, which is about 10% of our business. It's a standardized English test. Like TOEFL. It's like TOEFL. It's a competitor to TOFO. It's a test that people have to take to get into a university, for example. That thing, you can gamify it, you can not gamify it. It doesn't matter. The reality is people have to take it and they'll go through it. For something like education, I think gamification ends up really mattering a lot because it's all about motivation, actually keeping you there. So I think this was just a very good use of gamification in the end. Aaron Powell

• 8:27Speaker 1

  Do you have a sense of if you've expanded the market for people learning a language who wouldn't have sought to learn one otherwise? Aaron Powell

• 8:35Speaker 0

  Massively. It depends on the country. But in the United States, for example, 80% of our users were not learning a language before Duolingo.

• 8:41Speaker 2

  Your typical American, like was

• 8:44Speaker 0

  not learning a language. Yes. Yeah, but they are now with Duolingo. And it's a similar in the UK. It's a similar number in the UK. There are other countries, particularly countries where people are learning English. There, we probably haven't expanded the market as much. But in English speaking countries like the US, UK, Australia, we've expanded the market by a lot.

• 9:03Speaker 1

  Did you encounter when you were fundraising a lot of pushback that hey, this market isn't that big? I imagine it's probably hard to sell the thesis. Our product is so good that we will massively expand the market.

• 9:14Speaker 0

  Yeah, we did. Our first fundraise was 2011, Union Square Ventures. At the time, Union Square Ventures was very hot. They had just funded Twitter. They had just funded, it was like a big deal. Anyways, we went there and we were so fortunate that they funded us. Brad Burnham was our partner there. He got into our board. We became good friends, et cetera. Many years later, he told me, ah, we didn't think you were going to succeed at that. We just liked you. And uh we thought you were going to pivot into something else, and maybe you'll do something, but we actually didn't think you were going to succeed because we didn't have a product. So that was the first round. That was a series A. By the time we reached Series B and we raised it, kind of the resource book for themselves. We weren't making any money, but we had a lot of users, especially if we had only launched like maybe two years before that or something. We really already had millions of users.

• 10:00Speaker 0

  So it became a lot easier when we had, I don't know how much product market fit was, but at least we had some amount of product market fit. Aaron Powell

• 10:08Speaker 2

  I'm curious what your thought process and discussions were as you were starting the company, but even before then and you're looking for a PhD thesis. Were you always planning to be mobile first, mobile native? Because it was right around that generation when it switched from

• 10:23Speaker 0

  like You're exactly right. In 2011, almost no company was doing mobile first. Everybody was like website first because most of the traffic was on the web, like a big laptop or like even a desktop. When we started, we thought this was going to be a website, but relatively early on, really, maybe in the first few months, we realized almost everything is going mobile. And we had this insight, which was we're very fortunate that we had this insight that at the time most companies, they were website first and they had a companion app. Like banks. You could go to the main website, but in the companion app, you could check your balance, but you couldn't do anything else or whatever. It's just like a companion app. We had this insight that you know we could actually put all the functionality of the website in the app. And so it was not a companion app or anything.

• 11:08Speaker 0

  And as soon as we launched, we first launched the iPhone app, it overtook the website traffic within weeks. We had more traffic on our iPhone app than on the web app. And so right then is when we decided we were going to be mobile first. And at this point, my God, I don't know, the it's 95% of our traffic is on mobile and 5% is on web, probably.

• 11:26Speaker 2

  Were y'all watching Instagram as this was happening? Yes.

• 11:30Speaker 0

  Because like the the analogies are so close here. We were, and by the way, we were very fortunate with Instagram. And I also think we gained a lot from this because you know, we were not the first language learning app. There were others. But I think we were the first good one. And it's because before us, what people were doing with apps is they were basically making websites and making them tiny. It's like a tiny little website, but they just kind of shrunk everything into a mobile screen. And when we came out with a mobile app, by the time we came out, we were watching things like Instagram. There was another app called Path that was very easy to design.

• 12:05Speaker 2

  Yes, Dave Moran.

• 12:06Speaker 0

  So we were watching all these apps, and we were like, this is how you design an app, not make a website and shrink it. You actually have to change all these things. So I think we benefited a lot from the thinking of let's make an actual native mobile thing as opposed to uh you know a miniature version of your website. We were very fortunate with that. Aaron Powell

• 12:25Speaker 1

  I'm obsessed with this misperception of market size question. I'm sure you get pitched a lot of ideas now. Many people probably come to you with the idea, hey, just like Duolingo, this doesn't seem like a big market, but actually we're gonna build a big business in it. What's a good heuristic for when that is possible versus not? Aaron Powell

• 12:45Speaker 0

  I don't know what a good heuristic really is, but I can tell you this. I think language learning was massively underappreciated in the United States because in the US, people don't have to learn a language. The reality is if you already speak English, English is the monster language in the world. That's the big one. I grew up in a country where you had to learn English if you wanted to be mildly successful. This is true in my case. Had I not learned English, I wouldn't have come to the US and I would have definitely not succeeded the way that I did. And my co-founder was also not born in an English speaking country. And I think when you grow up in a country that is not English speaking country, I think you realize, yeah, you may think that the market is not huge, but pretty much everybody is either learning a language, in particular English, or wants to learn English, and everybody pretty much understands.

• 13:32Speaker 0

  So at some point you just realize, well, boy, billions of people want to learn English. So it just has to be a big market. There has to be a big need. So I think that was the case for language learning. You know, I don't know for the other ones. What we do inside Duolingo when we're looking at opportunities is we definitely look at just how many people are either doing something every day or close to every day. So this is why we decided when we're trying to expand to other subjects, like we're doing math and music. We went for those because those are things that literally billions of people are either doing or want to do. And we didn't go for something like, you know, you could go for chemistry or something. Love chemistry, it's just there's probably not a billion people learning chemistry or wanting to learn chemistry.

• 14:16Speaker 1

  What's your early signal on Duolingo for X? You know, you mentioned math, there's music. Did you start with the very best use case for the type of product you build, or do you feel like, oh, actually, that was a local maximum, there may be even better products that are more well suited. Aaron Powell

• 14:32Speaker 0

  There's a lot of companies out there. I mean, I get the ads all the time on like my Instagram or something, Twitter or whatever that says you know, the Duolingo for whatever. But there's a curious one, the Duolingo for anger. And I don't know if they mean that you like it teaches how to get angry or not. I don't, but anyways, I get a lot of ads that said the Duolingo for anger. Anyways, my sense is that there are probably other things that are pretty good, but it's hard to compete with language learning. I think math and music are going to be big. I don't know if they're going to be as big as language learning, but they're going to be big. But the reason I think it's hard to compete with language learning is that not only are there a lot of people wanting to learn English in particular, but also language learning has a few things that are good. One is that it's one of the few things that people want to learn both in school and outside of school.

• 15:17Speaker 0

  So most things people just kind of have to learn like in school. Math is an example. Not many people want to learn math outside of school. But language learning, if you ask the average person walking on the street, yo, would you want to get better at Spanish? Most people are like, yeah, why not? So I think that's a big thing. Another big thing that happens, this is uh in contrast with music. Music has one big problem, which is you need an instrument. And that's a big deal. I mean, unfortunately, an instrument is two, three hundred dollars with language learning, you don't need anything. It's just that. There's a lot of things that I think work really well for language learning. You know, I don't know if it's the best use case, but it's a pretty good one.

• 15:52Speaker 1

  Yeah. So thinking about the language learning business, you mentioned earlier English is the number one language that people want to learn. So most of your usage, please correct me if I'm wrong, is outside the US, people wanting to learn English. But most of your monetization happens in the US. I was wondering if you could just tell us a little bit about sort of how that evolved and how you think about it.

• 16:17Speaker 0

  Yeah, if you look at the numbers, about 45% of our active users are learning English. English alone accounts for 45% of our users. And that's mostly outside of the US. In terms of users in the US, it's about about 20% of our users, half active users are in the US. The other 80% are outside. In terms of revenue, about half comes from the US. And the reason for that is because you know the US is pretty large and wealthy country. But the other thing is that's important to understand is that people in the US are very accustomed to paying for digital subscriptions. What we're finding is that in certain countries, the US is ahead of most everybody. In certain countries, people are accustomed to digital subscriptions. In others, people are not, or they're really against them. And these are countries where even a Spotify or a Netflix are just not as penetrated as they would be in a place like the US.

• 17:06Speaker 0

  So I think that's a big thing. We just have a lot more propensity to pay in the US and wealthier countries. The other thing that happens with us is, you know, we have a really good free version. In fact, if you look at our monthly active users, only about 9% pay. The other 91% do not pay us. And it's because our free version is so good. And what we're finding is that in poorer countries, even if you bring the price down a lot, people have this mentality that is that they won't pay unless they have to. So even if it's like 50 cents, like super cheap, unless they have to, they won't pay. Whereas in the US, people are a lot more willing to pay for convenience for things like, well, you know what, I don't want to watch ads, so whatever, I'll pay. We're not seeing that type of behavior in a country like Vietnam or Brazil or something like that.

• 17:52Speaker 0

  I mean, some people pay in those countries, but it's just it's much less than the US. Aaron Powell

• 17:58Speaker 1

  And you do have an ad supported tier, right?

• 18:00Speaker 0

  Yeah, we this is the free tier, it has ads at the end of a lesson. But the ad load is pretty light. I mean, basically you get one skippable ad at the end of a lesson. So five seconds of ad after about three to four minutes of thing. It's pretty light.

• 18:14Speaker 1

  Yeah. And the other thing that I think is important as listeners, sort of understand this part of your business. When did your product launch and when did you charge the first dollar of revenue?

• 18:23Speaker 0

  Oh yeah. We launched in 2012. But when we launched, we made no money. We didn't have ads and you couldn't pay us. I mean, simply we made no money because there's just no way for us to make money. We we just didn't put it there. And that was true up until about 2017. We started thinking about monetizing in 2016. What happened was we had just raised funding from at the time it was called Google Capital, but it's now called Capital G. We had just raised funding from them, and I don't remember the valuation. It's about a half a billion dollars valuation, but with no revenue whatsoever. And the partner from there, Layla Sturdy flew to Pittsburgh where we were, and you know, she got us drunk and she said, you gotta start making money. Like tomorrow.

• 19:08Speaker 2

  And of course, as a founder, you clearly knew the secret that the only thing better than making a lot of money is making no money. Everything in between is not good.

• 19:18Speaker 0

  Yeah, exactly. No, that is the secret. And so uh went to the office the next day, and it was pretty interesting. Because for the first several years, we made no money, and we weren't even really trying to make money. And everybody that we were hiring was very mission-driven. We we were hiring all these people that were that loved the idea of giving free education. So when I came to the office and I told everybody, hey, we have to figure out how to make money. People were pretty shocked, and they were like, but why? But I said, well, kind of have to pay your salaries. And they said, You've been paying our salaries this whole time. Yeah, right.

• 19:52Speaker 2

  Hasn't been a problem so far.

• 19:54Speaker 0

  Yeah, why can't you just continue doing whatever the hell it is you've been doing? And you know, I had to explain you can't raise funding for

• 20:00Speaker 2

  Here we're going to do a seminar. We'll do a Duolingo session on late stage capitalism. And you know, like

• 20:05Speaker 0

  you know what it was crazy. It took about six months to convince the employees of the company that it was in fact not evil to make money. Eventually people got convinced. And I think the main way in which people got convinced is I think everybody understood that. If we made money, we could invest more in the mission. And I think that everybody was like, oh, okay, okay. So you mean we can still put most people to work on actually making a better product as opposed to making money. And I said, sure, sure, that's what we're gonna do. Today's day it remains true. I mean, the fraction of people that are working on making money at the company is it's relatively small compared to the rest. It took about six months to convince people that we should do that. The other thing that was complex is at the time we weren't sure how to make money in a way that respected the fact that we wanted to still to be the case that most people could learn for free.

• 20:51Speaker 0

  You know, at the time Spotify was kind of growing, et cetera, but it was not super clear that this freemium model worked. This freemium model, which is like you can use it for free, but you may have to see some ads, and if you don't want to see ads, you have to pay. It wasn't clear that we could make that work. It turns out you can make it work. It's great. It worked really well. And the fact that that worked was really good for us because it's very mission aligned. Basically, you can learn entirely for free. You can learn without ever having to pay us, but if you want to turn off the ads, you can pay us. That's very mission aligned for us. But we're fortunate that that worked. It really was not clear back then.

• 21:26Speaker 2

  It is so funny. That's like just such a basic assumption now. But even 2017, 2018, YouTube, you couldn't pay them. You could only watch ads. Netflix, you could only pay them. You couldn't watch ads.

• 21:39Speaker 1

  Uh actually, David, I think Google contributor did exist.

• 21:43Speaker 2

  I don't even know what that was.

• 21:45Speaker 1

  It was the predecessor to YouTube premium, and it worked across all their properties where you could basically pay the amount that Google would make on you from all the ads that you would see

• 21:56Speaker 2

  across the code. Wait, did it strip out ad words? There's no way we can.

• 22:00Speaker 1

  I think it might have been except for ad words. I don't know. I should get my facts straight on this. I was a paying Google contributor customer.

• 22:06Speaker 2

  I had no.

• 22:07Speaker 1

  I mean, it says a lot that it was called contributor.

• 22:10Speaker 0

  That's funny. Well, in any case, for us, it was not super clear that this was going to work. But it worked. Aaron Powell

• 22:16Speaker 1

  Was it strategic at all? Was there a network effect component? Like, is it good to not put any inhibitors on your growth rate because more users begets more users or something like that? Aaron Powell

• 22:29Speaker 0

  I am a huge believer that the reason that we have been able to grow so much is because we didn't monetize

• 22:35Speaker 2

  early

• 22:36Speaker 0

  on. Now I cannot recommend that to every single company. I mean, for some companies you probably should monetize early on. But in our particular case, what happened was this. We launched. Because we weren't making any money, we also couldn't spend money on marketing. Like we couldn't do performance marketing because performance marketing is this thing where you like you can pay Google 30 bucks and then they send you a user. But to me, that seemed really dumb to spend 30 bucks to get a user when we were not going to make any money off of that user. You

• 23:07Speaker 2

  want to be making more than 30 dollars from that user. Aaron Powell

• 23:09Speaker 0

  Yes, or whatever it is. You want to just do more than that. And so we couldn't spend money on marketing. We weren't making any money. And we had a bunch of engineers, so what were we going to work on? And the only thing we decided to work on was teaching better and increasing user retention. That's it, because that's the only things we could work on. So we spent from 2012 to 2017, basically optimizing retention rate for users. And we optimized it a lot. I mean, for example, we went day one retention, so that's the retention of people when they show up, what fraction of them come back the second day or the next day. When we first launched that number was 13%. Aaron Powell

• 23:53Speaker 1

  As in you lose 87% of your users the day that they're

• 23:58Speaker 0

  well, they may come back another day. 87% happen to not come back the day right after. It could be that they come back later, but that's what we saw. We ran who knows how many A B tests over this period of time, but by 2017, that number was I don't know, probably about 50%. And maybe misremembering exactly the numbers. But we were able to drive all of our retention numbers by a lot. Because that's the only thing we could work on. Had we not done that, we probably would have spent all our time optimizing our ad pipeline and optimizing all kinds of weird stuff that ends up making you money, but I don't think helps you grow that much. And I think we were in retrospect, this was a very smart move. I'm at the time I don't think we were thinking how smart this was. We just kind of did it.

• 24:39Speaker 1

  It gave you a durable asset. I mean, the fact that you want to have a product that that ensures that you have fully maximized the potential for a user to stick around. It means the user base you've built and the machine by which you eventually do make money is far more durable. Aaron Powell

• 24:55Speaker 0

  That's right. And I believe that. Now, of course, I believe all of this in retrospect. At the time, none of this, you know, we were just like, I don't know, make a thing that people like. It

• 25:03Speaker 2

  was also working. You were raising money at a half a billion dollar valuation. So

• 25:07Speaker 0

  this was when money was free in some sense. It's just like here, have more money. But it it worked. Aaron Powell

• 25:13Speaker 2

  Okay. So you joke that you weren't doing any marketing, and then in some sense that's true. In some sense, it's not. You have the green owl who have had a great brand forever. How did a couple PhDs from Carnegie Mellon of all places come up with a cuddly green owl and build a I don't think of CMU as wonderful as an institution as it is as like the top design and uh marketing place in the world. Aaron Powell They

• 25:36Speaker 0

  happen to have a really good design program. But look, this evolved over time. We did know one thing. We did want a mascot early on. We were like, look, a mascot is somehow this will help us. We didn't know exactly how, but we we did want a mascot. Eventually we chose an owl because of in Western cultures at least as a symbol of knowledge. It ended up being green because my co-founder hates the color green, and I thought it would be funny if it was green. So that's why it was green. Aaron Powell

• 26:01Speaker 1

  And now your whole office looked behind you.

• 26:04Speaker 0

  The first version of the owl was not very cuddly or I mean it was kind of not great looking, honestly. Over time it evolved. We hired some amazing illustrators. We hired this this guy who still worked for us, Greg Hartman, who's you know our head of art now. He just made an amazingly beautiful owl over time that that evolved. And so that's one thing. We just made a really good asset that looks good. And then the other thing that happened is it started getting a personality, but the personality was a little bit accidental. The first way in which it got to the personalities. The owl was actually not in the app at all at first. The only place where people saw it was the app icon, which is kind of a very close-up version of the owl's face. And then also when you get a notification, it has the app icon, so it looks like it comes from the owl. So the first place where the personality for the owl came up was basically what we said in the notifications.

• 26:52Speaker 0

  And we started saying some weird stuff in the notifications, and I think people started giving it a little bit of personality because of that. Then another thing happened with the notifications. At some point we decided, look, we can't be super spammy, so we're gonna stop sending notifications after five days of inactivity. So if you don't use the app for five days, we'll stop. Then it occurred to us, well, we're gonna stop sending a notification. We may as well tell you.

• 27:16Speaker 1

  Right, get the credit for it.

• 27:18Speaker 0

  Yeah. So people know we're good guys. This is what we thought. So that last notification on the fifth day, we wrote it. It's just said these notifications don't seem to be working, we're gonna stop sending them for now. That's what it said. Well, we didn't know this. We didn't expect this. It turns out this got people to come back a lot because they thought that Duolingo was giving up on them. It's like your mother, like it's kind of like guilt tripping you. Because of that, though, it started giving rise to all kinds of memes on the internet about like how the owl was obsessed with getting you to come back and he would kill you if you didn't come back. And but that you know, that was the community. We didn't do that. After that, we realized that we that we could push on those memes. And so we started making fun of ourselves with those memes. And that has worked really well.

• 28:03Speaker 0

  And at some point, we got the idea of getting an owl suit, a suit that somebody could wear. At first we thought it was good, it was gonna be good for recruiting events, like going to try to hire people at universities and the owl suit would show up. Turned out you could start putting somebody in an owl suit to do weird stuff in TikTok videos, and that gets a lot of people to watch it. And so but that was a little bit accidentally discovered. And in fact, the first person who started doing this was a very young woman, Zarya, who when she proposed it, I thought this is such a dumb idea. I was obviously completely wrong. It turned out this has given us a lot of prominence in in the world. Aaron Powell

• 28:40Speaker 1

  Have you ever tried to quantify. I mean, this is like a core pillar of Duolingo's business strategy at this point. Have you ever tried to quantify the business impact of having all this earned media?

• 28:51Speaker 0

  Yeah. I mean, we have. Our belief is that about 15% of our users come from that. On any given day, roughly 15% of our new users come from

• 29:01Speaker 2

  the owl, for the lack of a better word.

• 29:03Speaker 0

  Some sort of owl, owl thing. The majority of them come from people telling their friends about it. But about 15% come from this weird stuff.

• 29:11Speaker 1

  And just to multiply some other numbers together that you've talked about, if it really is 15% and you have a hundred million users, what's the value of 15 million users? Well, if it's actually at 30 bucks a pop, would be the paid cost to acquire them. That's what, 450 million dollars. Well,

• 29:30Speaker 0

  you're only taking the active ones.

• 29:34Speaker 2

  Aaron Powell

• 29:35Speaker 0

  A user that may not stick around. So it's more than that. This is probably when I you know I don't know the exact number, but it's hundreds of millions of dollars of of earned user. I don't know if that's true, but it's been really good for us. Aaron Powell

• 29:48Speaker 2

  It's

• 29:48Speaker 1

  fascinating. We want to thank our long time friend of the show, Vanta, the leading trust management platform. Vanta, of course, automates your security reviews and components. Compliance efforts. So frameworks like SOC 2, ISO 27001, GDPR, and HIPAA compliance and monitoring.

• 30:12Speaker 1

  Aaron Powell

• 30:13Speaker 2

  Yeah. Vanta is the perfect example of the quote that we talk about all the time here on Acquired. Jeff Bezos' idea that a company should only focus on what actually makes your beer taste better, i.e., spend your time and resources only on what's actually going to move the needle for your product and your customers and outsource everything else that doesn't. Every company needs compliance and trust with their vendors and customers. It plays a major role in enabling revenue because customers and partners demand it, but yet it adds zero flavor to your actual product. Aaron Powell

• 30:42Speaker 1

  Vanta takes care of all of it for you. No more spreadsheets, no fragmented tools, no manual reviews to cobble together your security and compliance requirements. It is one single software pane of glass that connects to all of your services via APIs and eliminates countless hours of work for your organization. There are now AI capabilities to make this even more powerful, and they even integrate with over 300 external tools. Plus they let customers build private integrations with their internal systems.

• 31:08Speaker 2

  And perhaps most importantly, your security reviews are now real time instead of static. So you can monitor and share with your customers and partners to give them added confidence. So

• 31:17Speaker 1

  whether you're a startup or a large enterprise and your company is ready to automate compliance and streamline security reviews like Vanta's 7,000 customers around the globe and go back to making your beer taste better, head on over to vanta.com/slash acquired and just tell them that Ben and David sent you. And thanks to friend of the show, Christina Vanta CEO, all acquired listeners get a thousand dollars of free

• 31:39Speaker 2

  credit. Vanta.com slash acquired.

• 31:42Speaker 1

  Okay, I'm gonna keep coming back to my like what's a heuristic type questions. A startup comes to you with the idea that they're going to run a similar strategy. Any advice for when it could work or not, or is this pure lightning in a bottle? I have no idea when it will work.

• 32:00Speaker 0

  Some stuff I think you can reuse. There's a few pillars in our strategy. One thing that we haven't talked about that I think accounts for probably half the value of the company is our obsession with A B testing. We have A B tested our way to where we are now, and we have probably run I'm gonna make this up, probably about 2,000 A B tests per year of all kinds of things. And so we had just built the company around our entire product organization. That's not just product managers, engineers, designers. The whole system is built around shipping A B tests as fast as possible. That has worked really well for us, and I would say you should do that everywhere. If you have a consumer product, I mean you shouldn't be A B testing stuff if you have like a B2B SaaS thing, whatever, do something different.

• 32:45Speaker 0

  But for a consumer product that has a lot of users, you can really A B test your way to improve almost any metric you can. That you should do. The other thing that I would give advice to is hiring really good people that you should continue doing. In terms of the specific strategy of having a freemium thing that you don't monetize for a while. I'll tell you one of the reasons why that may not work anymore. We have found ourselves, but we have seen it all over the place. We were fortunate that we put out an app in 2012. The app market was not yet set in stone. It's pretty close to set in stone these days. It's not a hundred percent, but if you look at what are the top 50 apps globally for the year this year versus last year, and you compare, they're pretty similar. Maybe one or two changed, but they're pretty similar.

• 33:32Speaker 0

  And it's similar to the year before, and it's similar to the year before. It's just at this point, it's pretty hard to enter. And I don't think that the strategy that we employed in 2012 would work today, because the app market is pretty set in stone. I mean, it's just kind of is.

• 33:49Speaker 1

  Yep. What about the strategy of something clever earning you hundreds of millions of dollars of uh sort of free earned media? Aaron Powell

• 33:57Speaker 0

  It's great if you can do it. You know, there's of course a lot of cleverness in this that many employees at Duolingo did, but I think we were lucky in a few ways. I think the fact that we're an education company gave us license to do some stuff that many other companies may not have been able to do because it's kind of hard to say that language learning is bad for you. So that means that we can do risque stuff on TikTok. But if a company that is trying to do like, I don't know, like some sort of fintech company or something. I think it's much easier to accuse them of stuff. And I think that helped with us that just come on, everybody, we're the good guys. I think we were able to get away with a lot of stuff because it's hard to say bad things about language learning. Aaron Powell

• 34:38Speaker 1

  Right. If I'm coming at you with you could argue it's a fair interest rate, or you could argue it's a predatory interest rate. If I have a mascot that's sitting there with a knife saying, use our app more, it's it's not gonna go well.

• 34:48Speaker 0

  Exactly. I think I think we benefited a lot from the fact that ultimately we're doing it for education, so people are okay to take some of this.

• 34:55Speaker 1

  Right. Fascinating. Aaron Powell By the way, you know, another place.

• 34:59Speaker 0

  When we talk to other companies, every company wants to talk to us. Not because they learn from us, but because they're like, oh, you guys, yeah, you guys are kind of fun. This is good. Nobody thinks of us as competitive. Well, that's great for us. And it's because almost every time we talk to whoever, fancy person at random company is like, my daughter loves you. You know, I think we benefited a lot from that.

• 35:18Speaker 2

  Which is crazy. Like how many other 10 billion dollar companies could that be said for? Like if you're a 10 billion dollar company, almost you'd think like de facto, you're gonna have people who are your mortal enemies. Again, because everybody's like, ah, green all. Yeah.

• 35:34Speaker 1

  Yeah. So it's an interesting comment on competitors. At this point, you probably don't have a whole lot of meetings about legacy language learning companies and saying how can we acquire users versus them? Who do you think about as competitors for your users?

• 35:52Speaker 0

  I mean, the way we see it is we compete for time. And so our competitors are Instagram, TikTok, et cetera. That's that's who we compete for time. And by the way, we're losing. I mean, we're we have much less time spent on our app than TikTok has or or Instagram, et cetera. But when we look at our users and we see that our users maybe stop using Dualink or something, we ask them, you know, why'd you stop? The most common answer is like, yeah, you know what? I'm spending more time on pick your random social media site. I'm spending more time on Instagram. That's it. The competitors is whatever takes their time. Aaron Powell

• 36:22Speaker 1

  All right. Well, you planted a seed of something that we talked about for like one minute in this podcast, but every person I talk to who's ever interacted with Duolingo, former employees and other podcasts you've been on is all about experimentation, A-B tests being data driven. There has been lots of inks built on this for anyone who wants to read excellent blog posts about the you know the AI behind your push notifications and how all those systems work. One high-level question for you. How do you, as a leader, balance a cultural tenet being a data and experimentation driven versus instinct driven?

• 37:02Speaker 0

  We're actually weirdly both. I'll tell you where this comes from. We realized early on that pure experimentation can lead you to really bad places. If you kind of just don't think about it at all, you just you're just gonna optimize some metric. That can lead you to really bad places. And one of the best stories that I heard about that, and we repeat this a lot at Duolingo is something that we learned from the founder of Groupon, who happens to be from Pittsburgh. I mean, we're located in Pittsburgh. He happens to be from Pittsburgh. Yes.

• 37:32Speaker 1

  We're huge D script users.

• 37:34Speaker 0

  Yeah, yeah. It's a great guy. He told us this story, which uh, you know, I'm gonna tell it, and at this point, it probably has changed a little bit because I you know, I probably remember now what is in the lore of Duolingo as opposed to exactly what story he told us. But you know, he told us a story about emails that group on set per day. So, you know, group one, basically you got a deal, and the idea is at the beginning you could only they could only send one email per day with one deal. He was strict on it. It's like only send one email per day because more than one email is spammy. At some point, some enterprising product manager shows up and says, What about if we send two? Andrew Mason says, no way. Like, no, we're only gonna send one. And then this product manager gave this amazing argument against it. We're not trying to launch two. We just want to know are you against knowledge?

• 38:22Speaker 2

  And

• 38:22Speaker 0

  of course, nobody's against knowledge. So you know, he's so he said. I'm using this even fine, fine. And so they tested two. Guess what happened when they tested two? Two is way better than one. So they started sending two. And then like, well, if two is better, test three. Turns out three is better than two, and four is better than three, et cetera. And they A B tested their way to like, I don't know what number, seven, eight or something, until the channel stopped working because people marketed a spam, and then it it died. The channel died. But so this is the thing about A-B testing your way. Like you have to you have to really do the common sense thing above the A-B test. And so what we do at Duolingo is we have we have this two-step process. I mean, the first one is before you even A B test something, and by the way, here we are against knowledge.

• 39:10Speaker 0

  There's certain things we just don't want to know. And it's because it's too tempting to do it. So the first step is before we do an ID test, we have this process called product review, which I, you know, I assume happens in many other companies, but in this case, every single change that goes to the app gets product review. And product review is basically a set of leaders in the company that includes me. There's about five people, and we look at every change and we have to agree for it to go out. We stop a lot of things before going out to prevent this type of stuff. Here we use intuition and just common sense. We're like, no, nobody wants to receive 32,000 notifications. That's just dumb. Even if the A B tests us so I think that has helped us quite a bit. Aaron Powell

• 39:54Speaker 1

  Do you have an example of uh way in which you've been anti knowledge in the past, something you decided you didn't want to know.

• 40:01Speaker 0

  Oh, yeah. I mean, pretty recently, somebody wanted to know what happens if you put an ad that takes over the whole screen when somebody opens the app. Like a product manager wanted to know that. And we were like, yep, but we don't. Particularly with monetization, it's so tempting because what happens with monetization is you run the experiment. And then it says it says something. Like, if you launch this experiment, it will make you 50 million dollars a year. And you look at it and you're like, that's nice. And then the finance team figures out that this happened. Like somebody from the finance team looks at this experiment, and suddenly the CFO is in my conference room or something, and he's like, yo, 50 million bucks a year, you gotta launch that. We try not to know the answer to many of these monetization experiments.

• 40:47Speaker 0

  Aaron Powell

• 40:48Speaker 1

  When you wear your CEO or your head of product hat, these things are sort of obvious. And what's underneath it is this implicit assumption that there is a future dollar value associated with treating the user with respect and having a feel-good association with your product. It's basically like all of the potential future dollars we could earn are jeopardized by doing this. We're pulling forward revenue. If you do something, it's

• 41:14Speaker 0

  very hard to quantify, but it's very hard to quantify.

• 41:17Speaker 1

  Complicated expected value calculation on it.

• 41:20Speaker 0

  Yeah.

• 41:20Speaker 1

  Have you ever tried to quantify that brand value or future value to

• 41:26Speaker 0

  We haven't. No, we haven't. What we just have is a group of I'm saying five, but it's on the order of five leaders that have a very good gut feeling about no, this is this is too much or no, this is not good. And I think their gut feeling is quite good, I would say. We run into this problem. Sometimes, you know, we do run an experiment and where we have to make a trade-off. Like for example, we're in some experiment because we know if we do this, we lose 10,000 users but make this much more money or whatever. We've never known how to make these these trade-offs. So generally what we do is we go for users, as in if something decreases the number of active users that we have, unless it's gonna make us 500 million dollars. We're really not gonna launch it. Statistic of laughing, you know, Ben joked

• 42:12Speaker 2

  about using that line on me. This applies to any business. Like for us, if you told Ben and me, okay, go objective function maximize the value of acquired. We could come up in 10 minutes with a whole list of things that we know exactly what we would do, and it would make an obscene amount of money.

• 42:30Speaker 1

  Welcome to this daily episode of acquired.

• 42:34Speaker 0

  Yes, it would, but for the next few months and not for the next few years.

• 42:37Speaker 2

  I'm not even sure that that like that might be the value maximizing thing to do. But we would never do it because it would just kill our hearts, you know. Like,

• 42:47Speaker 0

  yeah, that that also matters a lot. Yeah. It's funny though, your heart you can boil a frog quite well, and product managers at Duolingo have gotten really good at uh boiling my heart slowly. Things that I think I was not okay with five years ago, I think I'm a lot more okay with because they just kind of push the envelope every slippery slopes. They they get you.

• 43:10Speaker 1

  But do you end up feeling like, okay, you know what? They had a point. Like I no longer

• 43:14Speaker 0

  Yeah, sometimes I've definitely been wrong about stuff. I mean, that's that's for sure. But I still claim in the long term, it really is better to do what's best for the user. And I don't mean what the user's asking for, because the users ask for weird shit. I mean what you think is better for the user. I think I think that in the long term, I'm pretty sure that's that's value maximizing.

• 43:33Speaker 1

  I think this is a pretty specific thing to founder-led companies, which is we will do what I personally feel in my gut is best for the users. And like that just carrying the day every time. That does not happen in a professional management setting.

• 43:51Speaker 0

  It may not. What I'm happy about is that I would say five years ago, it was solely my gut. And I think at this point, there's a group of leaders, Jem, our head of product, Simi, our chief design officer, that the whose gut somehow is like identical to mine at this point. Maybe 1% different. But I I feel pretty good about the fact that if I were hit by a bus, these these people's guts is is pretty good.

• 44:18Speaker 1

  So we've talked a little bit about this idea that you have a bottom of funnel governor on what goes out. And we've talked about that there's a mid-funnel thing that's the A-B test. And I think you've said in the past,

• 44:28Speaker 0

  the latest number is 50-50.

• 44:29Speaker 1

  50-50, okay.

• 44:31Speaker 0

  It is eerily close to 50-50, as in like it really is 50-50. It's not 4852. I can't explain. It's one of those where I'm just like, really? 5050? Come on, people. It has to be something different. But now it's pretty close to 5050.

• 44:44Speaker 1

  So that gets to this question. Why is it that over time your collective gut has gotten better about what experiments will succeed? Aaron Powell

• 44:54Speaker 0

  The way it has gotten better is because we get the results of the experiments. Our gut is kind of a learning machine. We're learning. We're like, yeah, that type of stuff. At this point, we have years of experience. The people that have been doing product review have been doing it for years now. We have years of experience knowing this type of change does that type of thing. And we're pretty good at it. We're not 100%, but we're pretty good at it. Not only do you know which direction things are going to go, but you also know which ones have more impact. So for example, we know that the session end screen. So that is the screen that you get after doing a lesson matters a lot in terms of getting you to come back the next day. But we know that other screens, like a screen in the middle of a lesson almost matters not at all in terms of getting you to come back the next day. So we've just kind of learned what what screens matter and what parts of the app matter for certain metrics.

• 45:44Speaker 0

  We just have a really good gut on that.

• 45:47Speaker 1

  Do you have to treat big experiments differently than little experiments where your gut may not have as much training data in it? Aaron

• 45:55Speaker 0

  Powell Yeah. And there's some experiments that are a lot more complicated than others. Some stuff takes us a long time. Well, first of all, now we also have pretty sophisticated data science that tells us for how long to run certain experiments. We also have a pretty good gut feeling about this particular experiment should run for a pretty long time. Most experiments we run by the way, we run them for two weeks. We have enough data in two weeks to know what to go. But there are some experiments that we run for about six months because we know that that you're gonna see a very big difference at the beginning than later because people get used to the thing.

• 46:25Speaker 1

  There's a very heavy statistics component to the core competency of your company. Where did that come from? I mean, the

• 46:33Speaker 0

  first many employees were basically computer nerds from Carnegie Mellon. I think that's where that came from. We understood things like statistical significance coming in.

• 46:42Speaker 1

  Yeah, makes sense. Does it feel related to your crowdsourcing research, or does it feel like you you brought someone on who was a super key hire and then you know made that a pillar of the company?

• 46:55Speaker 0

  I don't know if it's exactly my crowdsourcing research, but it's generally the first few people that worked at the company, we had a pretty good grasp of statistics. What I will tell you though is that it's funny. This is true for most of the things, and I don't think this is unique to Duolingo. For most things, you know, we had a pretty good grasp of statistics. That's good. But the level of sophistication that we do analysis today compared to that is insane. I mean, we now have people with PhDs in statistics who actually, honest to God, know what they're doing. As opposed to we had quote unquote a pretty good grasp of statistics. Early on when we were figuring out how to teach things, you know, we kind of had an idea of how to teach things. By now, you know, we've hired dozens of people with PhDs in second language acquisition and the level of sophistication that they have in terms of learning outcomes and stuff like that compared to what we were doing at the beginning is night and day.

• 47:44Speaker 0

  So we had some expertise for a lot of things, but today we just we've become a lot more sophisticated about most things, including the statistical analysis of our experiments. Knowing what I know today, I'm pretty sure that experiments from eight years ago, many of them we analyze wrong. What do we know? We're like, yeah, we look at averages, we look at statistical significance. We were not looking at the stuff that people look at today. In fact, I don't even understand what they're doing. So they just very smart people come and tell me, do this. I'm like, sounds good.

• 48:13Speaker 2

  So all this to me, and I bet to many listeners sounds like the perfect TF for like AI. You are a perfectly suited company to AI. Maybe just at the highest level. What's going on? What's changing? How is it impacting everything we just talked about? Aaron Powell

• 48:34Speaker 0

  You are right. We are very well suited for for AI. I mean, we've been the whole time. You know, AI, of course, means something a little different today than it did 10 years ago, but AI has been around for a while. But since we're 10 years ago,

• 48:44Speaker 2

  AI meant everything we were just talking about.

• 48:46Speaker 0

  Yes, yes, statistics. You know, since we launched, we were very clear that what we wanted to do is we wanted to have a computer teach you a language, not another human teacher language. Another human can teach you a language very well, that works, no problem. But having another human teacher language is expensive. We wanted to have a computer teacher language. So of that, we're very well suited. We need to make a computer that is as good as a human teacher. And so we've been working on doing that as much as we can. Over the last couple of years, what has changed is these large language models. And again, I emphasize that the second L is language. We are a language learning company. Large language models are particularly good with language. And because of that, we've been using them a lot. I mean, so I'll tell you the two big things where we where we're seeing large language models be used.

• 49:35Speaker 0

  One is it used to be the case that a lot of the data that we created for our courses was half handmade. So by that I meant we made tools, but usually a lot of the stuff was humans doing things, and then you know, maybe maybe the computers would fix it up or whatever. But it's kind of half handmade. We've gotten to the point where most it's not a hundred percent, but most of the stuff that that we're we're creating data for language learning is made by An AI. There's a couple of good things about that. First of all, it saves us money. So it's a lot cheaper than doing the humans. But the most interesting thing that has happened is we can now create data so much faster. And it's not that we can go fast. Okay, that's nice. It's that that allows you to do things that you weren't going to do before. So for example, about five years ago, somebody pitched a feature on Duolingo, which was a type of lesson where it's kind of like a mini, call it like a two-minute podcast where you you basically learn how to listen in the language that you're learning.

• 50:31Speaker 0

  It's like a it looks like a little animated thing. We call it dual radio now. But it was like a thing where you could watch two minutes of something and listen to it and you're learning how to listen. And I was told that creating the data was going to take five years. And as soon as I was told that, I was like, no. No. This feature red light here. Do not proceed. I do not want anybody working on five years of seeing something that I don't even know is going to work. A very similar feature now, we can create the data for this in a couple months. And so now I'm like, yeah, sure, why not? Aaron Powell

• 51:04Speaker 1

  What is it about today that causes that incredible?

• 51:07Speaker 0

  All of that data is done by a computer as opposed to having it done by hand. I mean, for it to make a difference in the dualingu language learning app, we needed to make hours of content. Like many, many hours of content. And specifically in this case,

• 51:18Speaker 2

  you're talking about like animated graphics. Aaron Powell

• 51:21Speaker 0

  Not just animated graphics, the audio of it. Like actually, you know, what you're gonna say, you have to make it interesting. So you have to make something that people had to listen that was interesting, et cetera. So you had to do all that. It was super slow to make hours of content, and it was going to take five years to make content. And not only that, you had to make it in all the languages that we teach. We teach 40 languages and we have to make hours of content. So that was going to take an inordinate amount of time. But now, you know, with an LLM, you can do it pretty fast. And what we do is we do it, then we kind of spot check it. And if we don't like it, do it again. We're very happy with that, that you can basically do things that before were just it were not impossible to do, but practically impossible to do. Like why would you do that? So that's an important place where we use AI. The other big one is just conversational practice.

• 52:06Speaker 0

  We're working on this feature that some of our users already have. We haven't fully really launched it to everybody. But it is basically where you can have a conversation with one of the duolingual animated characters. And it looks like you're having a video call with an animated character, and it's a really good way to practice a language, an actual conversation. That's something we just couldn't do before. Yeah,

• 52:26Speaker 1

  I've heard Mark Zuckerberg say about Llama use cases a few times, or maybe it's about meta AI use cases that it's for people to practice conversations that they don't want to have yet in real life, but they need to sort of feel like they're having the conversation and get some authentic responses.

• 52:41Speaker 0

  Well, so that in the case of language learning, what's amazing about it is look, you want to learn a language, you have to be able to practice conversation. You just have to do that. And people don't want to do it. In fact, we've done insane amounts of user studies where people they may tell you they want to practice a language with another person, but when it comes down to it, they don't want to do it. There's some small fraction of extreme extroverts who like it. But the majority of people would just rather not talk to another person in a language that they're not very good at. But the beautiful thing here is that you can just talk to an animated character and you really don't care because you don't think they're judging you, even though technically they're really judging you. Because that's mainly what the thing is doing, but you just kind of don't think about that.

• 53:21Speaker 2

  Well, hey, that's just absolutely incredible. And AI and data also opens up the other direction for you too. Like you have an incredible amount of data within Duolingo that can be used and fed back into LLMs, your own, I would assume, and others that would just be incredibly, incredibly valuable. Does that open up a whole nother new business dimension for you all?

• 53:46Speaker 0

  Maybe. At the moment, that's not something we're doing. We're using the data for ourselves. More than one billion exercises are solved by our users every single day. So we have the data, and basically we're watching uh I don't know, tens of millions of people learn a language every single day. We're using it to teach better. Today, we know that Duolinguists are about as good as a classroom in terms of learning a language, but not as not yet as good as a good human tutor, like a one-on-one, good human tutor. The problem is those those are like $50 an hour, like a good one. We're not as good as that. You know, it's hard to predict the future, but um I'm assuming that if you give us three-ish years, we'll be we'll be about as good as a human tutor. Aaron Powell

• 54:23Speaker 1

  Well, and it requires making an hour. I mean, if you're doing three minutes at a time, that's a structural advantage where much better. Yes. You're just able to address people that will never ever make the hour. Aaron Powell

• 54:34Speaker 0

  Yes, and and you can't have a paid human tutor that is there for three minutes and then another three minutes later and then this

• 54:42Speaker 1

  all next to you to

• 54:44Speaker 0

  just waiting around.

• 54:45Speaker 2

  Well,

• 54:45Speaker 1

  it's

• 54:46Speaker 2

  cool.

• 54:46Speaker 1

  I mean, it's

• 54:46Speaker 2

  funny, you say three years. I wonder if it's fast because like you got two tracks of progress happening that are accelerating. You've got the foundation models of LLMs themselves, which are getting better. And then you've got your data, which you're getting, you know, what did you say? 10 million solved exercises a day or two. No, a billion per day. Sorry, a billion per day, right?

• 55:08Speaker 0

  So the quality of both of those are accelerating. So I'm really bad at predicting the future, how long things are going to take.

• 55:14Speaker 1

  Well, humans are, and you're a human. So

• 55:17Speaker 0

  it's very hard. I it's a matter of a small number of years to get as good as a human tutor. Aaron Powell

• 55:23Speaker 1

  You're in this education space. Are there any education companies that you look up to as shiny examples of people who've built great companies, or do you mostly look outside your sector for inspiration? Aaron Powell

• 55:36Speaker 0

  It's mostly outside. Education has been hard because you want to start an education company. You run into a lot of problems. First of all, for some reason, most education companies have decided to monetize by charging the school systems, which is fine, but we just can't look up to those companies because we're so different. We're a consumer company. We've have in the past looked at those, but there's very little to learn from somebody who has spent a lot of time making a sales force for K through 12 schools. I'm glad they're doing it. Good for them. It may work, I don't know, but it's not much for us to learn. So ultimately for us, where we learn because we are a direct consumer, we learn more from games, companies like Spotify, Netflix, any of the meta apps. That's where we learn the most, but it's because we are direct to consumer.

• 56:23Speaker 0

  There's just not very many direct-to-consumer scaled education companies. Aaron Powell

• 56:28Speaker 2

  Well, there are, but they're all nonprofits with brands like you know, CMU or Harvard or Stanford or you know.

• 56:37Speaker 0

  But yes, yes, they are there, but they're not apps and they're not touching a hundred million people. They're touching a few thousand people. But yes.

• 56:44Speaker 1

  It's kind of interesting to think about. I mean, they started as every institution starts as this sort of niche thing using the latest and greatest technology available. Then over time you build a durable brand and you try not to jeopardize that brand in any way. And those words that I just said could apply to Duolingo, and you're just like 75 years earlier on your journey than any of these other institutions. Like is the goal to become a accreditation bearing institution where people feel like when you've graduated from it, it means something.

• 57:19Speaker 0

  I would really like that. We are doing that for language. We're trying to do that for language. We have a standardized English test. We're doing it for English mainly, because that's where it matters of this test called the Duolingo English test. And the idea is that this is a test that you can go and take it, you can take it online and it gives you a score and it tells you how good your English is, and that is actually valuable. For example, it is now accepted by most universities in the US. If you're a foreign student that wants to come to the US, you have to take uh an English test. 98 out of the top 100 US universities, according to US News and World Report accept our test. We haven't quite connected the test and the app, but we want to. For some languages, you it's that's already there. We're giving you an estimated test score. So if you just use the app, we'll be like, oh, my test score is this much.

• 58:04Speaker 0

  And then you can go and certify the score. You know, what I would love to do is to be able to own the score for languages. So right now, if you ask somebody how much how much French do you know? The answer is usually something to the effect of I took four years of high school French or intermediate. But that doesn't really tell you what we would love to get to the point is if when you ask somebody how much French do they know, they'll say, Oh, I'm a Duolingo 65. We're not there yet. Right now, people will tell you they're using Duolingo, and they may tell you they're Duolingo streak, but that's something that we we want to get to. And I think that'll be really valuable. You're right. I mean, maybe in 75 years we'll just have a really stodgy stone buildings and a 30 billion dollar endowment.

• 58:44Speaker 2

  Well, it's funny you took, I mean, like, what would the market cap of Stanford be if it were a you know publicly traded corporation? Like it would be quite large.

• 58:52Speaker 0

  It would. And you know, we have this board member, Bing Gordon, who's who's amazing, and he'll happily tell you. He'll say, market cap of a university is twice its endowment.

• 59:03Speaker 2

  Oh, wow, that feels like a low multiple to me.

• 59:05Speaker 0

  I don't know where he pulled that from. But this is what he'll tell you. I don't know. I don't know where he pulled that from, but he'll tell you. So Stanford's probably, you know, I don't know what their endowment is 30 billion, so something like that. Yeah, okay. So it's an 80 billion dollar company. Aaron Powell

• 59:20Speaker 1

  This is such an interesting thought experiment because the mission of education is so unbelievably important, but it has historically, at least you know, last 20, 30 years been so hard to invest in education technology companies because they so rarely are using modern business models, the technology that all the other consumer companies are using, the distribution methods that all the other consumer companies are using. They're instead sort of trying to snap to how do I sell into the education supply chain. And you just have it. Like you I you might be the one example of this education technology company that said, I am not playing in that existing system. And that is how you invent something new. Aaron Powell

• 59:56Speaker 0

  It varies on the day, but we're certainly the most valuable ed tech. Company. We may be the most valuable education company. Some days Pearson is more valuable than us, some days we're more valuable than them. But what's interesting is we're doing it with just one app. And if you look at something like Pearson, they sell gazillion books and stuff like that. I mean, we we feel pretty good about that, that we really are actually able to teach better. But you know, we we feel pretty good about that.

• 1:00:20Speaker 1

  All right. I want to close with a question on headwinds and tailwinds. What's the most insane tailwind that just makes things easy for you running Duolingo? And then other side of it, what is a headwind that you think you are just always going to have to deal with? And it's a thorn in your side, but that's all right.

• 1:00:36Speaker 0

  AI is certainly a big tailwind for us right now. I think there is a growing proportion of people who want to do useful things online. And I think we get tailwind from that. I think if you use Duolingo for 30 minutes, you don't feel the same as if you use Instagram for 30 minutes where you're like, my God, I just lost 30 minutes of my life. I do think that's a real tailwind for us. In terms of headwinds, this is more for me as a person. Employee problems are tough. You know, we hire the top of the talent pool. I'll use the word entitlement is there. That's stuff for me. I grew up in a country where literally for two, three hours a day there was no electricity.

• 1:01:24Speaker 0

  There are some people here who are like, why do we only have two flavors of coconut water? That's always hard.

• 1:01:34Speaker 2

  It's funny to connect that with something you were saying earlier. Like part of why that exists is the XYZ elite educational institutions of these days and their brands and their brand caption, whatnot. But like if there were just a Duolingo score of like how good a product manager you are, uh that that would level the playing field a lot better.

• 1:01:51Speaker 0

  Yeah, exactly. We could get this this people that weren't uh fed with a silver spoon as much. Aaron Powell

• 1:01:57Speaker 1

  To go back to your comment of you grew up in an area where you know there were multiple hours of the day reading of electricity. How much do you feel like growing up in Guatemala? And I think you went to one of the sort of rare private English speaking schools to have the opportunity to learn English. How much of that formative experience for you contributed to you founding Duolingo?

• 1:02:20Speaker 0

  It's a lot. I mean, first of all, in a place like Guatemala, language learning is life-changing. That was a big deal. But I think the other thing is I was fortunate. I received a rich person's education. Even though I didn't grow up rich, I was fortunate that my mother spent basically all her money on sending me to this fancy school. So I went to school with very rich kids. You know, it's not like we were poor. We were not, but we were you know kind of middle class in Guatemala, which probably for US standards is poor. But in Guatemala we were middle class. I really saw the difference because where I lived in the neighborhood where I lived, the kids didn't go to the school that I went to, and I could see how much more I was learning. I mean, it was pretty clear that I was learning, I don't know, twice as much as they were. That was a big thing when when starting Duolingo. We wanted to do something that would give you access to education to everybody. So it's a big thing.

• 1:03:06Speaker 0

  Me growing up in Guatemala had a lot to do with starting Duolingo.

• 1:03:10Speaker 1

  Fascinating. Well, that's a great place to leave it. Luis, thank you so much for joining us.

• 1:03:15Speaker 0

  Ben, David, thank you. Great questions.

• 1:03:17Speaker 1

  And listeners, we'll see you next time. We'll see you next

• 1:03:20Speaker 0

  time.