Failure

In this special compilation episode, we delve into failure—an overlooked source of wisdom. From freezing onstage in front of thousands of people, to coworkers staging an intervention, to huge product investments that went to zero, we’ve pulled our favorite stories of failure from 100+ podcast episodes. I hope these stories serve as a gentle nudge to view failure not as a setback but as a crucial detour toward growth. — Brought to you by Sendbird—The (all-in-one) communications API platform for mobile apps: https://sendbird.com/lenny | Eppo—Run reliable, impactful experiments: https://www.geteppo.com/ Find the full transcript at: https://www.lennysnewsletter.com/p/failure Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ In this episode, you’ll hear from: (00:00) Lenny: Why I’m focusing on failure (03:25) Katie Dill: The single meeting that changed how Katie leads forever (08:03) Paul Adams: Freezing onstage in front of thousands of people (18:38) Tom Conrad: Lessons from Pets.com and Quibi—two of the most famous product disasters of all time (33:19) Sri Batchu: When you fail, make sure you fail conclusively (39:00) Jiaona Zhang (JZ): One of the biggest product misses at Airbnb (44:32) Gina Gotthilf: Everyone has an “A side” and a “B side,” and we should all share our B sides more (57:57) Maggie Crowley: Her favorite interview question about failure, and lessons from a personal failure (1:00:33) Thanks for listening Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected]. Lenny may be an investor in the companies discussed.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Lenny Rachitsky?

  Today, we've got another very special compilation episode. Something I've been pulling on more and more with the podcast and the newsletter, in case you've noticed, is failure. Normally, I spend a lot of time researching how the best companies and the best product leaders operate, but you can learn a lot, and often a lot more, from failure. And so what we've done with this episode is we've looked at all of the past episodes we've done and pulled out all the most interesting and insightful stories of failure and turned it into this very focused episode on failure. I hope that you find this useful and interesting. Let us know what you think in the comments on YouTube or on Lenny's newsletter.com, or just let me know on Twitter. If you like this, we'll keep doing this. If not, well, probably not. Either way, I hope you enjoy.

• 0:46Lenny Rachitsky?

  Before we dive in, here is a short word from our wonderful sponsors. Let me tell you about a product called Sendbird, the all-in-one communications API platform designed for both web and mobile apps. In a world saturated with multi-channel communication, product teams are discovering the effectiveness of in-app communication. With Sendbird, businesses can elevate their in-app experience with decluttered and branded communication, featuring AI-powered chatbots, one-way messages, chat, video calls, and live-stream capabilities, all tailored for commerce, marketing, and top-tier support. Forward-thinking companies such as Hinge, Patreon, Yahoo, Accolade, and more, use Sendbird to build in-app communication experiences that drive engagement,

• 1:31Lenny Rachitsky?

  conversion, and retention. In-app communication has the highest conversion, highest engagement, and highest satisfaction of any communication channel. And when it comes to investing in this channel, trust Sendbird to take your in-app communication experience to the next level. Start today with Sendbird's free plan. And as a listener of Lenny's podcast, you'll get an additional two months of unlimited usage and access to all premium features, including creating your very own generative AI chatbot. Visit sendbird.com/Lenny to begin your free journey. That's sendbird.com/Lenny.

• 2:08Lenny Rachitsky?

  This episode is brought to you by EPO. EPO is a next-generation A/B testing and feature management platform built by alums of Airbnb and Snowflake for modern growth teams. Companies like Twitch, Miro, ClickUp, and DraftKings rely on EPO to power their experiments. Experimentation is increasingly essential for driving growth and for understanding the performance of new features. And EPO helps you increase experimentation velocity while unlocking rigorous, deep analysis in a way that no other commercial tool does. When I was at Airbnb, one of the things that I loved most was our experimentation platform, where I could set up experiments easily, troubleshoot issues, and analyze performance all on my own. EPO does all that and more with advanced statistical methods that can help you shave

• 2:52Lenny Rachitsky?

  weeks off experiment time, an accessible UI for diving deeper into performance, and out-of-the-box reporting that helps you avoid annoying, prolonged analytics cycles. EPO also makes it easy for you to share experiment insights with your team, sparking new ideas for the A/B testing flywheel. EPO powers experimentation across every use case, including product, growth, machine learning, monetization, and email marketing. Check out EPO at geteppo.com/leni and 10x your experiment velocity. That's geteppo.com/leni.

• 3:26Lenny Rachitsky?

  Alrighty, first up, we've got Katie Dill, who is Head of Design at Stripe and former Head of Design at Airbnb and Lyft, sharing this amazing story of how the entire design team at Airbnb basically rebelled against her soon after she joined and what she learned from that experience.

• 3:42Katie Dill?

  I'm happy to talk about it because frankly, it was the biggest learning experience of my leadership career, or at least that happened in one moment. And it happened in my early days at Airbnb. So I was hired to take on the design organization, or the experience design organization, that's basically the product design team, which was 10 people at the time. And so they had been reporting directly to one of the founders, and they were going to start reporting to me. And during my interview process, I learned a lot about what was working and what wasn't working and some of the trials and tribulations with the design organization and its collaboration with others. So it seemed like there was room for improvement in how engineering, product management, and design all work together. And there was also really low engagement

• 4:27Katie Dill?

  scores in the design team. And so I kind of came in ready to go and excited to try to help make some change based on all the things that I had learned from various leaders and people across the company. And I came in swinging, ready to go. And then about a month into my time there, I got a meeting on my calendar, Thursday, 8:30 a.m. It was an hour and a half with half of the design team, so that was five people, and our HR partner.

• 4:58Lenny Rachitsky?

  Oh, no.

• 4:59Katie Dill?

  Usually.

• 4:59Lenny Rachitsky?

  That's never a good sign.

• 5:01Katie Dill?

  Yeah. And I remember this so vividly and remember walking into the office and, you know, all the rooms in Airbnb's office are very unique spaces that look like Airbnb's. But of course, this was the one room with like all white walls and just like a gray, you know, flat rectangle table. And I walked into the room and there were five of them seated around the table and they had a pack of papers in front of them. And they went on taking turns, you know, quietly reading from the papers, all the things that they saw that I was doing wrong and all the things that they didn't like about me. And, you

• 5:42Katie Dill?

  know, it was a really hard moment. You know, there, I went through all the usual kind of like stages of grief when one hears feedback, which is just like, you know, immediate want to like respond to be like, oh, like, you know, well, there was a good reason for that. And like, that's not how it actually was. And this is why I did that. But luckily, I had, you know, goodness, I had the sense to just like, listen and not respond in that way. I mean, clearly what they were telling me is that, you know, that was one of the things that was missing. And so I heard them out and, you know, took it all in. And, you know, regardless of each individual saying, what was very clear was that, you know, the missing piece, the theme that was across all of that, is that I hadn't earned their trust.

• 6:24Katie Dill?

  So whether, you know, how right or how wrong what I was doing was is like the key piece is that I wasn't bringing the team along with me. They had no idea that they could trust in, you know, what I was trying to build and what I was trying to shape and that I cared about them and that I had, you know, their best interest and shared goals at heart. And that was absolutely my fault. And in retrospect, you know, as hard as that was, I'm like very grateful and very amazed that, you know, that they could kind of come together and share that with me. It can be hard to bring, you know, feedback forward like that. And so it was an extremely valuable learning experience. And I took from that to then immediately shift how I was operating. And really a key

• 7:07Katie Dill?

  part in building trust was to, to listen, to hear out, you know, what, you know, the individuals on the team were, you know, setting out to do, what they cared about, that, you know, what motivated them. And so I started to make pretty fast change and still moving in the direction that, you know, was necessary for the org to make like the, you know, the really large impact in how we were operating. But bringing folks along with me, you can inflict change on people. But if you want to do it with them, you really, you know, trust is the key element there. And then, you know, a couple of months later, we had the best engagement scores in the company. So like it actually like, you know, it did objectively improve the situation. And, you know, since then, taken that on into next

• 7:49Katie Dill?

  steps and other companies that I've joined and just think about like, instead of coming and swinging, you know, come in listening so that you can really, you know, set out to make change that actually has like true positive impact on the folks around you and you bring along with you.

• 8:04Lenny Rachitsky?

  Next up, Paul Adams, Chief Product Officer at Intercom, sharing the nightmare experience of freezing on stage in front of thousands of people, having to walk off stage and then people hearing him curse because the mic was still on, and then how he recovered, plus a few stories of building some of Google's most infamous product failures.

• 8:23Paul Adams?

  Yeah, some things that happen in work, I remember at the time, they don't really scar you. This goes in the book that I've scarred for life. Yeah, it's good long story short, I was at Facebook just over a decade ago, loved it at the time, I think it was a great place to be at the time. And based in San Francisco, I did a lot of talks for Facebook internally and externally. Facebook had a keynote slot, always have a keynote slot at Cannes, the world's biggest advertising festival. And the year prior, Zuck had been interviewed, he was the speaker, he'd been interviewed, gotten a hard time on privacy. It didn't go well, as well as they'd hoped. So the next year, they asked me to do it.

• 9:02Paul Adams?

  Maybe it was the Irish accent, you know, that made the offer come my way. And yeah, I got out and spoke on the stage, you know, the world's biggest advertising stage. And I'd say I was like three, four minutes into the talk, a talk I'd given, a very similar talks I'd given lots of times. And I just froze. I couldn't remember what I was supposed to say. It was the first ever time in my life I'd rehearsed a talk word for word. You know, usually like I have talking points and I ad lib and, you know, things get mixed around and it's kind of informal. This was like, you know, media trained, like, don't do not say the wrong thing kind of talk. And I just could not remember what to say.

• 9:40Paul Adams?

  I had some version of a panic attack, walked off stage, I was still mic'd up, cursed, everyone started laughing. I was like, jeez, are they laughing at me? You know, oh my God, this is... But I could manage to turn it around. I walked back out. I'd kind of been disarmed internally in my head and the rest of it went well. But it was, and I was famous that night, you know, out in Cannes afterwards, like on the, whatever, the sea front, it's just like rosé everywhere. And yeah, I was famous and infamous for my performance.

• 10:12Lenny Rachitsky?

  I feel like you lived the worst nightmare that everybody has when they're thinking about giving a talk. And I think what's interesting is you survived. And I think that's a really interesting lesson, is like, you could freeze in front of thousands of people, walk off stage, and then it works out okay.

• 10:30Paul Adams?

  Yeah, and it all happened kind of organically, I guess, or very naturally, you know. But yeah, ever since then, every time I walk, go out onto a conference talk stage, still today, I ask like, ask myself, I have this tiny doubt in the back of my head, like, it's never happened since. But yeah, you just, I think you have to go with it with these things, you know, like when life kind of throws you these, whatever curveballs, you have got to kind of adapt and it's not that big a deal. None of these things are that big a deal at the end of the day, you know, you kind of move on, live and learn. So, yeah, but I still hope it doesn't happen again.

• 11:02Lenny Rachitsky?

  I also hate public speaking and I always fear this is exactly what's going to happen to me. And so I think this is nice to hear that even when the worst possible thing basically happens, things can survive.

• 11:13Paul Adams?

  You can turn it around, yeah.

• 11:15Lenny Rachitsky?

  A second area I wanted to hear from is your time at Google. And there's a couple of products you worked on at Google, both of them were not, not what you call big successes. And then there's a kind of a transition to Facebook, which was also kind of messy. Can you just share a couple of stories from that time?

• 11:31Paul Adams?

  Yeah, similar to the face, to the kind of like, you know, walking off stage thing. You live and learn and I was at Google for four years and I was on Facebook for kind of two and a half years or so. At Google, I worked on a lot of failed social projects, like you mentioned, Google Buzz, Google, then later Google Plus. I think a lot of the motivation for those projects came from a place of fear. You know, it didn't come from a place of let's make a great product for people. Let's like really understand the things people struggle with in communicating with family and friends. Like that's really, really trying to create something wonderful. It came from a place of fear and, and so during those times, I kind of, I learned, I think, how, how not to lead in places.

• 12:15Paul Adams?

  And by the, I should say, you know, at the time in Google, there was other things happening that were amazing, like Google were building Google Maps. Incredible product, one of my favorite products, I think one of the best products ever made. They built, were building Android, you know, I was kind of in, I was in the mobile team, in the mobile apps team at the time Android came out. So I can make incredibly good product. So I just happened to be in the social side, which was in this cut. And yeah, we, um.

• 12:39Paul Adams?

  Google Buzz is kind of a privacy disaster and Google Plus similar. And so kind of halfway through, I kind of published research about groups and how I've done, I've done a ton of research. An interesting kind of side note there is at the time I was being asked, I was working in the research, in the UX team as a researcher, I was being asked to do a lot of tactical research, like usability study type stuff, like can people use these products? And I ended up doing a lot of formative research as well in the same session. So I'd kind of say to the team like, hey, I'll do the research, I'll answer your questions, but also I'm going to do this other thing and I'm going to take 20 minutes doing that. And so what we used to do is, what I used to do with people was map out their social network, all the people in it, their family, their friends, how they communicate.

• 13:23Paul Adams?

  We'd map on all the channels, we'd talk about what worked well, what didn't. And we did this with dozens and dozens of people over like the course of maybe 18 months. And the same pattern emerged every single time, which was people need way better ways to communicate with small groups of family and friends. And I kind of look back now and go like WhatsApp, you know, or like it may be like iMessage if everyone's on Apple, but like really obvious in hindsight, but at the time not obvious. And so we kind of tried to build a product around that called Google Plus. But again, it was kind of motivated from the wrong, came from the wrong place. And so halfway through the research that I'd kind of done, all this research had been made public through a conference talk and Zook and Facebook noticed, got in touch.

• 14:07Paul Adams?

  One thing led to another and I left and joined Facebook, which was an amazing thing for me personally. Facebook was amazing, an amazing place at the time and exciting. And they were trying to do things for the other reasons, the kind of good reasons, like hey, let's build an amazing product for people. And this was during Google Plus being built, you basically shifted. Yeah, midway. It hadn't, I'm stressed to even tell you about it. The project hadn't been launched, it was still under wraps, you know, it was highly confidential. Google had done a lot of things at the time that were the first for them. I don't know if they've done them since, but things like everyone worked on Google Plus was sent to a different building, that building had a different key card. If you didn't work on Google Plus, you could not get in. All sorts of like kind of countercultural things at the time.

• 14:51Paul Adams?

  And as a result, there was a lot of, you know, antagonism internally for Google Plus. And so when I left in the middle of the project, kind of leaving with all of the plans in my head to the enemy. You know, some people saw me as a traitor, understandably. Other people thought I was enlightened, you know, fancy to talk to. But it was, um, I was like, it was the right thing for me to do, but at the time, you know, it was, um, was a hard thing to do.

• 15:17Lenny Rachitsky?

  I know there's also like a lot of scrutiny in what you took with you and the process.

• 15:22Paul Adams?

  Uh, yeah, when I left, uh, Google kind of assumed that I was one of the spies. You know, I was quarantined. I told them I was leaving. Uh, they, uh, you know, forensically analyzed my laptop, like all sorts of stuff like that. So it was pretty intense. Um, you know, looking back, I can understand why that happened. But the root cause for me is that the project has been run from a place of fear, competitive fear, which I don't think leads to good things.

• 15:53Lenny Rachitsky?

  So one of the themes through the stories you just shared is, let's say, failure is, is, I don't want to make it that harsh, but just things not working out. And I'm curious, as a product leader, how important you think that is for people to go through? If you think that's something that is almost a good thing. And I guess just is there anything there that you find helpful as a, as a coach, as a mentor, as someone? Two people that are trying to become basically you.

• 16:18Paul Adams?

  You're very, very, uh, it still is, it still is, you know, like, um. I've personally failed so many times, you know, like there are two stories and the Google one is like long, deep, deep tentacles. Like there are two, there are two stories. I, I've failed a ton of times, like at Intercom. I remember like, you know, when I was at Facebook, I was very happy and, um, Owen and I knew Owen and Des, who are the co-founders of Intercom. And, uh, they were trying to persuade me to join Intercom. We were like, it was like 10 person company at the time. But Owen said something, something to me at that time, which, uh, has stuck with me ever since. He said, you know, at, at Facebook, you can, you can design the product, but at Intercom, you can design the company. And that was extremely appealing to me, like a great pitch. He's like, just design the company with us that you want to work in.

• 17:03Paul Adams?

  And so the, and so part of that was a company that embraces failure, that says it's okay to try things. I'm a big believer in like big bets, you know, higher risk, higher reward. I, I, I don't get as excited about incremental things. No, I haven't said that. There's of course a place for that too, especially as companies get bigger. But I, I, I get excited about like bigger, big bets. And if you make big bets, you're going to get a lot of it wrong. So a lot of the principles that we built here at Intercom around building software, like we have a principle called ship to learn. And our, we've actually changed it since, still over on the wall here. Ship fast, ship early, ship often is what it says now. You say ship to learn, ship fast, ship early, ship often. It's like in that idea is the idea of failure.

• 17:45Paul Adams?

  You know, you're going to, it's not going to go right and it's going to go wrong more often than not. But if you ship early and fast and learn fast, you can change fast and you can improve fast. And that's kind of how we, that's the kind of culture that, that we as much as possible try to embrace and teach people. But it's much easier said than done.

• 18:04Lenny Rachitsky?

  Especially when you're in the moment, like, god damn it, everything's going to fall apart. I really, I really messed this one up.

• 18:09Paul Adams?

  Yeah. And there's a trade off with, with quality that people really struggle with. Like, you know, we, we have high standards of ourselves. A lot of Intercom comes from a kind of design founder background. We, we value the craft a lot. We never want to be embarrassed by what we ship. So there's a real tension there, a real trade off where people have these high standards, which we encourage. And we encourage to ship fast and learn and make mistakes. It's a constant kind of tension that we're navigating.

• 18:38Lenny Rachitsky?

  Next up, we have Tom Conrad, who is Chief Product Officer at Quibi and Engineering Leader at Pets.com. Two of the most memorable failures in product history, sharing his lessons from those wild experiences. Tom is currently CEO of Zero Longevity Science, which is a killer business and an app. In case you haven't come across it, I'd definitely check it out. Here's Tom. You brought up this phrase of notable disasters and I want to talk about that. You worked at two of the most famous notable disasters of product companies, Pets.com and Quibi. I think it's really rare someone sees the inside of so much hype and then such a fall at a company. And so I just want to spend some time in these two areas.

• 19:22Lenny Rachitsky?

  And maybe the way to set it up is just what's a lesson you took away from each of these two experiences that you've taken with you to future work and maybe advice you share with people?

• 19:30Speaker 03

  Probably the biggest lesson, it's not really about like the specifics of the business. The biggest lesson really is these things make you better. They, in some instances, actually I think in both instances, they became kind of dominos that, that opened doors for me in my own ambition and my own sort of, you know, professional life that maybe just wouldn't have opened at all if it, if I hadn't gone to those companies and learned those things and had those experiences. And frankly, even in the case of Pets.com, like even the high profile nature of it, I could have worked at, I could have worked at, you know, one of a thousand ecommerce websites in 1999. And when I went on to some subsequent job interview or something and talked about my experience, people were like, I would never heard of the thing that you worked on.

• 20:15Speaker 03

  But everybody certainly heard about Pets.com. It's a pretty funny example too of how some struggles are like timeless, you know, that was 20, 23, 24 years ago now. And while as a leadership team, we made, I'm sure, all kinds of mistakes. One of the things that happened was that there were three kind of overfunded pet ecommerce sites. And we all raised in excess of $50 million, which is a tremendous amount of money now, was a tremendous amount of money then. And we all thought it was a zero sum game and that we...

• 20:58Speaker 03

  As one player started to spend on promotion or to spend irrationally on national broadcast television advertising, we all did, and it became this kind of unwinnable arms race. So there was like, there is like, I think, a fundamental lesson about the like, the, you know, having an excess of, of investment can be its own albatross or, or, or lead you to take, make decisions that maybe would be unwise. And then, of course, it's just like, you know, timing is really important. You know, the, you know, Chewy is an online pet store worth $9 billion today. They were a private company and bought by PetSmart and then spun back out.

• 21:42Speaker 03

  But when they were bought by PetSmart, they were acquired for $3 billion. You know, biggest e-commerce acquisition of all time. And while I think it's probably unfair to compare Chewy, who executed exceptionally well over a decade, grew their business brick by brick and turned it into something really remarkable, to Pets.com, which was, you know, in a very, very different moment in time and, and tried to go to market in a really different way. The critique that is often leveled at Pets.com, or at least at the time, was like, this is just a stupid business. They're shipping dog food around. You could never make that work. And like, that, that's just wrong. Like, you absolutely can't make it work. Probably can't make it work when 80% of the country on the internet is still on dial-up. It's really, really early.

• 22:25Lenny Rachitsky?

  I saw a stat, I think you shared somewhere, that you took Pets.com from nothing to a public company to completely out of business in 19 months.

• 22:33Speaker 03

  Yeah, yeah, that's about right. You know, the, the, the other thing that's forgotten in the tale is that we, we actually didn't go bankrupt. We shut the company down and returned the, the remaining balance to the investors, which no public company had ever done before. And the leadership team just like reached the conclusion that, that given the way market conditions had evolved, there was just no way we were going to be able to get our capital into the company. And, and it was a company that required, you know, additional investment to get to profitability. And, and so it was better to sort of wind down early, take the money that we, you know, we had in the bank and get it back to investors than to just spend every last penny

• 23:16Speaker 03

  on like a, you know, what was sort of a fruitless attempt to salvage it.

• 23:21Lenny Rachitsky?

  Did not know that. Let's talk about Quibi. What went wrong there? Do you think there was a path to Quibi? Having worked out, any big lessons that you took away from that experience that you bring with you?

• 23:32Speaker 03

  The kind of miraculous thing about Quibi for me was it relit my enthusiasm for the industry for doing this work. I had left in, I think it was December of 2018. And I thought that maybe I was just done making software. I had done it for a really long time and after like 25 years or something. And I had changed a lot. The industry had changed a lot. And I thought maybe I just didn't have the same passion for it that I had, you know, a decade before. It also seemed like maybe it would be fun to have like another chapter of my life. That was just completely different.

• 24:15Speaker 03

  And I had a whole list of things that I thought I might want to do. I mean, they were really, they're kind of ridiculous. It's like, maybe I want to be a pastry chef. Maybe I want to be a landscape photographer. Maybe I want to, you know, learn to make bad music to put up on SoundCloud or something. And really, the only thing they had in common were they were all things that I knew nothing about. People would be like, oh, you think you might want to be a pastry chef? Do you like to bake? And I'd be like, no, I don't know anything about baking. You know, oh, you think you might have landscape photography? Do you take photos? No, I don't make photos. But I was kind of committed to the bit, actually to the point where when, when TechCrunch interviewed me about my departure from Snapchat, I was like, you know, I'm out. I'm going to do something else entirely. So that story is very much out there.

• 24:55Speaker 03

  But a few months after my last day at Snap, I got a call from Meg Whitman and Jeffrey Katzenberg, who were starting up, it was called New TV at the time. And, you know, the pitch was, we're going to try to take the best of Movable and Silicon Valley and consumer tech and sort of weld it to the best of sort of Hollywood style content production to build something like completely bespoke and purpose made for consumption on the phone.

• 25:28Speaker 03

  They were looking for both technology leadership and product leadership and wanted to know if I was interested in one or both. And I took the meeting, even though I wasn't really taking these kinds of calls from anybody, it just seemed like who's going to pass up the opportunity to have lunch with the two of them. So I listened to the pitch and politely declined and told them that I was going to be like a pastry chef or something. And we kept doing that every couple of months for like seven months. We'd go to lunch, they would give me an update on the progress they're making and I would decline, you know, the invitation to get involved somehow.

• 26:05Speaker 03

  And then late in that year, I went to lunch one more time and Meg explained that they brought on someone to lead technology and they brought another person to lead product. And both of them, for really, truly for reasons that are completely disconnected from, from Quibi itself, both of them had left after about six weeks.

• 26:29Speaker 03

  And Meg's like, we've raised all this money and we've told the world that we're shipping this product in about a year. We got an awful lot to do and I really could use some help and I would like consider it a personal favor if you would become, spend just a couple of days a week helping. You know, she's like, I'll continue to look for someone who actually wants the job, but it would really be, it'd be, you know, great if you could help me get this off the ground. And my wife is a freelance writer, marketing strategist and loves her life as a freelance contributor. And she's like, you should do this. Like, why not? It's two days a week. It's just a few months. What's the worst thing that could happen? Like, maybe, maybe you'll like it.

• 27:11Speaker 03

  And, and I'm like, no, no, the worst, here's the thing that will happen. Like, I won't do it two days a week. It will like immediately be three days, then four days, then five days, then six days. Like, I just know myself. And she's like, no. She's like, you know, just on Wednesday night at six o'clock, close your Quibi laptop and be like, all they're paying me for is for Tuesday and Wednesday, and then open it back up on Tuesday morning. Like, that's all you've got to do. Well. You know, she's right about most things, and she's wrong about this. I fell deeply into it right away. And it was just so fun to get to build a team from scratch and to design and build a product from scratch and to take advantage of all of the sort of modern software

• 27:54Speaker 03

  architecture stuff that had come into being over the course of the 15 years since we had started Pandora. And, you know, I'm embarrassed about some of what happened with Quibi, for sure. But I'm super grateful for the experience because I just really fell in love with the industry again and was reminded of just how rewarding it can be to build something and to try to put it out there, even if you stumble pretty mightily along the way.

• 28:28Lenny Rachitsky?

  Is there something that you took away from that experience that taught you what to try to avoid, what to try to pull towards?

• 28:36Speaker 03

  I think I sort of misunderstood or misjudged companies sometimes by thinking about them, like really focused on the product execution. You know, kind of if you find an interesting problem that people, you know, people care about, and you solve that problem in a really beautiful, elegant, delightful way, that's 10 times better than anything else that they can get in that same space. They'll tell their friends and like all the rest will sort of take care of itself. And so that was always my ambition. Find a thing that I cared about building, do a great job building it a really delightful way, go really deep on listening to people and their feedback and iterate your way

• 29:18Speaker 03

  to success and sort of breaking through that membrane that we all strive to get across, you know, kind of really great word of mouth. But I think the thing I've come to better appreciate is that

• 29:33Speaker 03

  Companies are also kind of like a, they're kind of a math problem that describes how you take, you know, investment and pour them into the equation and out the other side comes returns on some time horizon. And yes, there are variables in that equation that are influenced by the product that you build and all of the little details and decisions that you make about making that product great. But if the equation is fundamentally broken or a big swing in and of itself, no amount of like iteration and execution can like kind of get you out of the failed outputs of the broken equation. And I think, you

• 30:16Speaker 03

  know, Quibi made a bet that you could build an entirely bespoke content library that was sufficiently scaled to get people to subscribe and retain for a couple billion dollars. I mean, it was a huge amount of money. But, you know, we made, we made 70 shows in 18 months, which is like more content than all of the major broadcast networks combined made in a single year. So it's a pretty major accomplishment. And we made a bet that we would augment those sort of episodic and serialized or Hollywood style shows with

• 31:01Speaker 03

  a bunch of daily content that we produce at the level of like network television, nightly news and so forth, that would be an alternative to some of the sort of daily content that you might otherwise get on YouTube. And that was going to be about a third of the content spend. One like super interesting thing that no one talks about is that all of that content was designed to be made like day of or day before it aired. So there was no back catalog of it. And it was all designed to be shot in these professional studios that we built out. And it was really expensive. Like I said, it was like a third of the of the investment we were going to make in content, maybe half, almost half the investment we were making

• 31:43Speaker 03

  content. And we launched two weeks into COVID and we couldn't make any of that content, except literally in the garages of the host's homes. And so we had this thing that was supposed to seem like really set apart from YouTube that literally now was being made exactly like YouTube content, which is sort of like self-produced at home with, you know, very little sort of the support infrastructure of Hollywood. Now, you can argue, I mean, I think the content on YouTube is really, really exceptional in this category. And maybe we were never going to do better than that. But I think what was really fundamentally broken with Quibi was that the actual foundational equation of can you make enough premium content

• 32:28Speaker 03

  that's totally bespoke and made for the service and takes advantage of the unique nature of the phone? Is that enough content to get people to sign up and retain? And can you do that for a couple billion dollars? And I think the answer is no. The library has to be much, much bigger. And you have to have like any company, you have to have sufficient time and energy to iterate on the content format itself because we were really, our roadmap really wanted to innovate on the content format. And so I think part of what happened is pretty quickly it became clear that the math was just wrong. It wasn't going to take 2 billion. It was going to take 6 or 8 or 10 billion. And the risk reward

• 33:12Speaker 03

  profile of betting 10 billion on the format was just more than than anyone can stomach.

• 33:20Lenny Rachitsky?

  Next up, we've got Sri Batchu, former head of growth at Ramp, who shares something that I've thought about ever since we had this conversation, which is this idea that when you fail, make sure you fail conclusively. To make this failure an actual learning that you can build off of versus just a waste of time. Here's Sri.

• 33:38Speaker 04

  Growth experiments in my history are typically like thirty percentish success rate. So the vast majority of things that you try don't work. And so you want to create a culture where people aren't afraid to take risks and aren't afraid to fail. And for me, failure is not that you didn't drive revenue. Failure is not learning. Like, so it's really important that you learn when you fail. And so we celebrate failure as long as you're learning. And you can only learn if you've designed the right test and you failed conclusively. Because otherwise, I think many of us have been in situations where there's intuition that something might work, and it doesn't work. And then you end up doing it over and over for years.

• 34:20Speaker 04

  Like, because every time you execute it where somebody else has the same idea, you try it again. And it's because you haven't been able to design the test to fail conclusively. And it's hard to do, but at the end of the day, there's only two ways to, you know, make an experiment successful. Either you have a very large N, or you have a very significant treatment, which is like, you know, what you're doing in the experiment itself. And, you know, in B two B, you don't usually have the luxury of a large N, which you do in consumer, right? Facebook can get stats taken in two hours. You know, a B two B company could take two years to get to the same number of, you know, touchpoints.

• 35:02Speaker 04

  And so to counteract that, I recommend people like just trying to maximize the treatment effect, which is like, if you have a hypothesis that you're testing, just throw all of the possible tactics and resources that you think would move that needle. Because you can always cost rationalize later if it works, right? And so maximize the treatment effect. And if with all of that, it didn't work, then you can say, hey, like, we're not going to try this again. Because we literally did try everything that we could to test this hypothesis. And if it doesn't work in the best version, and it's expensive as it is, this is not worth spending more time on. But if it does work, great, then you, you know, do another version of the test with like half the tactics,

• 35:45Speaker 04

  or whichever tactics you think work better or worse, and you optimize over time.

• 35:49Lenny Rachitsky?

  Is there an example you could share when you did that?

• 35:52Speaker 04

  I mean, account-based marketing is something that is very common in enterprise software, right, where you selected, you know, certain customers that you think are high priority. And you're saying, I want to touch them in as many nuanced ways as possible to see if that drives conversion. And this is something, you know, I've seen tried many times where people do it, but they kind of, you know, do it halfway, where they're like, okay, we tried these three things, conversion of the control group, like, wasn't higher. And so we're, you know, we think it's, does not go work. And then a new go-to-market executive comes and they have to do it again, they have to

• 36:37Speaker 04

  do it again, they have to do it again. It's like a very common one wherever this happens. And, and so when we did it at Ramp, we, we did exactly, you know, what I just described, which is like, let's really be thoughtful about the experiment design, both in terms of like maximizing the number of people as well as, you know, maximizing the number of ways and types of ways that we're effectively touching these target customers to show the value.

• 37:03Lenny Rachitsky?

  So what it sounds like is the hypothesis isn't like this email will have a big impact on conversion. It's like this strategy of coming after customers is what we're testing?

• 37:15Speaker 04

  That's the example there, right? Like, and I think like, if you, for example, if you have the, it's, this kind of framework is more important for cross-functional, larger scale, bigger tests, rather than like, you know, an email modification, right? But we could even use it on a micro example, like an email modification, right? Where you are like, okay, I think this particular email is underperforming because it's not talking to this part of the customer's pain point or journey. And you could just, the simplest test would be, okay, let me make some tweaks and to the text and edit that. And that could be the end of that test. And if that doesn't work, you're like, oh, maybe those weren't the right text edits. Let me do a different

• 38:00Speaker 04

  text edits or whatever. And that's fine. That's low cost. It's not the end of the world and it's for you to be wrong there. But an alternative that you could do is like, oh, what are all of the things that I could change about this email in the same test? Is it the trigger of the email? Is it the text content of the email? Is it, you know, additional personalization? Is it the design of the email? Like trying to think about what are all of the various levers that you think could be wrong and put them all together to test your hypothesis of this touch point is wrong and how do I improve that?

• 38:34Lenny Rachitsky?

  Well, obviously, the downside of that is you don't, if it doesn't work, you don't know if it's like, oh, maybe it was, this thing could have worked in the subject.

• 38:40Speaker 04

  Yeah. So there's always trade offs on this. And so, but what you're hoping is like, you've done a complete refresh where you did all the things that you thought were intuitive, that should work. And if it doesn't work, then you're like, okay, maybe my hypothesis is wrong. But you're right, there's always going to be a challenge of maybe the execution is wrong. And I did too many things potentially in that case.

• 39:00Lenny Rachitsky?

  Our next story is from Jay Z, who is a colleague of mine at Airbnb, head of product at Webflow when we recorded this episode. And this is her sharing the story of one of the biggest product misses at Airbnb. You've seen a lot of new PMs and you've seen these PMs succeed, you've seen some fail. What are the most common mistakes that you find new PMs make in this experience of kind of helping new PMs get into the field?

• 39:26Jiaona Zhang?

  I think something that is really hard to untrain, but I think every human does it, is you jump to solutions. And so one of the biggest things I see, not just in my course, but also just as a PM and like some of the mistakes that you make as a PM, is the idea of you get really attached to a solution, a way of implementing something, something that you can see in your head that you want to build. And so that's the first thing I really want to like unteach in our course. And so a lot of people will literally come in, they'll be like, I want to build X startup, or I want to do this thing, or I'm in blank school and I've been doing a lot of research on this particular area. And so like untraining that and being like, hey, we're going to go out there, we are not going to think at all about the thing that you want to build. But instead, we're going to be focused on Users and people like people in the real world and their problems.

• 40:11Jiaona Zhang?

  And the first step is to understand their problems and then to understand if there's an opportunity here as opposed to, hey, you want to build X thing for Y person. So that's the biggest mistake that you really have to unteach and like retrain thinking around.

• 40:25Lenny Rachitsky?

  So let's kind of go to the other side of this question. We talked about what mistakes new teams make. I'm curious, what's the biggest product mistake that you've made?

• 40:34Jiaona Zhang?

  Wow, that's a good one. It's so interesting. I feel like as product people, we're always making mistakes and we're always learning. Maybe I'll give an example from Airbnb, since you and I were both there. And this one does stand out to me. So, you know, we were working on this concept called Airbnb Plus. You know, if you took a step back, what we're really trying to do is to be like, hey, not everyone trusts Airbnb in terms of, you know, it's a platform. It's not like it's managed inventory. It's not a hotel. How do you go in and really make sure that we're like, all the Airbnbs are meeting the quality bar? But I do think we were very solution first and I think we're also competitor afraid at the time, right? So it was during a time where there were managed marketplaces. There were the Saunders out there. And I think that as a company, we're very much like, oh goodness, like what's, what are we going to do in the world of managed marketplaces?

• 41:19Jiaona Zhang?

  And so we went really hard down the solution space. We essentially were like, let's go inspect our inventory. Let's actually try to manage our inventory more. And really what we should have done is taken a step back and be like, what's the real problem? The real problem is people want to know what they're getting themselves into. We need to represent the homes a lot better. And I think the other piece here that's really important is what as a company is your strategic strength? And like, what's in your wheelhouse? So for example, Airbnb, we weren't that strong in operations. We again, we're this platform with this marketplace, right? And so if you don't have that muscle and then you're asking the company, the team to essentially build it from the ground up, that's really, really difficult. Not to mention the unit economics. Are the unit economics actually going to work even as you scale?

• 42:03Lenny Rachitsky?

  Yeah, I feel like Airbnb Plus is an untold story that somebody should tell and that could be its own podcast, I guess.

• 42:09Jiaona Zhang?

  You and I can tell it.

• 42:10Lenny Rachitsky?

  We could tell this could be Airbnb Plus, the hidden story. As you said, the problem is trying to solve was people don't really trust. They don't want to even consider Airbnb because it's like, no, I don't want to stay in someone's home. I don't know what it'll be. It's unpredictable. And so as an outsider, it felt like a really clever approach. We're going to vet them. We're going to make sure they're awesome. There's a minimum bar. I think the, and I guess this is the question is, do you think it was just like, this is never possible because they'll, we'll never make money as a business doing this because we don't make that much per booking and investing time, resources, sending people pillows, all that stuff is ever going to be economical? Or do you think there was a path and it was just not executed well?

• 42:51Jiaona Zhang?

  I think there wasn't really a clear path. I think it was less about execution. Exactly. And it's more just like, if you understood, again, this is what my point around unit economics. There are things where I think you have like magical thinking around unit economics. You're like, well, when we get to the scale of X, it's all going to work out. We can make these things happen. I think you actually need to really make sure the unit economics work quite at the beginning. So that is definitely one lesson. And I think the other thing is, and going back to spirit of what are you trying to achieve? If you're trying to achieve this idea of like really knowing the quality of the place and, and for a platform like Airbnb, the right way to go about doing it is through our reviews, right? Through our guest reviews, which are essentially free as opposed to literally sending out inspectors.

• 43:32Jiaona Zhang?

  And I think the other things are, if you can get signal on what are the things around quality that people care about? Is it cleaner, cleaning? Is it, is it the, hey, I'm locked out? And I think that there are other solutions besides inspection that then get at that. So for example, it is actually cheaper to go send everyone a lockbox than to deploy an inspector and go look at your property, right? It is actually cheaper to maybe do a partnership with like a, you know, a bunch of cleaners in different local areas and then get that as part of the feat, like as opposed to doing the inspection. So again, it's really about what are you really trying to achieve? What is the user problem in each of these areas? And can you target that problem with the particular listing that is, you know,

• 44:13Jiaona Zhang?

  that you're looking at? And so I, yeah, I personally don't believe the unit economics ever would have really worked out. I think we should have known that. Or we should have dug into that more at the very beginning and then to get very tailored instead of like one blunt instrument to solve it all. Hey, we're going to go inspect. It's like, what is the problem for this listing and what's the best solution to fix that problem?

• 44:32Lenny Rachitsky?

  Our second to last story is from Gina Gotthilf, who is an early growth leader at Duolingo. She's currently COO of Latitude. And this is her sharing a wide-ranging and important point about how everyone has both an A-side and a B-side to their career. And people often only share their A-side. So this is Gina sharing her B-side.

• 44:53Speaker 06

  We are very encouraged in our lives, especially professionally, to talk about our A-sides all the time because that's what impresses people, that's what opens doors, that's what allows us to keep growing, and it's so important. So it means that a lot of what you hear in podcasts and onstage ends up being the Instagram of both version of someone or a company or a country's trajectory. It's just the highlights. You know, and when I talk about my A-side, it's very impressive. You know, I did things like, we'll talk about, I met President Obama. I worked on the Mike Bloomberg presidential campaign. I helped Duolingo scale from 300 to 200 million users. I worked with Tumblr, helping them scale Latin America. Andreessen Horowitz invested in my company, et cetera. But between all of those highlights, there were so many B-moments

• 45:38Speaker 06

  that get sort of like shoved under the rug because it's just easier for me and it's more impressive for others. But I really like to highlight those because I think that most of us have a lot of B-moments every day, every week, every month, and every period of our lives. And it's easy to think that things aren't just not going to work out for us because we're in one of those B-moments if we don't recognize them as moments.

• 45:59Lenny Rachitsky?

  I love this concept. We're going to talk, as you expected, about a lot of your A-side stuff. Is there any example of a B-side story of your life that would be interesting to share?

• 46:09Speaker 06

  Look, I think those are the most interesting because they're funny or ridiculous, you know? I had a lot of B-sides and I still do. For example, I, you know, I had no idea what I wanted to do. I actually wanted to be, I thought I wanted to be an actress. I either wanted to be that person in SeaWorld who goes like this with the dolphin. This is before SeaWorld was canceled. Or I wanted to be an actress. I applied to schools. I didn't get into any Ivy League. I didn't get into any of the top schools I wanted to go to. When I got to college, I actually ended up dropping out because I got so depressed, like incredibly depressed, couldn't get out of bed depressed. Ironically, I dropped out of Reed College, which is the same college that Steve Jobs dropped out of. So, you know, I was just destined for greatness. I knew it at that moment.

• 46:53Lenny Rachitsky?

  It all makes sense looking backwards, as he said.

• 46:55Speaker 06

  Totally. I was dropping out, being like, yes, this is exactly the path. No, I was miserable. I thought there was no path forward. And when I finally went back and graduated, the college counselor looked at my curriculum and said, what have you even done with your life? There's nothing to show for. And it was shocking because I was always like the overachiever who wanted to do the maximum curriculum and ace all of my classes and do whatever, you know, like I did three diplomas in high school, the internationally American, the Brazilian. And so that for me, I think my one learning there that has stuck with me, and I think can can work for other people too, is that it's not just about doing things that actually matter and like and and learning.

• 47:34Speaker 06

  It's about being able to tell the story and it's about understanding what other people perceive as valuable. I applied to a hundred companies. I didn't hear back from most of them. I finally got an internship at kind of like a tier B slash C digital marketing agency in New York City because I wanted to live in New York so badly. And they forgot to apply for my visa on time, so I lost my visa and had to go back to Brazil. And then I ended up leaving that organization to go work for another one. And like, I won't even go into the details of the the shadiness of that company that I worked for, but then they ended up laying me off, so I lost my visa again, had to go back home,

• 48:14Speaker 06

  found another opportunity, got fired that time. So there's just a lot of rockiness in my start that I don't think you would imagine when you see someone up on stage like leading a conference for 5,000 people that I think is important. And even when I started working for Tumblr, I was like, this is it. Like, I made it. This is a really interesting company. This is gonna work out. That was super rocky because it was an early stage startup. So for example, you know, they couldn't figure out how to wire money to Brazil. So I was not paid for six months. And at one point, me and like my colleagues were trying to get money out of the teller to pay contractors because we had no money to pay them. And like, we borrowed money from people. And and finally, they also laid me off because they decided to sell to Yahoo. And then I had to figure out like, what am I gonna do? No one's gonna hire me. I've been fired and laid off so many times.

• 48:59Speaker 06

  So this is all before I started like an agency to help US-based tech companies and startups grow in Latin America because I figured I was in like this really great place to make that happen. And it eventually worked for well-known companies such as Duolingo. At the time, they weren't well-known. They were a tiny little startup. They didn't have an Android app. And that's how I started working with Duolingo because Their head of marketing connected with someone they had worked with at Flickr and said, "I noticed Tumblr grew a lot in Brazil last year. Can you recommend like a company or an agency to help?" And they said, "This girl." And I was twenty-six. And so that's how they connected me with Duolingo. And I started helping them grow in Brazil as a consultant. They were like, "This is great. Can you help us grow in Chile, Argentina?" And I was like, "Yes." They were like, "How about Mexico?" And I was like, "Yes."

• 49:44Speaker 06

  You know, "Did I know anything about these places, Lenny? That I know people there?" "No." But like, "You can figure it out." And then they ended up asking me to come on full-time, do that across the world, Japan, China, Korea, Turkey, Spain, France, et cetera. And then to own growth, which ended up meaning communications, social media, government partnerships, anything to grow. And then eventually it became an A/B testing growth engine with engineers and PMs and designers, of which I knew nothing about. And, you know, even after that, like, I left Duolingo five years later, didn't know what to do with my life. You'd think, "Oh, wow, you have it figured out. Now you left Duolingo, you have the world in front of you." And I'm like, "Maybe I can finally go work for nonprofits,

• 50:28Speaker 06

  which is what I actually wanted to do in the first place." Tried to handle that, like had a couple of experiences before going to work for the Mike Bloomberg campaign. Working for the Mike Bloomberg campaign is impressive, but you know what? Mike Bloomberg didn't win. He's not the president. So that was not a successful campaign. You know, if you really look at it. And yeah, latitude seems like it's a really promising path. But there's A-days and B-days. So it's just a lot of that and just staying resilient and believing in yourself and getting back on the horse when you fall on your face.

• 51:00Lenny Rachitsky?

  Amazing. That's such an important message. I think one of the threads from what you're describing, something that I think about a lot is people kind of underestimate how long their career is. There's just so much time to do stuff and for things to start to work. This is gonna sound really fancy, but I think Marcus Aurelius has this quote about how "Our life is actually very long. We just use it really badly. And we just waste a lot of our time."

• 51:20Speaker 06

  I think you're so right, Lenny. And I love that because people are going around being like, "Life is short, life is short." But that's so true. We waste so much time. But also, I think we don't recognize how much opportunity we have in front of us. And as a twenty-six-year-old, I definitely thought my career was over. You know, I was like, "I blew it." And like, looking back, it's funny, you know?

• 51:40Lenny Rachitsky?

  Yeah, I know exactly what you mean. Like, I spent nine years at my first job at a random company in San Diego in a startup. And I was like, "What am I doing here? So long." And then it turned out that was really useful for the thing I did next. And then eventually, wow, things started to really take off. So I think that's a really good lesson for people. Just for a long time, like this is my fourth career. Like I've switched careers many times. I was a engineer, then I was a founder, then I was a product manager and I'm whatever this is, whatever you call this thing.

• 52:06Speaker 06

  I guess me too. I was, I was an operator. I was a consultant. Well, I was an employee. I was a consultant. Then I was an operator, which is a fancy way of saying employee at a startup. And then now I'm a founder and a VC and an angel and whatever this is.

• 52:22Lenny Rachitsky?

  Awesome. Awesome. So I think, I think that's a really important takeaway. It's just, there's a lot of time to do stuff and don't stress if things aren't moving as fast as you want. I'm curious, what's like a mess up or a big mistake maybe that you made or your teammate that was like, oh, wow, that was a big waste of time?

• 52:38Speaker 06

  Yeah, look, a lot of things didn't work out. More than 50% of our, of our A B tests didn't work out. We made bets that didn't make sense. I will say though, that like in the spirit of A and B sides, I, and I think in general, we are really good at forgetting the B stuff. Like I, I talk so much about all the stuff that worked, that it's hard to remember, like all of those moments that didn't actually work. And the, and the moment, and the thing that I'm, I tend to talk about, which is like this mistake that we made as the growth team. Is almost like one of those, like when you get asked in an interview, like what's your, what's like your biggest weakness? And you're like, I'm a perfectionist, you know, it's like one of those things that actually makes you sound good. Because it's the story about how my team really wanted to implement badges.

• 53:20Speaker 06

  We spent a lot of time playing all the games that were popular at the time, trying to understand how those gamification growth hacks that we could find in, in those apps would potentially overlay onto Duolingo and how, you know, we would do that. And badges was just pervasive in all of the top games. And so it seemed like a no brainer. But since we ranked all of our experiments in terms of ROI and return being like, how many users we think we're going to get from this, the AUs and in that time investments, it was never, it never made sense to focus on this because we thought that the time sync would be too high. So I actually ended up not letting the team run this experiment for like six months so that we focus on lower hanging fruit. So that's like a mistake on my end.

• 54:04Speaker 06

  Then we decided to, we decided to run this experiment in the most lean way possible. We're like, you know what, like there's MVPs, there's like minimum viable experiments. We don't have to run a whole badges thing. We can just like do something more simple and actually see if that like leads to growth in an interesting way, and then we'll know. And we ran this very simple experiment that was like, you signed up and then you get a badge. And it was like this girl with a balloon, I don't know, she was like happy or whatever. And of course, of course, in retrospect, it led to no results because no one is proud of signing up. It's not an exciting moment and you don't even have badges to collect. You can't show it to other people. Like none of the things that make badges compelling were there. But we were like, okay, well, we decided it didn't work and then we moved on. So we moved on for like another, I don't know, eight months and we didn't look back.

• 54:47Speaker 06

  And then when we did look back, first of all, at that point, we, we discovered that we hadn't been dog fooding, okay, which also was embarrassing looking back. We hadn't been dog fooding in the growth team. We'd just come up with hypotheses. We'd like, we were super careful about prioritizing them and making sure that we were doing the best possible like write-ups and all these things. But like the dog fooding piece, it just, I didn't come from a product background. I was a marketer and I hadn't even, I didn't really understand the term dog fooding. But when I, when we thought, we had a conversation, we were like, you know what? If we had just tested that, we would have all known that this was a super lame badge. And I was like, why are we not testing our experiments? And so like that became part of our, of our practice. It's still relevant. I, I just had a conversation yesterday with engineers at Latitude.

• 55:30Speaker 06

  I, I haven't explained what we built yet, where we're building it, maybe we'll get there. But I was talking yesterday to engineers at Latitude and they're awesome. Like in terms of our product team, we have like the number eight employee at NewBank. You might've heard of NewBank, but it's this like massive banking, like fintech in Latin America. And, and we have people from like other fintechs and then we have this guy who was the lead PM at Twilio. And I was explaining to them why we should be dog fooding and they were all like, oh yeah, we should dog food. It's just like easy to forget stuff like that. So, so that was a mistake that, you know, we could have probably gotten to the growth that we got to with badges much earlier, earlier on. And not only did we get to growth with badges, but it became like this amazing treasure trove of opportunity because once you have badges and people want them, you can now ask people to do anything.

• 56:14Speaker 06

  Like go find friends, go buy things, like, you know, whatever it is. And so like we, we had, we impacted almost all metrics across the company positively, but including some we hadn't expected. But it's like easy to talk about a mistake that ended up being a win. So that's why I compared it to the interview thing in the beginning. But you know, we, we tried making Duolingo a social app really early on and failed. Um, it was called Du- Duels, dual duels, like you could duel, you know?

• 56:42Lenny Rachitsky?

  Very clever.

• 56:43Speaker 06

  Yeah, I know, but we were, we were clever, but people didn't use it and we didn't figure out why. We, we tried making a Duolingo for schools platform, like we couldn't get it to pick up. I went and launched Duolingo in China and it got downloaded by a million people in the first day and then the app got blocked and I, we, you know, because of the government. And then we couldn't figure out what to do and then everyone rated the app like a one star because it didn't work. And so then we had a really like a lot of trouble actually recovering from that. We launched Duolingo in India and didn't realize because we couldn't have unless we went there, which we finally did. That most people set their phones, phone UI in India to English. Because typing in Hindi is hard and of course, there's a lot of languages throughout India.

• 57:26Speaker 06

  And we were making it so that when you downloaded Duolingo, whatever UI you open your app, your, your phone was set to, we offered not that language for you to learn. That was your base language. So we were telling people, learn French, Spanish, German from English and they were all trying to learn English. So they didn't find what they were looking for and they left. Um, there were so many mistakes, you know, and luckily I think we were able to, to bounce back from most of them in terms of how Duolingo is doing today.

• 57:53Lenny Rachitsky?

  And our final story is from Maggie Crowley, VP of Product at Toast. And one of the most beloved episodes of the podcast. This is Maggie sharing something a little bit different. Her favorite interview question about failure and what it tells you about the person you're interviewing. Plus a story of her own product failure. Here's Maggie.

• 58:12Speaker 07

  A question I ask in every product interview is, uh, what's the worst product you've ever shipped? And that's because I, I don't think you're a good PM if you haven't shipped something that's really shitty. Like you just haven't had enough reps, you haven't done it enough time, like. And it, and it's not only that you've done it, but that you can admit it and you know which one it is. Like that's so important. And I remember.

• 58:35Speaker 07

  What? This is so dumb. I can't. I'm still so mad about this, that we did this. It was, I won't name which team, which company. I'm not gonna call that out, but we wanted, we decided we needed to do a rewrite. Red flag number one of a existing product and, you know, engineer who I'd worked with many times. We had a really good relationship. And this person was like, yeah, it's gonna take six months. No problem. Core part of the product, been around for forever. Like, one of those things that the code is still, is still the code written by the founders kind of thing. It didn't take six months. It took two and a half years. It still wasn't done. It almost never, like, it went on for so much longer than it should have. It took us forever to get to feature parity.

• 59:18Speaker 07

  It was the worst project. Like, so many people rotated in and out of it. Everyone thought it was dumb. Like, sunk cost fallacy. Just the worst. And it's because, A, we got arrogant and we thought we could do it. B, we skipped discovery. We didn't really write a one pager. We just went for it. We didn't do enough technical and design research into what the requirements would actually have to be.

• 59:44Speaker 07

  And there you have it.

• 59:45Lenny Rachitsky?

  And did not work out. Or was it a huge success in the end? And it changed the trajectory of the business?

• 59:50Speaker 07

  Absolutely not. But you know what? I didn't get fired. So like, it's fine.

• 59:54Lenny Rachitsky?

  I feel like I've gone through those experiences and then like three, four years later, it's like another, maybe this rewrite and redesign may work, because just, we haven't updated this thing in a long time.

• 1:00:04Speaker 07

  Just don't do it. Don't rewrite. If anyone ever tells you to do a rewrite, don't do it. A side by side rewrite? Nope.

• 1:00:11Lenny Rachitsky?

  Yeah, I've never had a, like, what I run into is once you get too far down a redesign slash rewrite, everyone's building in that new world and then you launch it and the experiment's negative and then it's just like, oh, we just gotta launch it. We're gonna claw back. We're gonna get, figure out how to get back to neutral someday.

• 1:00:25Speaker 07

  Yeah.

• 1:00:26Lenny Rachitsky?

  Yeah.

• 1:00:26Speaker 07

  Yeah, don't do it.

• 1:00:27Lenny Rachitsky?

  Good times. And that is a wrap. I hope you enjoyed these stories of failure. I want to give a huge special thank you to all of our amazing guests for being vulnerable and sharing these stories of failure in their career. I hope you leave this episode with a new perspective on how setbacks and challenges and failure can often be exactly what you need to get to the next step of your career or your life. If you've got a great story to tell about failure, I'd love to hear it. Leave a comment either on YouTube or on lennysnewsletter.com or just DM me on Twitter, or you could reach out at lennyrjitski.com and click the big contact button. Thank you for listening. Bye, everyone.