Humanizing product development | Adriel Frederick (Reddit, Lyft, Facebook)

Adriel Frederick is VP of Product Management at Reddit X, where he helps incubate and scale new products. He is a former Product Lead at Facebook, as well as a former PM and Director of Product at Lyft. In today’s episode, we focus on what it takes to become a better product leader. Adriel shares anecdotes from his time at Lyft and Facebook, insights about how to lead through tough times, why there isn’t an algorithmic solution to everything, why R&D teams need to be a part of the core mission, the tangible benefits of working on diverse teams, and his thoughts on the future of AI. He also introduces the concept of cannonballs, why you should focus on the marginal user, why organization and empathy are the most important PM skills, and so much more. Find the full transcript here: https://www.lennyspodcast.com/humanizing-product-development-adriel-frederick-reddit-lyft-facebook/#transcript — Where to find Adriel Frederick: • Twitter: https://twitter.com/drellf • LinkedIn: https://www.linkedin.com/in/adrielfrederick/ — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • Twitter: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — Thank you to our wonderful sponsors for making this episode possible: • Linear: https://linear.app/lenny • Flatfile: https://www.flatfile.com/lenny • Eppo: https://www.geteppo.com/ — Referenced: • Jules Walter on Twitter: https://twitter.com/julesdwalt • Jules Walter’s guest post on Lenny’s newsletter: https://www.lennysnewsletter.com/p/product-sense • Mark Zuckerberg on The Joe Rogan Experience: https://open.spotify.com/episode/51gxrAActH18RGhKNza598 • Sam Harris’s TED Talk on AI: https://www.youtube.com/watch?v=8nt3edWLgIg • Facebook’s 7 friends in 10 days: https://www.linkedin.com/pulse/how-chamath-palihapitya-dramatically-improved-user-malinda-senanayake/ • The Prize: The Epic Quest for Oil, Money & Power: https://www.amazon.com/Prize-Epic-Quest-Money-Power/dp/1439110123/ • The New Map: Energy, Climate, and the Clash of Nations: https://www.amazon.com/New-Map-Energy-Climate-Nations/dp/0143111159/ • Revisionist History podcast: https://podcasts.apple.com/us/podcast/revisionist-history/id1119389968 • Tuned In podcast: https://www.hpacademy.com/blog/tuned-in-high-performance-academy-podcast/ • Mo on Netflix: https://www.netflix.com/title/81134264 • Radiant Nuclear: https://www.radiantnuclear.com/ — In this episode, we cover: (00:00) Adriel’s background (06:13) What he does at Reddit X (07:27) Reddit X’s avatar marketplace and NFTs (08:33) Why R&D teams need to be a part of the core mission (11:12) What it’s like to be the first black PM at Facebook (14:58) How to foster diversity (19:40) Being a PM at controversial companies, and how to evaluate criticism (28:25) Adriel’s most stressful time at Lyft (30:35) The importance of operational control and what it means (32:35) Why there isn’t always an algorithmic solution to everything (37:47) Thoughts on AI (42:42) Growth hacking and algorithms at Facebook (48:18) Cannonballs in growth—fundamental changes in the product for optimization (49:07) Facebook’s “7 friends in 10 days” push (51:30) What is a marginal user, and what can you learn from their experience? (56:06) How to think about doing experiments (59:10) Why organization and empathy are the most important skills (1:02:59) Lightning round — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  They're probably, I call them techno utopians who would say, feed all data to the algorithm, given an objective and it will do the right thing. And I was like, yeah. The reason that falls down is the algorithms don't understand long-term effects often, nor do they understand how people might respond to it, nor do they understand your intent for the product. And I think it's really important for product managers to play that role. That is our job. When you are working on algorithmic heavy products, your job is figuring out what the algorithm should be responsible for, what people are responsible for, and the framework for making decisions.

• 0:41Lenny Rachitsky?

  Welcome to Lenny's Podcast. I'm Lenny and my goal here is to help you get better at the craft of building and growing products. Today, my guest is Adriel Frederick. Adriel is a VP of Product at Reddit, where he focuses on incubating and scaling new products within Reddit. Before that, he was Director of Product at Lyft, where he led the marketplace teams and the pricing teams over the course of five years. And before that, he was an early PM at Facebook, where he spent four years leading the user acquisition team. Adriel is one of these incredible product leaders who's way too under the radar because he doesn't spend all day on Twitter and instead is executing and building great products. One of the goals of this podcast is to highlight incredible product leaders who you may not be aware of,

• 1:24Lenny Rachitsky?

  and Adriel is a great example. In our chat, we talk about the origins of growth hacking, how to get better as a product leader, ways to increase diversity at your company, what it was like to work on Facebook's growth team early on, the future of AI, and a lot more. It was such a joy chatting with Adriel and I'm really excited to share this episode with you. With that, I bring you Adriel Frederick. This episode is brought to you by Linear. Let's be honest, the issue tracker that you're using today isn't very helpful. Why is it that it always seems to be working against you instead of working for you? Why does it feel like such a chore to use? Well, Linear is different. It's incredibly fast, beautifully designed, and it comes with powerful workflows

• 2:09Lenny Rachitsky?

  that streamline your entire product development process, from issue tracking all the way to managing product roadmaps. Linear is designed for the way modern software teams work. What users love about Linear are the powerful keyboard shortcuts, efficient GitHub integrations, cycles that actually create progress, and built-in project updates that keep everyone in sync. In short, it just works. Linear is the default tool of choice among startups, and it powers a wide range of large, established companies, such as Versale, Retool, and Cashout. See for yourself why product teams describe using Linear as "magical." Visit linear.app/leni to try Linear for free with your team and get 25% off when you upgrade.

• 2:54Lenny Rachitsky?

  That's linear.app/leni.

• 3:00Lenny Rachitsky?

  Hey, Ashley, Head of Marketing at Flatfile. How many B2B SaaS companies would you estimate need to import CSV files from their customers?

• 3:08Speaker 02

  At least 40%.

• 3:09Lenny Rachitsky?

  And how many of them screw that up? And what happens when they do?

• 3:12Speaker 02

  Well, based on our data, about a third of people will consider switching to another company after just one bad experience during onboarding. So if your CSV importer doesn't work right, which is super common considering customer files are chock-full of unexpected data and formatting, they'll leave.

• 3:32Lenny Rachitsky?

  I am 0% surprised to hear that. I've consistently seen that improving onboarding is one of the highest-leverage opportunities for both signup conversion and increasing long-term retention. Getting people to your aha moment more quickly and reliably is so incredibly important.

• 3:46Speaker 02

  Totally. It's incredible to see how our customers like Square, Spotify, and Zora are able to grow their businesses on top of Flatfile. It's because flawless data onboarding acts like a catalyst to get them and their customers where they need to go faster.

• 4:03Lenny Rachitsky?

  If you'd like to learn more or get started, check out Flatfile at flatfile.com/leni.

• 4:14Lenny Rachitsky?

  Adrielle, welcome to the podcast.

• 4:16Speaker 00

  It's good to be here, Lenny. Thanks for having me, man.

• 4:19Lenny Rachitsky?

  Absolutely my pleasure. I actually found out about you through a guy named Jules Walter, who we both know. He's a PM at YouTube. And I actually asked him, who should I have on this podcast? That is maybe a little bit under the radar, that is just amazing. And immediately he suggested you. And so I'm really excited to be chatting.

• 4:37Speaker 00

  Man, that is high praise coming from Jules. Jules is my boy. I love him. He's such a great guy. Awesome product manager and dedicated to the craft that like, just like being in his presence.

• 4:47Lenny Rachitsky?

  Yeah, we're gonna get him on this podcast at some point. He's busy with some kind of secretive project that we can't talk about. He's scheduled. I've been talking about Jules all day, but he actually has the 10th most popular guest post on my newsletter still. How about that?

• 4:59Speaker 00

  Wow. He's awesome.

• 5:01Lenny Rachitsky?

  Enough about Jules. So to give listeners a little bit of context on yourself, can you just give us like a 55 second overview of all of the wonderful things that you've done in your career?

• 5:13Speaker 00

  Oh, we'll do real fast. So, big highlight about me is I'm originally from Trinidad and Tobago, an island in the Caribbean. Came to the US for college, double E. Got seduced by consulting, went to that for a couple of years, worked in oil and gas, electric power, heavy industries, love that stuff. But also like writing code on the weekends for fun. So I thought I should move into tech and I did. Worked at Intuit and helped develop their first iPhone app, which was, you know, a thing back in the day. Worked at a startup, growth team at Facebook for four years, working on user acquisition, which was really fun. And I get kind of like strong performative experience I had. Quick stint in biotech and then worked on marketplace at Lyft. So rider pricing,

• 5:54Speaker 00

  real time driver incentives, matching riders with drivers, and then a lot of the operational tools that we use to manage our marketplace. And so that's a bit of my journey in maybe 45 seconds.

• 6:06Lenny Rachitsky?

  That was great. I don't have a timer for these, but that sounded right. So we're gonna talk about a lot of the things that you learned along the way at all those places. Can you also share what you do now?

• 6:13Speaker 00

  Awesome. Yes. So I'm the Vice President of Product Management for Reddit X, which sounds like we're out launching balloons into space, but that's not exactly what we're doing. We're more of a team at Reddit that's thinking about the evolving modes of interaction with Reddit. So content, temporality, the audience that you're talking to. If you think about it, Reddit is primarily about asynchronous conversations between anonymous strangers about shared interests. Sometimes other people find answers to their questions on Reddit. But we're looking into, on the X team, evolving that to look at problems like helping people communicate faster and easier about shared interests. Perhaps changing who they're having conversations with.

• 6:56Speaker 00

  Maybe it's about something other than a shared interest, or maybe they have something else in common that brings them together. Maybe bringing video, audio, and other media into being a part of the product and playing with permanence and things like that.

• 7:09Lenny Rachitsky?

  Whoa, I felt like you were going to go into a metaverse direction. Is there metaverse angles to this?

• 7:14Speaker 00

  Not really. I think we look at that as a potential technology, but our primary focus is a lot more on, I see, modes of interaction and platforms that are a lot more at scale today.

• 7:27Lenny Rachitsky?

  Got it. Is there anything coming out in the near future we should be looking forward to? I imagine you can't talk about too much of what you're actually working on more concretely.

• 7:34Speaker 00

  I think there's a few things that we've done recently that have been fun. We have an avatar marketplace that we've been working on recently where creators have been able to make art, put it up for sale on Reddit and make that available for other folks to buy and use. And that's been performing amazingly well. The underlying technology behind it is NFTs. And we thought that technology was really important to use because it gives a creator a public way of acknowledging their rights to a piece of content. And so they have some form of IP protection, especially in a marketplace where you're doing something like selling digital art. We felt like that was incredibly important. I think the technology behind NFTs has been used for some really mysterious things,

• 8:16Speaker 00

  but I think we're still in the infancy of using these technologies appropriately. There's a lot of like terrible use and a lot of uses that are wastes, but I think there's some gems in there and we're hoping to find some of those.

• 8:28Lenny Rachitsky?

  Sweet. I will avoid getting pulled into a web three rabbit hole here, but that is very cool. Something I wasn't planning to ask about, but I'm curious because I was just talking to some other guests about this topic is the idea of these kind of R and D ish teams at larger companies and companies that have been around for a while. I know you're relatively new there and this kind of maybe a new thing, but I'm curious, is there anything you've learned about how to set up teams like this and investments like this, these kind of long-term horizon bets, R and D teams?

• 8:55Speaker 00

  Yeah, I think it's really good coming to this from being on the other side of it. If you think about where I've been, I've been on growth and a marketplace, which is as far as you get from seeing like we're on the new stuff kind of team. And what I've seen happen a lot is organ rejection. That like this thing looks so different to the rest of the body and the rest of the organization that you get some form of rejection of the ideas entirely. So I think what I've learned is a few things. So first is the rest of the company needs to see what you're doing as being core and critical to the mission. It can't seem like these guys are just playing off in a corner on something that isn't related to what we are doing every day. Because I think that leads to some of the like resentment.

• 9:39Speaker 00

  Because you can imagine any team internally is fighting for resources and they look at this group as having resources that they can't get. They're like, ah, we gotta get rid of that because they're not helping us do what we are here to do. So you have to be part of the core mission, otherwise you're going to have problems culturally with that. So I think that's one thing. The second is it has to be everyone's success. So if you end up doing something on one of these R and D teams, it should just be the R and D team that wins. Everyone should feel like they win. And that is kind of related to that first goal I was talking about. I think the third is you have to set up the work that these teams are doing such that people don't believe all innovation is going to happen on that team. It can't be that like, okay, we're just stuck with the operational stuff.

• 10:23Speaker 00

  And they're getting to have all the fun. Other teams are still going to innovate, but maybe we're taking on something that other teams don't have capacity for, that the organization needs and is part of the core mission. So I think that's been a lot of what I think about when working on and setting up these teams is to make sure that we are part of the organization and everyone wants to hug us. This being, yes, you are one of us, not, you kind of need to go off in your little corner and behave.

• 10:49Lenny Rachitsky?

  Amazing. That is really helpful. So just to kind of recap, you want it to feel like it's core and critical to the company. You want it to feel like it's everyone's success. It's not just, oh, Adrielle over there is doing great, but we're like stuck with these terrible hard problems. And then this idea of not all innovation is going to be just coming from that. We can all innovate, but they're just working on this one specific innovation.

• 11:09Speaker 00

  Yeah.

• 11:09Lenny Rachitsky?

  Awesome. Okay, great. Another question I definitely wanted to ask you. You said you were born in Trinidad and Tobago, not something that you hear very often in tech. I'm curious your background and your journey to what you do now. How did that impact the way you lead, the way you build product, the way you just think about your career broadly?

• 11:26Speaker 00

  Yeah, it's not something that I really think about consciously, but it affects me every day. And it's tough not to see it in retrospect. I was the first black product manager at Facebook.

• 11:36Lenny Rachitsky?

  Oh, wow.

• 11:36Speaker 00

  And so it's tough for me to not see that having some effect on what was built or how things were built or on me. So it's pretty meaningful. But I think one of the ways to see how it affects things is actually just to understand a little bit about Trinidad. It's kind of its own little unique animal. So Trinidad is an island in the southern Caribbean, all the way at the bottom next to Venezuela. It's a really diverse place. So ethnically, it's 35% Indian, like from East India, 35% African, 25% mixed. And that last 5% is everything under the sun, European, Chinese, Arab, et cetera.

• 12:17Speaker 00

  And then for religions, it's about 60% Christian, but then that's a lot of different forms of Christianity in that 60%. 20% Hindu, 7% Islam. Media diet is a mix of British and American TV. You have a really broad range of incomes, but then schools are a melting pot. So you don't have as much of the kind of class and income segregation with schools that you get in most of the West. And so when you have that kind of a melting pot of ethnicities, religions, media consumption, and kind of socioeconomic status in one place, you learn a lot of them. Because in school, you're mixing up with everyone. One of the jokes we have is Trinidad probably has the most public holidays of any country.

• 13:02Speaker 00

  Because you have to celebrate everyone's holidays. From Diwali for Hindus, Eidul Fitir to Christmas. I have friends who were fasting for Ramadan. I know a lot of the names of Hindu gods and I always love shocking my coworkers with my knowledge of this stuff. So that gives me this really different perspective that shows up at work. So I'll give you an example, something I've noticed in almost everybody I worked with in tech. As we work on mobile devices, people make an assumption that one phone number plus one device tied to one person. And growing up in Trinidad, I just knew that wasn't true. Someone who is using a prepaid phone could have their number changed all the time.

• 13:44Speaker 00

  So that one person could have multiple phone numbers just because they were using prepaid. You have phones with two SIM cards. That was pretty common and a phone is and definitely was a really expensive digital device. It's a computer. So it was often shared and people couldn't just have one for themselves. So when I was working on user acquisition and designing registration for Facebook, that knowledge was incorporated into the design of the product in ways that I think other companies not caught on to yet. And I know for a fact that a lot of that thinking that went into designing, how you think about a phone number and an device and its use among one. It's, how to say, pairing with an individual has been helpful for Facebook's growth back then.

• 14:27Speaker 00

  And even after I left, I know that's still been providing benefit. So that's a simple example of how just being in that environment and soaking up information could help product design in a way that I think wouldn't have happened if I and others like me weren't there.

• 14:42Lenny Rachitsky?

  You said you were the first black PM at Facebook. I didn't realize that. How many PMs were there at that point when you joined?

• 14:47Speaker 00

  Oh, man. I remember we all fit into this conference room called Canada and that was probably like my second week. It was probably about maybe thirty of us in there. Yeah.

• 14:58Lenny Rachitsky?

  Is there anything that you learned from that experience about just like how to how to help with diversity at a company? Like, did Facebook do this well? Have you seen other companies do this better? Is there something you could share there for folks that are trying to work on this?

• 15:12Speaker 00

  Man, that's a tricky one. So the two parts of that, what was that like for me? I think it went quickly from being a little bit of imposter syndrome. Like that date when I'm sitting in that group, I was like, dude, I'm one of thirty people working on Facebook. Yo, what am I doing? I don't belong in this group. This is crazy. And then I recognized after talking to a lot of the other PMs and the engineers, it's like, no, no, no. They want me for what I know from my perspective, because they're really trying to grow this product globally. And being this guy from Trinidad working on growth with the perspectives I just mentioned was appreciated. I think I was lucky enough to be on the growth team and having leaders on that team who really valued diversity.

• 15:57Speaker 00

  I think about some of the teams I was on and they were awesome. I joke about them sometime. I remember being on a team where I was a black Trinidadian product manager with a female Israeli engineering manager, a female Brazilian tech lead. Then the rest of the engineering and design team was from all over the world. We had Russians, Chinese, some folks from Slavic countries. And it made designing products fun. Because a lot of times when you're building a product and you want to think and get into your head of your customer, you have to go out and talk to them because you don't necessarily get them really well. Man, we didn't need to on that team. We would just argue with each other. We would think about how our friends would use it, how our cousins would use it.

• 16:39Speaker 00

  And we are covering a broad swath of the world when we were arguing about how to design a product. And I think the original leadership of the growth team, I think, starting with Chima, but then followed up with Havi, valued that and kept bringing in that diversity of, again, ethnicities, religions, cultures from all over the world so that you could actually build a product that way. And it just makes you so efficient because an argument that might take two weeks to resolve because you have to go recruit a panel of users and talk to them and figure out what's going on. We cannot knock out in fifteen minutes. We're just throwing it back and forth with each other. And I can't stress how much that's important for building products that you want people across the world to use. You gotta have your teams look like the world. It just makes you so much faster.

• 17:24Speaker 00

  It's not perfect. You still have to go out and talk to folks because we still have our own kind of monocultures that form that we need to get out of, but it helps a lot. To your second point about diversity and how to foster it. Man, from the beginning of my career at McKinsey to today at Reddit, I've been in lings where everyone's asking the same questions about how to fix it. And here's what I've seen work. When you recognize that you get business value from it, then it all of a sudden becomes something that you look out for and you take care of. That's it. And there's definitely a lot more to it.

• 18:04Speaker 00

  But I think when it goes from frankly, something people feel they need to do to be PC or for cultural reasons, or because they're getting social pressure to do it, to something that you really recognize concretely, no, I get value from this. And you're willing to take the other steps to have a culture in your company that utilizes it. Then it becomes easy. Because when you bring folks in from diverse backgrounds, they retain. And that's always the number one step to growth, as you all know. Attention. You have to retain them. You have to retain diverse talent. And so you have to have an environment that values it, cares about it, and uses it and rewards it because it's part of the core system of the company.

• 18:49Speaker 00

  Then, once you have that working, it becomes a lot easier to recruit because people see you valuing it and bringing it in and wanting it. And it's not just like lip service that you're paying. That's been what I've seen to be true in all the conversations I've had on the topic.

• 19:04Lenny Rachitsky?

  That first piece is interesting that it answers the second piece, which is the point you made about how having a large, diverse, global group of employees early on, especially for a company that's trying to go global, international, so powerful, you just save all this time. You don't have to necessarily interview people that you don't already have.

• 19:19Speaker 00

  Yeah, there's something that feels like the approach to not doing it that way feels colonial. It almost feels like we're a group of people sitting down in this like tower in this country, in this relatively like sterile environment. And don't worry, we know exactly what you need in these other parts of the world. It's just, it just doesn't work well. So, doesn't feel right to me also.

• 19:42Lenny Rachitsky?

  Yeah, awesome. Thanks for sharing all that. That was really helpful. There's another topic I definitely wanted to spend a little time on, which is this interesting trend that I noticed when I was looking at your LinkedIn and your background. You worked at Facebook, Lyft, Reddit, and interestingly, they're all very like in the news, full of controversy type places. People like to tear them down and show all the reasons that they're doing bad things to the world. And imagine as a PM, that's just like a challenging place to be. And the fact that you've been at three different places, I imagine you've learned some stuff about how to operate as a product leader at companies full of chaos and fires and bad PR and things like that. So, is there anything to share about what you've learned there?

• 20:18Speaker 00

  I think the biggest thing is that as a PM, you are a leader. You have to provide a buffering or damping effect on the team. And that goes two ways. Sometimes we're doing stuff that everybody thought was amazing. This is the best thing we've ever seen. You kind of got to bring people back down to earth and go, look, that was cool, but we got a lot more stuff. We are really not there on providing the value that we want to provide to people in the real world. So, slow your roll and recognize that there's a lot more to do. And then when it's terrible and the press is telling you that like you're the worst thing to ever happen in the world, you kind of have to also go back and say, guys, slow down. We're not anywhere near as bad as what they think. You see and know what we're doing and they're going to misunderstand us sometimes. And so, pull your team up at

• 21:02Speaker 00

  this point in time and keep charging forward with the mission. I think some controversy is necessary. And so, I may be at a different point on that one. I don't think you're going to have any meaningful influence on the world without changing some pattern of behavior. And if you're changing a pattern of behavior, there's somebody who's invested in that pattern of behavior, and that's going to create some conflict. The most fun news stories to read involve conflict. So that's always gonna make for a great story and put you in the press. For Facebook, it was traditional media and other social networks were one side of the fight and then Facebook was the other side of the fight. And then it became other tech companies now. That always makes for a great story.

• 21:47Speaker 00

  With Lyft, it was taxis and unions. And so you have to recognize that you're always gonna have some bit of a challenge. Now, the really hard part about dealing with this is understanding what criticism is valid and how much of it is just because a source of power is being changed. So I'll give you an example. Let's see with Lyft, rich medallion owners in New York. I had no sympathy for them when they were complaining about trying to ban Lyft. Because when I was in New York City putting my hand out to get a cab, they would drive right by my black ass. And so I'm sorry, I'm like, I do not feel that much empathy for it. But I think there were really legitimate complaints about the structure of driver pay

• 22:33Speaker 00

  that were coming up and that were behind, I think, some of the complaints and some of the big stories in the press and some of the big kind of legal action that was taken. Paying for pickup time when a driver's on their way to pick you up or when they drive somebody far out of town and they have a deadhead to come back into a place where they can work. That's real. That's a real problem that I think we got called out for, that we weren't paying enough attention to. And it got us off our ass to go fix it. I don't think we've, and I say we, but I'm not there. I don't think the problem has been fully solved. But I think as a PM, listening to this, you kind of have to find the truth behind it and try to find a way to work on that and not get too lost in responding to the specific criticism.

• 23:16Speaker 00

  And so to walk this line between kind of going, yeah, some of this controversy is just part of the game versus like, no, this is really valid. Dude, to figure out where that is, you gotta do what is so cliche. But like, you gotta stay close to your users. And so, to give you an example of how I did that, when a lot of the complaints were happening about driving on Lyft. I drove. I would just pick up the car, I don't get up and I go drive and I'm like, let me go feel this for myself. Let me go see what these guys are talking about. Matt, I can give you a story about Rick. I still remember this drive I did with Rick in Berkeley. So, I'm at home, I just get in the car, I turn on the app, it's time to go driving.

• 24:00Speaker 00

  I get up ping fifteen minutes away. And I'm thinking, dude, if I go do this right now, it's got my cancel on me. I'm not really getting paid for this, but maybe the ride is worth it. So, I drive on over, I'm dodging traffic, pedestrians, drunk college kids, stop signs. I make my way over to Rick, he's coming out sheeping ease. And he's about eighty years old. Jumps in the car, and then I push the button to figure out the destination. And it says the ETA to the destination is two minutes. So I was like, hey, Rick, you get this right? What's going on? It's like, hey, I had a little bit too much to drink. I'm worried about breaking my hip. So that's why I called it right.

• 24:45Speaker 00

  And so I went from wanting to curse Rick out for making me drive fifteen minutes to come pick him up. To feeling like, alright, no, no, no, there's real value I'm providing you and driving him just two minutes. But I recognized that wasn't embedded in the structure of P. Rick would have been happy to pay for my fifteen minutes to come pick him up. But we weren't, one, giving drivers compensation for that, nor were we finding a way to pass that through into pricing for Rick. It's a much more difficult problem than it seems from that simple example. But it clued me into why drivers were complaining. So then I went, got it.

• 25:26Speaker 00

  I understand what we need to do. So when there were all the PR was going on about AB5 and Prop 22. I was out driving and I was out sitting with the team trying to figure out how we're going to design a product that helps pay a driver for this. Still keeps prices reasonable for users, doesn't create bad incentives. Where you end up with riders not getting picked up when they really need a ride because I didn't want Rick to break his hip. He still needs a price that makes him feel like it's okay for him to take that ride and finding a way to balance this out. Is actually more complex than you might think. And that's what I stayed focused on. Whether Prop 22 passed or not. I was ready for either side with a solution that was going to work for riders and drivers.

• 26:09Speaker 00

  That was the job. And so I think for PMs, it's, it was so easy to get sucked into like the press and it's like, yo. Plan the work worth the plan. Go back to your job. That's what you're supposed to do. Solve for customers in the middle of this. And then you figure out how to communicate it well.

• 26:27Lenny Rachitsky?

  What I love about that strategy is it also helps you see that it's not everybody that is worried about something. I think of Airbnb, like all hosts are pissed off about this one feature. There's going to be a revolt. And then to your point, you talk to some, like, nobody even knows about it. Nobody cares. Everyone's fine. And so this, there's so many benefits to what you're talking about doing, which is, yeah, talking to customers, not just paying attention to the loud voices.

• 26:51Speaker 00

  Absolutely. You know, I also have empathy for reporters too. The story that with the headline, some Airbnb hosts are annoyed by the change. I mean, come on. That is not, that is not a great headline. I recognize that they have a job to do and sometimes they hold people accountable. And sometimes they get people to read a story that maybe has a bit of hyperbole in it. And so they have to do their job and I have to do mine too.

• 27:14Lenny Rachitsky?

  Yep. This episode is brought to you by Eppo. Eppo is a next generation A/B testing platform built by Airbnb alums for modern growth teams. Companies like Netlify, Contentful, and Cameo rely on Eppo to power their experiments. Wherever you work, running experiments is increasingly essential, but there are no commercial tools that integrate with a modern growth team stack. This leads to wasted time building internal tools or trying to run your experiments through a clunky marketing tool. When I was at Airbnb, one of the things that I loved about our experimentation platform was being able to easily slice results by device, by country, and by user stage. Eppo does all that and more, delivering results quickly, avoiding annoying prolonged analytics

• 27:58Lenny Rachitsky?

  cycles, and helping you easily get to the root cause of any issue you discover. Eppo lets you go beyond basic click-through metrics and instead use your North Star metrics like activation, retention, subscriptions, and payments. And Eppo supports tests on the front end, the back end, email marketing, and even machine learning clients. Check out Eppo at geteppo.com, get E P P O dot com, and 10x your experiment velocity. You shared this really heartfelt story about Rick. What's your most stressful memory of working at Lyft?

• 28:32Speaker 00

  I think the most stressful time was when I had to unwind up that product I did and actually make a better version of it. It was really a pricing algorithm change of something behind the scenes that nobody would really see, but this was a fairly big initiative that we worked on. We had experts in revenue management to work, like PhDs and the people who wrote the textbook on the subject, helping advise us on this. We build this model, we launch it, and you're expecting like this big change and it goes poof, just does a little bit. And then we work at it, and we work at it, and we work on it, and eventually we get it to be good. And it works really well in three cities. We start rolling it out to more cities and it's a pain in the butt to roll it out to more cities because it's super complex.

• 29:17Speaker 00

  And eventually we get it rolled out to maybe a hundred cities. And then someone says, all right, cool, I want to change prices. And oh, we struggled for months to implement price changes. And man, the sentiment like around this product was just rough for a while. And I remember being on our walk after like a particularly bad week of this and trying to figure out what I was gonna do about this thing. Like, do we stay at the course? After a while, the answer was kind of simple, even though it was emotionally difficult. And the answer was like, yo, we gotta rebuild it. There was no answer where we couldn't have a product like this.

• 29:57Speaker 00

  We needed some ability to be able to influence prices so that we could actually run an effective marketplace. The current solution didn't work. It wasn't as operationally flexible as we needed it to be because we didn't consider that requirement when we were building it. And we got caught up in the kind of algorithmic complexity and sweet sauce of it. And so I recognized that we just needed to own up to it, tell everyone we didn't get it right. And we needed to come at it in a different approach that was actually more flexible operationally. And we did it. I think the big learning, at least in that business, was you have to think about operational requirements and operational control as a first order requirement.

• 30:42Speaker 00

  And I think when a lot of us were building product at a lot of the other consumer internet companies, you didn't have to think about operational control. You give the algorithms an objective, you feed them some data, you let it run, you observe it initially, it's doing nothing crazy and you tweak it. But you didn't need to have D to D operational and strategic control over the product. And we just needed to snap our brains into being able to put people in the loop with the algorithm.

• 31:08Lenny Rachitsky?

  For folks that haven't worked at a company with this kind of on the ground ops team, can you just unpack what that is like operational control? What does that actually mean in in practical terms?

• 31:16Speaker 00

  Okay, so I'll give you an example. So Lyft is in, back in the days, Lyft is in three hundred cities, probably roughly across the US. And in every single one of those cities, you don't have exactly the same pricing. It's a little bit different. And so sometimes you might need to make a change seasonally because traffic gets worse or because fuel prices were different or because there's a new tax or because your competitor did something that you need to respond to. And your algorithm cannot see this. It has zero visibility into this. And so you need a person in the loop to not only give that visibility, but also to make a decision about how you respond. Because I think also in, let's say you're in Chicago

• 32:01Speaker 00

  and there is a snowstorm and you need to change the way, let's say, you need to update prices so that it handles the increases in driver pay that you need to create to get people out during a snowstorm. You don't know exactly how you want to respond. Every snowstorm is different and a person has to make that judgment call and provide the right information to the product to be able to utilize it. Now, algorithms were handling a lot of that and they could generally respond, but to be a lot more precise, you needed a person to help handle that, to make that call.

• 32:33Lenny Rachitsky?

  Got it. Cool. Thanks for sharing that. So you're making this point about when you're at a company that has a big operations component and obviously the core central product team. You were sharing some learnings about what you've learned to work in that environment. So yeah, I just wanted to come back to that.

• 32:49Speaker 00

  For sure, for sure.

• 32:50Lenny Rachitsky?

  So the main thing you said is just treat ops as a first order component when you're designing the software. Is that the big learning?

• 32:55Speaker 00

  I think it's not just treating ops as a kind of first order requirement. The bigger picture for me was like, when I look across my career is the algorithms need people to help make judgment calls. And so I saw it really, I got a heavy lesson in it at left. But when I look back, I recognize it was there at Facebook too. It just wasn't in my domain. There is always a judgment call that has to be made between. How often are there going to be ads versus how often are we going to show organic stories from your friends and family? How often are we going to show content that you might be interested in? That's not quite in that group.

• 33:38Speaker 00

  How often might we want to show you things that help you find your friends or help other people find their friends? And that is a judgment call that varies for different markets and different situations. And there may be algorithms behind the scene that are making that call for every single person in real time. But there still have to be people applying some strategic judgment to that. And I wasn't in a position of needing to do that at Facebook. But once I saw how much I needed to do it at Lyft, and I kind of looked back at history, I saw that it was there too. But I think there are too many people who don't see this and believe that there's an algorithmic solution to everything. And I think as a product manager, and especially product managers working on systems that are

• 34:19Speaker 00

  heavy on machine learning or operations research and optimization, to think about where you want a person to make a decision and where you want the machine to be off to the races. And to think about that as a product design problem, because there actually is a human computer interface that you have to think about there. You need information about what's going on. Let's see at Lyft, what's going on with my market? How long does it take for somebody to get picked up? How expensive am I versus the competition? What are my goals in this market? And like, how am I performing today with that? Give somebody information, but also give them the tools to execute the right decisions without creating trouble. And that's like a product design problem.

• 35:03Speaker 00

  That's a first order product design problem like anything else that you have to think about. And I'm not privy to it, but I would guarantee that there are people thinking about those same kinds of problems at other companies.

• 35:15Lenny Rachitsky?

  That reminds me, I was just listening to Zuck on Joe Rogan. And he made this point that when you look at a post, you can like add a little emoji reaction and you can have a little angry emoji reaction. And he made the call that we're not going to use the angry emoji reaction in our algorithm in any way. We're just going to ignore that because naturally you'd be like, okay, people are angry. That's interesting. Let's show that because it's interesting to people. But he specifically wants to avoid anger and facilitating anger, probably because a lot of the feedback that they've gotten.

• 35:47Speaker 00

  Exactly. And I think they're probably, I call them techno utopians who would say, feed all data to the algorithm, give it an objective and it will do the right thing. And I was like, yeah. The reason that falls down is the algorithms don't understand long-term effects often, nor do they understand how people might respond to it, nor do they understand your intent for the product. And I think it's really important for product managers to play that role. Like that is our job. When you are working on algorithmic heavy products, your job is figuring out what the algorithm should be responsible for, what people are responsible for and the framework for making decisions.

• 36:27Lenny Rachitsky?

  Is there an example that comes to mind where you did that or didn't do that well? Or someone on your team should have, just something to make it a little more concrete even?

• 36:36Speaker 00

  Let's assume that you are a person working on pricing and you say like, great, I have an objective that is I would like to win market share in a region. Okay. And you left that to an algorithm to say, I need you to optimize prices such that you maximize market share. But what would the algorithm do? Jump your prices to the floor, all the way to the floor and then you don't make any money. Okay, great. So then you say, okay, what's the next step on that? Let's give it a constraint. Let's set some target that we might want to have for how little profit we might be willing to take. Okay, go do it now. What if the guy on the other side is doing the exact same thing? Both of you will hit your constraints and then the game will stop.

• 37:17Speaker 00

  Okay, great. So now it then flips to, oh, we have to choose where we want to win. And so I think one of the things we did that I'm particularly proud of is building products that help people see and understand that game a little bit more and decide where they want to. I think the first few pieces of that are not shockers,

• 37:38Speaker 00

  but like that conclusion at the end where you get to, oh, wait, I need to create a tool that gives people information to then decide how to play this game is actually what's critical.

• 37:46Lenny Rachitsky?

  Interesting. So kind of what I'm hearing is a lot of the work is giving humans more information versus giving machine learning algorithms more information. And there's a lot more leverage potentially there, giving humans more ways to tweak and dial.

• 38:03Speaker 00

  Let me refine that a little bit more. It's more about giving people the information that they can use for decisions that they alone are good at and giving machines the power to amplify a person's intent. So one of the ways I like to think about it is all software in any form, including ML, is just a tool like a screwdriver. And you could try to put a flathead into a Phillips and maybe it'll work a little bit, but it's better to use a Phillips screwdriver. And we're tool designers generally, and especially in product development function, you figure out how much do I put into the tool? And how much I leave it up to the person? And I give the person the ability to choose what

• 38:46Speaker 00

  they want to do. I give them a screwdriver, a flathead, a Phillips, a torx, and you let them decide how they want to use the tool for the application at hand. So going from that analogy to concretely with ML, you say, look, machine learning is going to be amazing at optimizing for a given objective, but it's not going to understand the constraints or strategic choices I need to make. The constraints and strategic choices that we need in the external world are always going to have to be decided by a person. You make that incredibly easy for people to do and intuitive for them to do. And then you go, that algorithm can then amplify their effect.

• 39:25Speaker 00

  By making decisions on hundreds of thousands, potentially millions of individual decisions to take that person's intent and amplify it. Given all the information that they can learn in that single context. So I think about it as designing an interface and making an extension of yourself rather than a black box on its own that you just feed more information to. Is that helpful?

• 39:49Lenny Rachitsky?

  Yeah, it makes me think about a Neuralink and what Elon's trying to do. I don't know if this is how he thinks about it, but the way but why guy described it as Elon's worried that AI will take over at some point. And so he wants to build a tool that connects straight to our brain that can access the power of AI to kind of have a chance against just a rogue computer in the future.

• 40:10Speaker 00

  Even then, you've got to make sure the person is still in control. I hear that thought and I go, okay, you build the interface. But then who's in control?

• 40:21Speaker 00

  Right? Is the person still in control or did they become a slave to the machine and you just made a better interface to make them a slave?

• 40:27Lenny Rachitsky?

  Oh shit, we're in trouble.

• 40:29Speaker 00

  I am not yet as worried about these versions of them taking over thus far, and maybe I haven't fathomed what they can do. They still seem like tools that need our guidance to be useful. Even the most amazing, we've, we've been seeing the image generation and I've seen some of the cutting edge like text generation stuff. They can fool you into believing that they're like mere human capability, but there is a lack of decision making and judgment that I see coming out of them. I see them as being again, extensions and useful like text generation algorithms. A lot of them can't write a paper for you.

• 41:06Speaker 00

  And that's what I think people are scared of because it's, it still requires your judgment to decide. Now, when you decide what the Cilian topics are in something you've read, let's say you're doing a book report. You've decided what the topics are. He can help you write the paper faster for sure. But it can't write the paper for you. It can't choose the topics that like your background and history and interest find useful or compelling.

• 41:31Speaker 00

  To tease out.

• 41:32Lenny Rachitsky?

  This isn't where I was expecting your conversation to go, but I'll add another thought here because it's interesting. The way I think about it is there's nothing like magical about our brain. And so if that's true, why isn't there a world where we could just completely simulate it? Sam Harris talks about this a lot, that it feels like, once you get close, then it could just accelerate so quickly beyond human potential. Like it'll start from like 20% as good as a human to like 40, 50, 60. And then it goes to like a million times better. It can move so fast beyond us very quickly. So I think that's where a lot of the, not that I'm afraid of this, but I feel like that's where a lot of the fear comes from. It could just like Dolly coming out and co-pilot, just like, holy shit.

• 42:11Speaker 00

  Yeah, our brains are good with linear thinking, not exponential. I saw through that argument that like, yes, this is increasing exponentially and you can't fathom it. I'm like, yes, that is definitely potentially true. Completely see that possibility and recognize that I have that like cognitive defect in being able to understand it. And even if it's a remote possibility, we should be paying attention to it. So I am all for paying attention to it, given the, let's just say, the high cost of a low probability outcome is still a high cost. And so it's still worth paying attention to.

• 42:40Lenny Rachitsky?

  Yep. Okay. Good tangent. I wanted to chat about your learnings at Facebook. We've been chatting about all these other places and especially about growth, just stuff you've learned about growth and growth hacking. And I was thinking about this interesting world that Facebook is in slash meta, where on the one hand, when they started, and I'm talking about growth hacking, like Facebook did a lot of growth hacks, emailed all of Harvard. He had all these interesting dating thing happening and got a lot of controversy and there's all these interesting tactics to start Facebook. But now people use Facebook to growth hack and grow like Zynga famously, a few other places. So all that to say, I'm curious, what have you learned about growth slash growth hacking from your time at Facebook and other places?

• 43:22Speaker 00

  I think growth hacking, as traditionally you find, like finding those small changes you can make to a product to give you outside impact, that is absolutely valuable. What I've seen people get losses, they assume that if you do that alone, it will work. You can grow your way into something successful if you just find those few hacks and patch them together. And there's something about that, that I find disrespectful to the people using the product. It's like you assume that they have no intelligence and they will catch on to what you're doing eventually. You know, they all saying fool me once, fool me twice, you know? Kind of applies. So if you don't have a product that's providing real fundamental value to people, you can be a one hit wonder, have a flash in the pan and grow that key

• 44:04Speaker 00

  into something that might last for a few months, but like people will catch on to it and it'll disappear. So I think that stuff is helpful, especially early on to get your initial traction. But you gotta have something people like and wanna continue using. And when I think back over products we did that really moved the needle, they were all things that just focused on the marginal user and figured out how to make the product easier. It's easy to get seduced into thinking that there is a fast, secret way to do it. And I'm like, no, the vast majority of it was just hard work and finding ways to solve the real problems. And what are those real problems? They were pretty damn simple, but we just grinded on them for a long time and just stayed on it.

• 44:47Speaker 00

  One, make it easy to find the product. Number two, make it easy to get into the product. Three, stupid easy to find your friends. And then, once you did that, you were off the races. And like, those were the things we were doing over and over again. I think another big piece of it is reminding people that there's something interesting here and building the habit of coming back to the product. It was also part of it, but like, we just grinded on those few things over and over again. And some of the really big wins weren't hacks. They were just paying attention to little things. I'll give you an example. I remember sitting one day thinking

• 45:29Speaker 00

  about how to help people find their first few friends. And we would do this thing where we'd have recommendations. If you get one or two friends, you'd be off to the races and we could find you more people that were in that same friend group. So I thought about the way the people you may know algorithm worked. They get one or two friends, they would find your mutual friends and then would help find you more of those kinds of folks. And I was like, you know, what that does is it spirals you down one friend group, but it doesn't get you all your other friends. I remember just like looking at somebody using the product and recognizing that we were only taking them down this one path. So I was like, man, how do I see all your friend groups? And so we had this idea that we came up with that would do it. I'm not gonna let that one out.

• 46:09Speaker 00

  And um, it was like game changer, like absolute game changer, especially for new users, helping them find those first few friends in a few different friend groups. Which then meant we could get you down one group and another and just continue building out that graph just by using recommendations. Because we had a great tool for seeding it and that was not easy. That was not a hack. That was hard work. I also remember like, one of my favorites is something Tom Allison did. Tom Allison, I think now is responsible for Facebook app. And when he was working on like the engineering manager for one of those teams. There was a change we wanted to do to one of these algorithms. And it was a big change. It wasn't a hack. And it was gonna take a few months to pull off.

• 46:56Speaker 00

  And Tom just hid it in a corner. He just didn't let everybody know that we were really gonna change the way this product worked. He had a really smart guy working on it as a chain. And like, they just hid off in a corner, rebuilt the product in the way it needed to be built to make it easier for us to operate it and scale it. And then put it out there and of course it crushed it and they were incredibly modest about it. But it was not a hack. And it came from them looking at this deep problem of finding that thing that mattered. And then saying, we need to make a fundamental change to make it easier to recommend friends to folks and just grinding on it. And so one of the things I recommend for people when they're thinking about growth for their product is figure out what the core actions are and then grind on them.

• 47:40Speaker 00

  Think about removing them, removing friction in some of them, but just keep staying at it. And as you grind on it, you'll do little hacks. You gotta figure out how to put, you know, right text in the button and get it above the fold. Create the right copy, like all the things that we traditionally associate with growth marketing. You gotta do those things. But to me, that's stable stakes of just doing good product communication with your user. But then like, you gotta think about this person who can't yet figure out your product and is trying to take this action and making it stupid easy for them. I got a million more examples of that one, but that's the game. It's not just finding some trick to spam a site.

• 48:18Lenny Rachitsky?

  I love that. The way I think about this that I've heard well described is just there's no silver bullets, just many lead bullets.

• 48:23Speaker 00

  Yes, and a few massive cannonballs every now and then.

• 48:29Speaker 00

  Every now and then there's some cannonballs.

• 48:31Lenny Rachitsky?

  What's an example of a cannonball as you think about that?

• 48:34Speaker 00

  Sign up with four numbers, which is now like par for the course. That was a cannonball. Getting SMSs delivered to people all over the world. Doesn't sound glamorous, really hard to do. That was cannonball. Good friend recommendations. Another big one. There's more. I'm not going to go into all of them. What I mean by cannonball here is that there were sometimes some really big fundamental changes you needed to make to the product to make these things work.

• 48:57Lenny Rachitsky?

  Got it. So you think about that in terms of investment, not necessarily the impact. Impact plus massive investment. Cool. I have so many questions along these lines. Okay, I'm gonna pick a couple. One is Facebook is famous for this kind of activation milestone of getting 10 friends or seven friends, whatever it was. Like there's some number of friends you gotta get and the good things will happen. Were you involved in that? Do you have any insight into like how that came to be? Is that real?

• 49:21Speaker 00

  That decision came before me. I saw it. I understood the data and I worked on this problem. What I thought was brilliant about that was not the, it's not the metric. It was the designing it to be understood and communicated. What I think is fabulous about it is that you're talking about it now because it's memorable. And it got people to take the right actions to start chasing the gold. There was literally nothing magic about the number or the date. But basically it was a way of seeing like, get people as many friends as possible as fast as possible. And if you said that generically to someone, they'd be like, yeah, I kinda get it, but yeah, I'll go do that. When you create a discrete number and a discrete time and there is a concrete goal to chase

• 50:05Speaker 00

  and there's a number and a graph that everybody can look at and see, we are gonna go make that thing go up.

• 50:11Speaker 00

  The organizational effect of that is galvanizing. So what I thought was brilliant about it is, as I've heard the stories, you know, this is all secondhand. There was a lot of debate about what the number should be, what timeframe should be. And at some point, Zuck just said, ten friends, fourteen days, go. And it just, just got people past the academic debate of like, all right, got it. As many friends as possible, as fast as possible, let's go.

• 50:35Lenny Rachitsky?

  I love that. That's exactly how I've always thought about it. That it's not the number exactly, it's just a rallying cry that everyone can just get around and just go. Doesn't need to be this perfect number that has like incredibly correlated link to retention or anything like that. It's just like, this is good enough. It's probably directionally, I just tried to do this.

• 50:52Speaker 00

  Let's just go. There are downsides of it. Some of them are really funny. I remember looking at a graph of like retention versus number of friends. And like would actually drop with eleven or twelve versus ten. Because somewhere in code, somebody had done something with ten friends as the limit. To help improve retention and it shot off at eleven or twelve and it came back up. But I was like, you know what? That's fine. That's completely fine. Because the, if we didn't get that like organizational momentum, that graph would have just been lower. So I could take the kink where it drops. It's fine.

• 51:30Lenny Rachitsky?

  You also mentioned this term marginal user and I thought it'd be helpful just to unpack what you mean by that.

• 51:35Speaker 00

  For me, it's a person who is just on the cusp of taking the action you want to take. Talking of the concrete example, when working on registration, I would try to find a country where we had a lot of growth, but for some reason, our conversion rates were terrible. So we had a lot of traffic, but conversion rates were terrible. And I was like, okay, that's the marginal user. This is the person who is just on the cusp of coming in, wants to come in, as you can see by the traffic, but we can't get them in. So why? And when you go to the extreme and you find that person who's the worst, right? And they, most likely, it was a person on a feature phone on edge.

• 52:13Speaker 00

  Trying to access Facebook in a country that was far from one of our data centers. And then you go like, all right, what's wrong with this person's experience? Let's go check it out. You're like, oh, you see everything that's wrong with the product. So then it gives you a list. Okay, the language is probably wrong. We didn't get that. Are we detecting the country properly so that we can actually get their phone number formatted? Probably, probably not. Oh, man, it's far from the data center, so their connection's slow and they're on edge. Oh, that's terrible. And you just see and package up all those. It gives you everything that's wrong. And then you just start figuring out what to do with them. Something I caution people against, though, is don't use the data alone to figure out who the marginal user is. It'll give you a clue where they are and what might be wrong. It'll give you some hints. It's not going to give you the answer.

• 52:58Speaker 00

  You have to go watch them to find the answer. Because I think in a lot of these, like, data-driven places, somebody will say, great, just create a funnel, figure out all these drop-offs of the steps in the funnel, look at it yourself, and then figure out what might be wrong and go fix those things. But what I think happens is often there's a problem that's orthogonal to that funnel that you can't see from looking at the data. And you have to go look at the person and talk to them. I remember one example we had is like, was watching someone sign up for the very first time for Facebook in India. And they're about to put their name in. And I asked them, like, so what, what name are you going to put in? They're like, okay, my full legal name. All right, cool. Does anybody in the real world call you that? No. And I was like, oh, dude, we're screwed. If you send a friend request, it's not going to get accepted because nobody knows who this person is.

• 53:42Speaker 00

  And then the reverse, if they find you, they don't know who this is. And so it was like, yo, you're, you're going to look at some problem deeper in the funnel. Yeah, what's going on with my accept rate? And then you're going to go tear apart that little mini funnel and then recognize that you had a problem that happened ways back. And so when you're thinking about that marginal user, you gotta go and look at them, talk to them, watch them use it, try to get into their shoes yourself as much as you can, and then make the call from there on what you do. But that data isn't going to tell you. It's going to give you the answer. It'll just tell you how bad it is.

• 54:12Lenny Rachitsky?

  Wow, I love that connects back to the same advice you gave in all these other contexts. Just talk to people. Don't rely on just this aggregate data.

• 54:19Speaker 00

  And now, don't get me wrong, like, I built the experimentation platform at Lyft. I'm a guy who loves data and loves using it and looking at experiments. I think it's just too easy to try to sit on your laptop, pull up a funnel or pull up some charts or look at an experiment results and think that's going to give you the clue to what to build. It's a compliment. It's not the only thing, and I've watched people fall into that trap of assuming, especially when you're working at companies with lots and lots of data. You fall into the trap of thinking that you're swimming in answers because you have all this data and just need to tease it up. Just go out and talk, you'll find it faster.

• 54:53Lenny Rachitsky?

  I really like this advice of when you're trying to optimize things, focus on your marginal user. And there's two parts to it that you talked about. There's the next most likely person to sign up, and then there's like the worst case and going to like them to see all the things that are wrong. And that being your like North Star, make this person successful, make so many more people successful. Is that how you think about it?

• 55:12Speaker 00

  Yeah, I do. So, you know, marginal user, I think, is a fun word to think about because you think of families think about shit. The person is right on the cusp, but I like to go to the worst. It shows me everything that's wrong, but the marginal user thinking helps you prioritize what thing to do next. So like that person, that example marginal user I was talking about, they're in a future full with edge. Dude, there's a lot wrong. That's just going to be tough, but I might look at that experience and go, all right, let's see. Somebody was like perfectly equipped, the best phone and a great internet connection in that country. What would still be wrong? I was like, oh, language is still wrong. And the latency to their phone is actually still pretty high to our data centers, which is why it's taking a long time to sign up.

• 55:52Speaker 00

  I could still fix that. So that's how you can see the worst case, tell you everything, but then decide what is marginal. By removing a few of the barriers, and you know what's difficult for you to attack and then see which ones are closer to being resolved.

• 56:06Lenny Rachitsky?

  Awesome. I wanted to ask one more question about experiments at Facebook back in the day. So we talked about there's all these lead bullets or some cannonballs, maybe a silver bullet somewhere. In your experience, what percentage of experiments end up being impactful and successful?

• 56:21Speaker 00

  Again, that's a difficult and different question. So I'd say probably 60% successful, 40% like you should turn off. But within that 60%, I think there's a hidden cost to the experiment, which is that like you're futzing around with something small. You could have used your time on something bigger and more meaningful, but you're futzing around with a bunch of these small things. Some of the small things were incredibly meaningful and you needed to do them. So I think this is actually, it's almost like the same problem about, I don't know which of my marketing is best. You have to try a bunch of stuff and then figure out what was terrible. You don't know before you do it, before you do the experiment, what the impact is. But sometimes what I've seen is let's be take a bunch of them as a program. And let's say you have over the course of three months, you're going to experiment with 10 things. You might have been able to push on two really big ones.

• 57:07Speaker 00

  And what I've seen is there's a laziness and this is not, this is broadly, this is not just a baseball. This is broader. There's a laziness that can creep in where you're just finding a lot of little things because they're easier to come up with. And they're easier to design and think about, it's easier to build, it's easier to talk to your boss and see. We moved the number by point oh two percent. And like, you feel good about doing those few small things. And so it creates this incremental thinking. We're just trying to a bunch of small things that just don't meaningfully add up to something big. I think what's healthy is having a good portfolio. Because basically you say like, look, I'm going to have, using our analogy from further from before, I'm going to have some cannonballs. I'm going to work on a couple cannonballs and I'm going to have a bunch of red bullets.

• 57:49Speaker 00

  And maybe it's 80% of your energies on those big cannonballs, 20% on the light bullets. And what it like having a constraint like that forces you to choose the few experiments that are actually probably the really good ones. And it's not just a whole bunch of crap that you're trying out.

• 58:03Lenny Rachitsky?

  And is that actually how you divide up those bets broadly? Is that like a rule of thumb you have or is that just numbers you're putting out there?

• 58:09Speaker 00

  Those are just numbers I'm putting out there. It's always going to be, I've got call beast on where you are. I think depending on the stage your product is at, it should be a different set, a different bias. Very early on, when you're building a product, you kind of know what the big things are. You've talked to enough people, you have enough, just go build it. You should not be playing around with experiments. It might be a hundred percent cannonballs. Just go knock the big pieces out, don't worry it'll work. Also, the cost experimentation is time. So if you're experimenting on every little thing and waiting for the data to come in. And then also screwing up some other part of the product because your experiments on 50 50. It's just not worth just bang the big things out. As you get more mature, the balance needs to switch in the portfolio.

• 58:54Speaker 00

  Probably, yeah, probably aren't that many big cannonballs anymore, probably just one. And there will be a lot of little refinements that you need to work on. And by then, you have the scale that the time to experiment isn't as high and the cost of experimenting is lower. So it's fine. It's good to do without.

• 59:10Lenny Rachitsky?

  Okay, one last question before we get to our very exciting lightning round. So you've moved from IC a while back at this point to now VP of Product at one of the most trafficked sites on the internet. And I'm curious, what skills have you grown or had to grow most as you've gotten more senior in your career?

• 59:31Speaker 00

  Organization design and empathy.

• 59:33Lenny Rachitsky?

  Wow, I love that.

• 59:34Speaker 00

  Oh dude, for a long time, and I think this is, I and many others, I had this idea that the people who were the smartest are the ones who rise. The people who are the most technically competent are the ones who rise. People who are the best individual contributors are the ones who rise. And somewhere along the way, I had that idea disabused of me and I recognized like the job's different. It's more about building a great team, creating the right incentives for the team, unblocking them, guiding them, and helping them work efficiently. Those mattered way more than anything else. And I guess one of the ways I slowly recognized it is like, as I started going up in my career,

• 1:00:15Speaker 00

  I recognized that if I wanted to have more impact, I couldn't do everything myself. There was just more that needed to be done. And in today's world, you can't do anything meaningful by yourself. You need a lot of people to do stuff with you. There's nothing meaningful that gets done by any single person, even though people like to make you think that in their hustle porn that they post online. So it made me just step back and think about what helped me be productive environments when I was productive, and how I could do that for others. Cause then that would just actually help me. And so there were simple things like clear goals, helping people feel safe. And understand that like you've got their back, making it easy to do their jobs.

• 1:00:59Speaker 00

  But my job is to make sure the processes for you doing your work and the people who you have to interact with are just buttery smooth and everything just runs easily. And so that was like lesson one. It was just like designing a good organization, culture, skills, people, processes, et cetera. All necessary. That's one piece. The second is like empathy. Worst step of that was just like, you have to have that as a PM for your user. But I think it's different to having it for a peer in another function or somebody else on one of your teams. And the hardest part of it is they say getting in somebody else's shoes. The hardest part is taking my own shoes off. Basically going, yo, okay, I came into this, there's something I wanted, I wanted to

• 1:01:45Speaker 00

  get rid of that. Now, just talk to this person and try to understand what's going on with them. What they care about for like their life goals and motivations, what they're scared of, what they're excited by, how you might be able to help them. Once I was able to get out my shoes, clear my mind, try to get into their head. Then I could be like, all right, cool. Let's find a nice happy middle ground in the middle here. But that's something that works for both of us together. And sometimes for me it was, yo, what I care about, I'm good. I'm gonna let you do your thing. I've gotten into your shoes. I need to leave you alone. Like, you're good. Other times, you know, I'm ready to push. But I think once I have the empathy, I'm then able to think about what we as an organization broadly want to achieve

• 1:02:25Speaker 00

  and try to put the two shoes on at the same time and find something that works for both of us towards that common objective. And I think that's how I try to approach almost every conversation is especially being a guy who looks different, talks different, comes from somewhere else. First thought they might have is and unconsciously might be like, this guy isn't one of us. But then once then they get clear to them that we have the same objectives, we're about the same thing, and I want to know what's going on with you so that I can help you achieve what you want to achieve. Do it. Go, turn this away.

• 1:02:57Lenny Rachitsky?

  Okay, I know I have to let you go and you have to get back to real work. So we've reached our very exciting lightning round, the final part of our little chat. And basically, I'm just gonna ask you five quick questions, whatever comes to mind, share it, and we'll go through a pretty quick. Sound good? Okay. What are two, three books that you recommend most to people?

• 1:03:16Speaker 00

  The Price and probably now The New Map by Daniel Yergin. They are books about the history of oil and the geopolitics of oil. It is a fascinating way to understand the world. It's like the best books I've seen to understand geopolitics and how they work and why they work. It does it through the lens of oil, which explains way more than you might think. And so this comes from the early part of my career working in energy.

• 1:03:40Lenny Rachitsky?

  I will link that in the show notes. I've not heard of that one before. What's a favorite podcast of yours other than this one?

• 1:03:45Speaker 00

  Oh, of course. It's a pretty easy one away. Revisionist History with Malcolm Gladwell just gives you a different look into things. And I'm also a huge car nerd, like deep into modifying and tweaking and tuning cars. So there's this esoteric one called HP Academy that I'm into, but most of your listeners will not be into that.

• 1:04:04Lenny Rachitsky?

  Wow, very out there and awesome. And I think there's a new season of Revisionist History coming out soon.

• 1:04:09Speaker 00

  Yep.

• 1:04:10Lenny Rachitsky?

  Okay, favorite recent movie or TV show?

• 1:04:13Speaker 00

  Last night I discovered Mo on Netflix.

• 1:04:16Lenny Rachitsky?

  Mo?

• 1:04:17Speaker 00

  And it's short for Mohammed. It is semi-autobiographical about a Palestinian refugee living in Houston, his journey to seek asylum and live and work and date in this multicultural environment. He speaks Arabic, Spanish, and English fluently, funny as hell, but also dramatic. It is fabulous.

• 1:04:38Lenny Rachitsky?

  Amazing. Okay. Wow. These are all very unique. I love it. In a different direction, what's a favorite interview question that you like to ask?

• 1:04:44Speaker 00

  You know, these days of work, I have to go through the standard interview questions. But when I got to play and sometimes when I feel like playing a little bit more, I'll say something like, teach me something you don't think I know. It's a really good test of what you've heard me say a lot, empathy. I heard Chamak use it once and I kept trying it to see what it was good for. And it- Helps you understand how good somebody is at reading you, how much knowledge they have, and their ability to communicate and share knowledge. So it was like, it's actually can test a lot of things at once. And a lot of times you learn something. It's awesome.

• 1:05:16Lenny Rachitsky?

  Win, win. Okay. Final question. Who else in the industry would you say that you most respect as a thought leader?

• 1:05:21Speaker 00

  Well, look, on the discipline, it's Shreyas. Discipline of product management, definitely Shreyas. I think just in terms of technology development, it's the team behind Radiant Nuclear.

• 1:05:32Lenny Rachitsky?

  What is that?

• 1:05:32Speaker 00

  While taking a break between jobs, I'm studying climate change and energy because of my background. And I just basically became convinced that nuclear is a bigger answer than we're giving it credit for. A lot of the barriers are political, not technical. But the solution they're working on, I think, is a technical solution to have some of the political problems we have around nuclear, which seems really interesting. And I am like, really hoping that they pull off what they're trying to do.

• 1:06:01Lenny Rachitsky?

  Wow. I love how out there all these recommendations are. These are great. Adriel, I am so appreciative of you making time for this. I'm also really appreciative of Jules for connecting us. This was amazing. You're awesome. Two last questions. Where can folks find you online if they want to reach out, learn more? And then how can listeners be useful to you?

• 1:06:19Speaker 00

  Awesome. Before I jump into that, thank you for having me on here. It's just good to reflect about life and work for a little bit and hopefully share some insightful stuff with the folks who listen to your podcast. So thanks for having me. You can find me on LinkedIn, Adriel Frederick. There might be one other. I'm pretty sure I'm the only one. And then how can listeners be useful to me? Number one, keep listening to this podcast because if everybody keeps listening to the insights that you're teasing out, a lot of things will work well and not necessarily me. Another thing they could do to be useful to me is find somebody that's just different to you and talk to them for five minutes. That's it. I think that will come back to me eventually.

• 1:06:55Lenny Rachitsky?

  Love these and really flattered, really appreciate it. Thank you for being here.

• 1:07:00Speaker 00

  Thanks for having me, Lenny. Take it easy.

• 1:07:03Lenny Rachitsky?

  Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.