Pod Insights Y Combinator Podcast #143 - Startup School Week 2 Recap: Michael Seibel, Adora Cheung, and Ilya Volodarsky About this episode We've cut down the second week of lectures to be even shorter and combined them into one podcast.
First, a lecture from Michael Seibel https://twitter.com/mwseibel. Michael is CEO and a partner at YC. His lecture is about How to Plan an MVP https://youtu.be/1hHMwLxN6EM.
Then, a lecture from Adora Cheung https://twitter.com/nolimits. Adora is a YC partner and a cofounder of Homejoy. Her lecture is about How to Set KPIs and Goals https://youtu.be/lL6GdUHIBsM.
Last, a lecture from Ilya Volodarsky https://twitter.com/ivolo. Ilya is a cofounder of Segment. His lecture is about Analytics for Startups https://youtu.be/LLerCc7MOQo.
Y Combinator invests a small amount of money ($150k) in a large number of startups (recently 200), twice a year.
Learn more about YC and apply for funding here: https://www.ycombinator.com/apply/
***
Topics
00:00 - Intro
00:43 - Michael Seibel - How to Plan an MVP
1:25 - The goal of a pre-launch startup
3:10 - Iterating vs pivoting
3:59 - Lean MVP
5:06 - Heavy MVP
6:19 - Launching
7:35 - Learning is easier with an MVP
8:07 - How to build an MVP quickly
10:17 - Adora Cheung - How to Set KPIs and Goals
11:51 - What are the right KPIs to set?
15:50 - Revenue or active users
16:28 - Why choose active users?
18:55 - Biotech or hardtech KPIs
20:25 - Secondary metric
21:25 - What if you haven't launched?
22:20 - Setting goals
23:47 - How fast should you grow?
25:13 - Defining your own goals
28:40 - Tracking progress
30:27 - Ilya Volodarsky - Analytics for Startups
30:46 - Why analytics?
31:36 - Funnel
32:51 - Collecting data, analytics, and data flow
34:16 - Metric - Signups per week
35:28 - Metric - Retention cohorts
36:30 - Which metric to pick?
36:57 - Have I reached product market fit?
37:27 - Metric - Revenue
37:51 - Dashboards
38:35 - Advisor updates
39:01 - The startup stack
42:13 - Recommendations for the MVP process
Transcript 0:00 Speaker 00 Hey, how's it going? This is Craig Cannon and you're listening to Y Combinator's podcast. Today's episode is a recap of the second week of Startup School. I've cut down the second week of lectures to be even shorter and combined them into one podcast. First, we'll have a lecture from Michael Seibel. Michael is CEO and a partner at YC. His lecture is about how to plan an MVP. Then we'll hear a lecture from Adora Cheung. Adora is a partner at YC and a co-founder of HomeJoy. Her lecture is about how to set KPIs and goals. Last, we'll have a lecture from Ilya Volodarsky. Ilya is a co-founder of Segment. His lecture is about analytics for startups. All right, here we go.
0:43 Speaker 01 My name is Michael. I work here at Y Combinator. I helped run the accelerator. Before that, I did two YC startups, one in 2007 and one in 2012. And today, I'm going to talk to you about minimum viable product, so MVP. We always yell at founders to not use jargon, yet we have this whole set of stupid startup jargon, and MVP is one of them. When you think about MVP, you should think about something ridiculously simple. This is the first thing you can give to the very first set of users you want to target in order to see if you can deliver any value at all to them. That's all it is. It's extremely simple. Okay.
1:26 Speaker 01 So, the goal of a pre-launch startup is extremely simple. Step one, launch quickly. This is something that's been part of the YC ethos from the very beginning, and it's been great advice for 10 years, and it continues to be great advice. If you can walk away from one thing from this presentation, it's launch something bad quickly. That's it. Like, literally, the rest of what I'm going to say is basically going to be re-summarized versions of that same thing. The second thing that an early-stage startup needs to do is get some initial customers. Get anyone using your product. You don't have to have a vision of how you get everyone using it, but just anyone interacting and seeing if they can get value out of the product.
2:09 Speaker 01 You'd be surprised at how many founders' journeys end before a single user has actually interacted with a product they've created.
2:19 Speaker 01 It's very, very common. So please get past this step. It's extremely important. The next one is talk to your users, any of them, after you've launched this MVP and get feedback. This is one that's also extremely common mistake because most founders in their heads have an idea of what they want to build. And so they kind of have this weird feeling that if I haven't built the full thing yet, getting feedback on the shitty initial thing is kind of useless. Of course, it's not going to work. It's not the full thing. The full thing is going to take three years, $10 million, a whole team. So feedback on the little thing is useless. The reality is that in some ways, the full thing is this really awesome idea in your head that you should keep in your head.
3:03 Speaker 01 But it should be very, very flexible because it might turn out the full thing that you want to build isn't what your customers want at all. And last, most important, iterate. And I like to kind of distinguish between iterating and pivoting. A lot of founders, once they've figured out how to build something, fall in love with it. And so if it doesn't work for a certain set of users, they start thinking, well, I wonder what other problems this thing can solve. Well, you know, the screwdriver is not actually good at screwing in anything. But I wonder what other problems it could solve. And they're like, oh, maybe you can use it to cook. Maybe you can use it to clean. And it's like, no, like, the problem was I need to screw something in. The user was like a mechanic. And if your screwdriver doesn't help the mechanic solve the problem,
3:46 Speaker 01 keep the mechanic, keep the problem. I need to screw something in. Fix the fucking screwdriver. Like, that's the thing that's broken, right? The broken thing is not the mechanic. And it's not the fact that they need to screw something in. In most cases, most people should be building a very lean MVP. So by that, we mean you should be able to build it fast, in weeks, not months. This can either involve software or honestly, we see startups just start with a landing page and a spreadsheet. But most startups can start very, very fast. The second, extremely limited functionality. You need to condense down what your user needs, what your initial user needs,
4:30 Speaker 01 to a very simple set of things. A lot of times founders want to address all of their users' problems and all of their potential users.
4:39 Speaker 01 When in reality, they should just focus on a small set of initial users and their highest order problems and then ignore the rest until later. You should have a vision of everyone. You should have an MVP very small. All this is is a base to iterate from. That's it. It's just a starting point. It doesn't, it's not special in any way. You just have to start. And so please make sure you don't feel like your MVP is too special. In very few cases, you have to build a heavy MVP. I just invented that term heavy MVP when I made this presentation two days ago. So, uh, you know, maybe it becomes a thing. If you're in an industry with significant regulation, like insurance or banking,
5:23 Speaker 01 sometimes drones, although sometimes not, um, it's hard to launch. It's harder to launch. You have to pass through a bunch of regulatory bodies first. If you're doing hard tech, if you are building rockets, it is hard to build a rocket in a couple of weeks. Biotech, it is hard to invent a cancer drug in a couple of weeks. Moonshots, well, fill in all the other blanks. It's hard to bore tunnels in the earth and have extremely fast vehicles that replace cars in a couple of weeks. So if you're in that situation, um, please remember that your MVP can start with a simple, simple website that explains what you do.
6:05 Speaker 01 It's helpful when you talk to people and you act with people that they can refer back to something. So that can be your start and you can build that simple website in days, not weeks. So many ways, maybe your heavy MVPs are faster than your lean MVPs in some weird, strange way.
6:20 Speaker 01 Now I want to talk about launching for a second because a lot of founders have this misconception about launching. They see big companies launch stuff and they assume that's what startups do. In fact, they see companies they kind of think about like startups. Facebook's not really a startup anymore, but they see them getting a lot of press and getting a lot of buzz and yada, yada, yada. And they have in their head that that's what a successful company looks like when they launch. Well, let me ask you this question. How many here remember the day that Google launched?
6:50 Speaker 01 No. How about Facebook?
6:54 Speaker 01 Okay, how about Twitter? No. Great. So it turns out that launches aren't that special at all. Okay, so if you have this magical idea of your magical launch you want to do, throw it away. It's not that special. The number one thing that's really important is to get some customers. So to make people feel better, let's use different terms. How about launch is when you get any customers? And how about like press launch? Press launch, really impressive, is when like people write about things and it's all exciting and you get all this buzz. Let's push the press launch off and let's push the get any customers launch really, really soon. That's our goal here.
7:36 Speaker 01 It's a lot harder to learn from your customers when they don't have a product they can play with. You know, you can talk to your customer all day, but you have no idea whether the thing you want to build can solve their problem. If you put the thing in front of them and it doesn't solve their problem, you know right away. And so all the research in the world is good, but until you can put something in front of people, you have no friggin idea whether it's gonna work. So spending all that time on a pitch deck is not as valuable as spending your time building anything that you can give to a customer. Finally, some hacks for building an MVP extremely quickly. First, time box your spec. So your spec is a list of stuff you need to build before you launch. Time box it.
8:19 Speaker 01 Say, "Okay, what happens if I want to launch in three weeks?" Okay, well, the only things that could be on my spec are things I can build in three weeks. That makes your life a lot simpler. It allows you to remove all the features you can't build in three weeks. Second, write your spec. This seems really straightforward, but most people fuck this one up. It's really easy to change what you're working on before you ever launch it because you never write it down. You start working on something, you talk to a user, they say, "Oh, I would never use that." Or god forbid, you talk to an investor and they say, "Oh, that could never be a company." Because investors know everything. And so you decide to change what you're working on. And because you never wrote it down, you don't even really realize you're changing it. And so your three-week plan turns into a three-month plan.
9:03 Speaker 01 If you write shit down, at least you can be honest with yourself that you're changing your spec all the time.
9:09 Speaker 01 The next one is cut your spec. A week into your kind of three-week sprint, you probably realize that you added too many things to your spec and you're not going to make your deadline. That's okay. Just cut the stuff that clearly isn't important. And if there's no non-important things, start cutting important things. Most of the goal here is just to get anything out in the world. Once you get anything out in the world, the momentum to keep anything going is extremely strong. If you don't have anything out in the world, it's very easy to just delay, delay, delay, delay. And then last, don't fall in love with your MVP. So many people fall in love with the vision in their head and none of the products I showed you before
9:52 Speaker 01 was the initial vision, what it ended up being. So please don't fall in love with your MVP. It's just step one in a journey. You wouldn't fall in love with a paper you wrote in the first grade. And that's like the level of impact often your MVP has.
10:12 Speaker 00 All right. Now for Adora's lecture on how to set KPIs and goals.
10:17 Speaker 02 All right. So I am going to be talking about setting your KPIs and goals for early-stage startups. So I'm going to be pretty pedantic in this lecture and the reason why is doing this correctly is a necessary condition for starting a successful or building a successful startup. So the acronym KPI stands for Key Performance Indicator. If you Google around for it, there are actually many definitions of what this actually means, but for the purpose of today, for this context, I'm going to define it as a set of quantitative metrics that indicate how healthy your business is doing. So this is important because obviously you should know what state your business is in at all times.
11:00 Speaker 02 So setting the right KPIs and goals will objectively tell you if you're doing Well, just okay, or bad. So, nothing keeps you more grounded, humbled, and realistic about where you are than a bunch of numbers, because if you interpret those numbers correctly, they don't lie. It'll also act as a feedback mechanism for whether your current strategy, like user acquisition, launching new features, and so on and so forth, are actually working. So if you do something and things go up, that's probably good. If you do something and things go down, that's probably bad. And it will not only help you prioritize your time, but also course correct. So it follows if you do this incorrectly, if you set your KPIs and goals incorrectly,
11:42 Speaker 02 you can direct your startup into a bunch of circles, or if you do it for too long, it'll onto the wrong path, it'll lead to its unnecessary demise. So what are the right KPIs to set? I'm going to break this down into two pieces, primary metric and secondary metrics. And most of today is going to be focused on the primary metric. So every week in Startup School, we've asked you in the software to fill out, to define your primary metric, and then update its current value. By definition, you can only pick one, one primary metric, and as the metric if you had to, you'd be willing to bet the whole company on. So why just one metric? It's a way to focus and keep things very simple.
12:26 Speaker 02 If there's a way to get 90% of the job done with just one variable, that's better than having a bunch of variables that gets, let's say, 91% of the job done. In this case, the job to get done is quickly determining how well your startup is doing. So what are the characteristics of a good primary metric? There are four of them. One, so your primary metrics should quantify how much value you're delivering to your customer. That is, you obviously want to build something that people want, now how much do they actually want it? And users often indicate the value through either training you through money or time. So revenue is always the best metric.
13:05 Speaker 02 I pay you $100 to use your product, your software, I must at least value that $100. Active users, using the product once a week, or once a day, we call that weekly active user or daily active user, is a weaker, but another good, decent indication of whether you're delivering value or not. The second one here is your primary metric must capture whether your product has recurring or enduring value to your user, or it should anyway. So for example, in a SaaS tool, most SaaS tools use MRR, monthly recurring revenue, as their primary metric. I commit to forking over 100 bucks a month continuously every month because your product
13:48 Speaker 02 demonstrates to me every month that it has value to me. Another example is if you're building an online digital daily newspaper, then obviously DAU, daily active user, is a good one because I expect to be delivering content to you that is valuable to you every single day. So hopefully you'll come back every day. The third one here is your primary metric should be a lagging indicator for success. So a common trap that founders do to trick themselves is by picking a primary metric, let's say something like email signups, because one, it's easy to move, but while it may eventually influence revenue or actual usage, it actually doesn't represent real value the best. So the best indication is when the value has already been delivered, it's already
14:33 Speaker 02 occurred. So when someone has already forked over their time or money to use it, then that is what a lagging indicator is. So if revenue increases, it's because more customers have already paid for the product's value versus a potential customer who came to your site, gave you an email, and maybe they'll sign up one day, or maybe they'll use your product one day to buy something. And lastly, your primary metric should be usable as a feedback mechanism. That is, it helps you prioritize strategies and make decisions quickly. In a startup, one of the key things to being successful and getting past product market fit stages is to iterate very fast, right?
15:15 Speaker 02 So while you want it to be a lagging indicator, you also don't want it to lag too much. So for example, a lot of people pick MAU, monthly active user, but this is usually not a great metric because it takes time to understand the impact of movement, especially in a startup this early, as in your startup. And so many things can happen within a month. And also, another reason why I don't like MAU generally is because if your user only comes back once a month, they only value something that you're building once a month, I really question actually if you're solving a real problem. Alright, so you may have guessed from me talking about these four characteristics of a primary metric that there are really two primary metrics to pick from.
15:59 Speaker 02 So one is either revenue or active users. Ideally, you're picking revenue because nothing tells you more about delivering real value than people forking over, handing over real, hard-earned dollars to you. And even better is picking revenue that people keep giving you over and over and over again, like monthly recurring revenue, MRR. It's the best test for whether people really want what you're making. Alright, so what are reasons why... Kevin in an early lecture said 99% of you should actually use revenue as your primary metric. So what are reasons why you should consider active users?
16:38 Speaker 02 So one main one is because building a large audience is actually a prerequisite to monetization. So an example of this is if your business model is advertising-based, like a Facebook or Google, then yeah, you need millions and millions of users coming back to your site every day before you can actually get brands and people to buy ads. And so in this case, active users is actually a reasonable proxy for revenue because eventually when your startup starts making money, it's usually just... revenue is just a multiple of your active users. Another reason is also, but much, much more, much more rare, is if you have very strong network effects. That is, if you're like a marketplace that requires tons of users to just get the flywheel
17:22 Speaker 02 going and grow, then maybe that's a reason for you to focus on active users today versus revenue and then just do revenue later down the road. Now, that being said, if you're using active users as a metric, it's important that you define user appropriately. I hear often, I ask, "Okay, how many users do you... or what's your primary metric? Active users. How many users do I have?" "I have 100 users." What does users mean in that situation? Sometimes to people it means 100 users that just signed up and gave you an email. Sometimes it means 100 users that signed up and started using your product and come back every day for about 10 minutes a day, which is by far much better than just people just like diddle-daddling on your site, right?
18:04 Speaker 02 So you really need to get that definition correctly and don't trick yourself by just saying users and having a really easy definition of users.
18:14 Speaker 02 Another example of users where it's not exactly users is if you're in a marketplace and there are two types of customers, there are two types of users. So a good example is Airbnb. Who are your two users? You have not just the guests, but you also have the hosts. So what are you to do? How do you pick just one? Well, you pick a value that actually represents them both getting value. So in Airbnb's case, it would be nights booked, right? Another example is Uber. So who are your two users there? You have riders and you have drivers. And so an example of a primary metric you could pick there is weekly trips. Okay? All right.
18:55 Speaker 02 So there are always exceptions to the rules and there is one exception in which your primary metric is neither revenue or active users. And that is if you run a biotech, a hard tech business and you're still trying to figure out whether the science or tech is actually going to work. Can you actually build a product? And another definition of this is for biotech, hard tech businesses, it often takes a lot of time and money to get your first product to market. So what's a founder to do, especially you have little funding? So there's two answers to this. One is if there are no regulatory issues to doing sales per product, you should actually do the same as everyone else.
19:39 Speaker 02 It should be most likely revenue. Your primary metric should be revenue in the form of paid contracts, LOIs, POCs, proof of contracts, proof that if you build it, they will actually come. Now, if you are in a space with regulatory issues, meaning you can't sell it at all without having to go through like FDA or some kind of body like that, then your primary metric is actually less quantitative per se and more of a binary thing. So it's about figuring out the technical milestones that you need to demonstrate to mitigate the risk of whether the drug or tech is working. So if you have to think about experience to prove this out, you can ask a question like, what are the minimal things I need to do to truly answer the question of whether
20:23 Speaker 02 this works or not? So, one suggestion I have is to select a set of three to five other metrics, secondary metrics, to pair with your primary metric. This gives you a good 360-degree overview of the health of your company. So, there are a ton to choose from, so many to choose from. What you choose is actually very dependent on your business. Next week, we're going to have two lectures on these sorts of metrics for consumer. I'll be giving one on consumer startups, and another YC partner I knew will be giving one on B2B companies. And so, we'll deep dive into the metrics, these metrics, next week. The key here, though, is just picking a few, right?
21:08 Speaker 02 At most five, three to five, closer probably to three. You don't want to boil the ocean and pick everything. It's totally fine to track all this kind of stuff, but it's really not a good idea to optimize too many at once. You'll really just suffer from analysis paralysis. All right, so, a common question I have when I say, "You should, what is your primary metric, or you should set one," is, "Well, what if I haven't launched yet?" Well, obviously, metrics don't matter if you don't know what the problem you're solving is, you don't even know who your customer is yet. You should really just focus on that first. You'd be really putting the cart before the horse by worrying too much about this kind of stuff. That said, once you get to the point where you're building the product, it's really a good idea to get this down, even if you haven't launched yet.
21:52 Speaker 02 By at least defining your primary metric, you'll be able to think about who your user really is. You get everyone on the same page on who you're targeting, and even you can use the metrics and goals to hypothesize on how you might get your first few users. And trust me, nothing is more motivating than staring down the barrel of zero users and zero dollars of revenue for weeks on end. You're going to get very antsy about launching very quickly, and that's actually the effect you want. All right, so I'm going to go into how do you set goals for your primary metric for your KPIs? So Paul Graham actually wrote a great essay a few years ago called Startup Equals Growth, and it explains why startups should focus on growth, and I really urge you to go read it.
22:37 Speaker 02 And this section of this lecture draws a lot of insights from it. The goal of your startup is to grow your primary metric. By doing this, it does two things. It proves that you're making something lots of people want. And second, it proves you're making something that has the possibility of reaching and serving all those people. Each week, your goal should actually be to set a weekly growth rate. Now, we use weekly increments because startups early on need frequent feedback from their users to tweak what they're doing. But also, we use weekly growth rate because it helps you divide up the progress you need into doable chunks. So say your goal in a couple of months is to get 10,000 daily active users, which requires growing new users, let's say, 10% week over week.
23:21 Speaker 02 To grow 10% this week may amount to actually just getting 100 new users, which is a different problem to solve than trying to get 10,000 new users, right? You should be focusing on what's directly ahead of you in that week. Do things that don't scale today if that's actually the best way to get those 100 users. And don't worry about the eventual goal of 10,000 too soon. So naturally, the next question is, how fast should it be growing? What should this rate actually be? Well, there's no good formula, there's no right formula for this, but one angle that we could tackle it from is looking at good startups and seeing how fast they were growing in the beginning stages of their life.
24:06 Speaker 02 So I actually went back and I looked at the good startups who pitched in recent YC demo days. So if you think about these startups, they were, three months prior, they were all in the phase that you are probably in today. And it turns out the growth rates ranged anywhere from 20% to 200% month over month, but clustered more closely to 20% to 50% month over month. Which if you back it out, it amounts to about 5% to 10% week over week. So growth is a little hard to grok, but if you look at this chart, you'll see that how small variations in weekly growth rates can make a huge difference on the monthly and yearly time horizon.
24:44 Speaker 02 You also get the sense that to get big fast, it actually seems doable if you have something people want. On the flip side, if you can only manage 1% weekly growth, it's a sign you haven't figured out things yet. It doesn't mean that you have a horrible business. You can run a great, small profitable business, growing 1% week over week, but it's not a good sign that you have a startup with a billion-dollar potential. So you should think about that trade-off there and what you really want out of your business if you're growing at that rate. That said, the main thing in terms of setting your goals is to think for yourself, is to define your own goal based on not what others are doing, but what you think is ambitious and achievable based on the products you're building.
25:28 Speaker 02 So you know your users and business better than everyone else. What does success look like for you? And what does being on track look like to you? So here are some general guidelines when defining a goal. All right, first, if you're solving a real problem in a large market, then that means there's a ton of latent demand out there. People would use just about anything to use your product, even if it's half-broken, half-baked, or just solves a bit of their problem, which means that startups usually have fast initial growth. That said, where you are today matters. So if you have a ton of users and a ton of revenue, you will probably know that at that volume, as a volume increases, what you need every week to grow gets harder over time.
26:11 Speaker 02 So again, most startups, they grow very quickly and then over some time, they kind of, the growth rate kind of slows down a little bit. The second one is time to sale. So when you're trying to set your goal, you need to consider how long it takes to acquire a user and make a sale. So for a consumer startup, generally you have an app or a website. I show up to it, I look at it, I see if I want it, and then if I do, bam, I buy it, or I sign up for it. And so it's instantaneous. For an enterprise startup, where you're actually probably going through some red tape, you have a bunch of stakeholders you have to deal with, like you can show up to the company and they're not going to even buy it right away because you're maybe not even talking to the right person. So it might take some months to actually get your first sale.
26:56 Speaker 02 So you'll have to take that into account. Over time, this time to sale should actually decrease over time. Like good enterprise startups, that time to sale goes from months to hopefully days. If not ours, and so it shouldn't impact your growth rate in the future, but in the near term, it actually might. Third is you really want to focus on organic versus paid users, or paid growth in the beginning. Organic means they discover it through word of mouth. Basically, you're not paying for the user. They kind of just may be searching for it and using it themselves. I think in the early days, using paid users is actually cheating growth, and you should avoid it as much as possible. And finally, because you're a startup, startups equals growth. You should focus on exponential goals and not linear goals.
27:43 Speaker 02 All right, so in terms of picking the goals, I think there's two ways to do it. One, you can just pick a growth rate,
27:51 Speaker 02 and then pick a growth rate that you can, you think you can hit. And if you hit it, great, you probably should change it if you're hitting it consistently to something higher. If you don't, if you're not hitting it, then you should be a little bit alarmed, and you should figure out why. Another way to do it is time box an absolute goal. So what I mean by for that is, say, for the purpose of startup school, at the end of startup school, how many active users or how much revenue do you want to have? What would, what would it look like? What would something meaningful look like at the end of 10 weeks? Then go back out your weekly growth rate, and then go week to week, figure out the obstacles and how you should hit that, hit that weekly goal. In the beginning, if you're somewhere close to zero users today,
28:34 Speaker 02 often you'll get something higher if you, higher if you do this method than 5 to 7% week over week. Tracking progress. So metrics and goals obviously don't mean anything if you don't leverage them. Use these as a motivational tool. So one way to do this is get a piece of paper, draw a forward-looking graph of what the growth you want to hit in the next 10 weeks, print it out and put it everywhere. Put it on top of your desk, put it on the bathroom mirror, put it on the fridge, and update once, once, once a week. This is in fact what Airbnb founders did in the beginning. And if they hit the numbers, great. If they did not, then that's all they would talk about. And so I would follow something to like this. Now you want to leverage your prior metric and goal to help you prioritize your
29:19 Speaker 02 time week over week. So week to week, you should be stack ranking all the ideas you have of how to grow it and make a good guess on what's going to have the biggest impact for the next week to meeting your goal and then choose accordingly. Occasionally, you won't hit your goal for the week. We can dream that our growth will be flawless and look like this, but in reality, in the beginning, it always looks something like this. It's okay if you don't hit your goal one or even two weeks in a row, as long as you understand why. You should be always asking yourself, what is the biggest obstacle in my way of hitting my weekly target? How do I overcome this and be obsessive of this?
29:58 Speaker 02 If you don't know the answer, then the answer is go talk to more users and don't spin in circles trying to figure it out yourself. A good startup idea will keep growing at some point, so not hitting your weekly targets week on end will maybe just help inform you you're not working on the right thing or even the right idea.
30:23 Speaker 00 All right. Now for Ilya's lecture on analytics for startups.
30:27 Speaker 03 Hi, everyone. My name is Ilya. I'm one of the co-founders at Segment and I'm here to talk to you about how to set up analytics and analytics foundation to build your MVP and to measure these primary and secondary metrics. So this is going to be a little bit more of a tactical guide around what tools are there in the analytics space and the marketing space? Which one should we actually be using? How do I set them up? So why talk about, why even focus on analytics? So obviously, primary and secondary metrics drive the MVP and product market fit process and you're using that to actually test product market fit. You're also using it once you get out of product market fit strict search to actually focus the team. So maybe there's going to be an acquisition issue in the company that's preventing your growth, or maybe the users that you're getting aren't as engaged,
31:10 Speaker 03 or maybe you're having some monetization issues. And so the funnel actually is a forcing function to understand your business and where founders should be actually spending their time. And then finally, all the way from, you know, two, three-person team to a Google with 1 million employees, you're actually using metrics to operate and drive teams. So eventually, you have an engineering team, you have a marketing team, and so what goal do you send in front of the marketing team? Use analytics for that. Okay, so to start, we started the funnel. So we'll make an example funnel for Netflix, which is a company that we're all super familiar with. Any B2B product or B2C product actually has this type of funnel, where you acquire a user, you engage a user over a period of time. That loop is called retention.
31:54 Speaker 03 And then finally, you monetize the user. And then, metrics, both primary and secondary, are performance indicators on top of each stage in the funnel. So on top of acquisition, you can ask yourself, how many net new users did I get this week versus last week? And what's my growth rate there? For engagement, you take a cohort of users. So from Sunday to the following Monday, you have 16 people sign up. And then you can track that cohort of users week over week and see what percentage of them are still using the product four weeks later, which is a good example of how to track retention. And then we talk about monetization, which is how much net new revenue did I make this week versus last week? Okay, and then you apply your own custom business funnel to this.
32:38 Speaker 03 So if you're Netflix, we're all familiar that users sign up for Netflix. Then they play videos in a loop. Netflix is obviously very sticky, watching it a lot. And then finally, when the trial runs out, you do subscription upgraded and you get access to more content. Okay, so how do you collect data once you have this funnel? So there's analytics APIs out there. I'm using Segment as an example. You basically want to say, user, user 1, 2, 3 in this case, has done user signup event. And they happen to be an organic user, which means they're not invited by someone else. Then, if you're Netflix, you might say the user is video played and eventually subscription upgraded. And so this is how you instrument your tracking in your mobile app or your web app. Then you think about event properties. So imagine you're Netflix and you're holding
33:23 Speaker 03 one of these video played events in your hands. And you're wondering questions about it. So what video is the user actually playing? How long is the video? How far did the person get inside of the video? Right? Equivalently, if you're holding a subscription upgraded event, you're going to want to derive monetization as a North Star metric. So if you're a subscription business, you want to send your monthly recurring revenue. If you're a transactional business like e-commerce or retail, you want to send the actual value of the transaction. Okay, so then you push this out into your web app, your mobile app, then you start seeing the data come in. You look at the debugger, you see, okay, user signup is here. Everything looks good. You add your first analytics tool. We'll use Amplitude as a good example here. Amplitude and Mixpanel are pretty awesome analytics tools out in the market right now. And then you start seeing data flow inside of one of these analytics tools.
34:08 Speaker 03 So this is Amplitude. You can start seeing user signups growing as soon as you launched the real mobile or web app. Okay, so now that you have analytics set up, it's time to focus on three different metrics. The first one is the acquisition metric, signups per week. It's really nice, if you're a B2B business, to cut this by the invite type. So you have organic users, which are just signing up from coming to your website, direct signup, and then some users are inviting other users. So those are invite type, right? So when you're thinking about growth, it's really important to think about the organic user in that case, right? Another example of why event properties are important. The way you create this is you go to Amplitude, you say "event segmentation report, user signup, next." Here's your graph, right? So it's as easy as that.
34:51 Speaker 03 Just website, send data, gets to Amplitude, and then you can see the amount of organic users every week. And then if you're working on the acquisition step as a secondary metric, you can basically say, "Today, 218 users organically signed up in the last week, but by the end of the month, we want that to be at 300. And we're going to execute projects A, B, and C this month, and then we're going to watch this graph every single day on a TV dashboard in our office or our apartment, wherever we work. And then we're going to see if our efforts are actually driving this, right? So that's an idea of data-driven operation of a team. You set a metric and a goal, and then you drive towards that every day. Okay, the second one is retention cohorts. So someone recently asked about retention. We'll talk about that right now.
35:35 Speaker 03 So with retention, you want to think about cohorts of users. So you want to say, "Monday to Sunday, let's say, December 10 through December 17, 16 users signed up." And then you look at those 16 users as they use your product on week zero, which is their signup week, week one, week two, week three, and week four. And the general idea here is like you can convince your Or your grandma to use your product once, but even your mom or your grandma won't continue to come back and use your product over time every single week, right? And so if you see users that are addicted, that are coming back week over week, that's a really good sign of product market fit. So this business can see that the December 10th cohort, only 6.25% of those users are still around on week four. And that's a pretty low amount, right?
36:18 Speaker 03 So you probably want that to be somewhere between 20 or 30 at least. And so you can set a goal saying, "I'm going to talk to a bunch of these target users and try to figure out, you know, why they're not getting value out of the product and then make some changes as well." Okay, so what metric do you actually pick? This is taken from one of Gustav's slides, pretty awesome. You think about what value your company is giving to your users. So Airbnb gives you value by letting you stay at different rental properties around the world, right? And they want you to do that at least one time a year, otherwise they consider you a churned user. Equivalently, Facebook gives you value by letting you look at the news feed and, you know, connect with your friends. And they want you to do that at least daily or monthly once. And so when you think about product market fit, you basically have these two different curves that happen.
37:02 Speaker 03 So we have that cohort of 16 users that signed up in one week and we track them over time. And so what ends up happening is for products that don't have product market fit, they end up tending to go to zero because people just don't care about the product, right? And that's definition of product market fit. For those tools and the products that do have product market fit, you'll see some kind of natural plateau. Don't mind this axis, it should be somewhere, you know, between 20 and 30% at least.
37:28 Speaker 03 Okay, finally, revenue. So this is the primary metric that you want to be thinking about. You'll use for a subscription business, the subscription upgraded event. You'll do a property sum over new plan monthly recurring revenue. You press next and out comes your weekly net new revenue graph. And then you could set monthly goals on top of this to make sure you're growing at the rates that you want to be. Okay, finally, if you have a founding team, it's really good to basically put all the stuff on a dashboard, and then put this dashboard on a TV in your office. It's incredibly, incredibly important. Basically, this is kind of the difference between being a data-driven team and not a data-driven team. A lot of founders actually set up their analytics, but then don't look at them ever again because it can be painful, right?
38:11 Speaker 03 While a data-driven team will put it on a TV and talk about projects, talk about whether those projects are actually changing the metrics that they're trying to drive, and then just completely understand the business every single day, right? And this kind of company and this kind of founder will actually scale to build better, high-performance companies because the next team of employees they hire will also be looking at that same TV dashboard and be driven off those same metrics. So, really important, get a TV. Next, what you want to do is have some kind of social accountability around your metrics. So, if you have your friends, your parents, your advisors, your investors, package up how your business is doing into an email. This helps you synthesize what is actually happening, and then send it out to those advisors, and tell them where the business is struggling and what your plan is to fix it.
38:55 Speaker 03 This allows the advisors to quickly understand the business and then respond back with much more appropriate advice.
39:02 Speaker 03 Cool, and now we'll go into the startup stack. So, these are tools that we recommend that help this kind of tactical process of setting up these metrics. So, I'm going to talk a little bit about that MVP business workflow that Michael talked about earlier. So, initially, you're building an MVP. Segment built about seven different MVPs before we actually found Segment, and all of those failed, and eventually we found one that worked. And the process of actually building MVP is incredibly important. So, once you have that little experiment built, you want to enter private beta, which basically just means getting like 10, 20, 30 customers to actually try this product, and then having very direct lines of communication open with them. What Segment does nowadays, every new product we ship,
39:46 Speaker 03 we open Slack channels with each one of our customers, and we have the product managers sit in those Slack channels and talk with the customers. For the products that don't get product-market fit, the customers just stop responding, and we're asking, asking, asking, they're not responding, right? And for the products that do have product-market fit, the customers are immediately being like, "Oh, why don't you have this feature? This is broken. I tried inviting my teammate, this is not working." So, instead of you kind of...
40:12 Speaker 03 Pulling at the customer, the customer starts pulling at you. That's a good feeling of product market fit. Okay, so at some point, the private beta is going well. You feel like people really care about this. You understand your target customer. Then you want to get a larger market segment to use it. That's the launch that we talked about earlier. Try to get there as quickly as possible. And then a launch is just more users that you get to test product market fit on. And so if you feel product market fit there, then you can start scaling the company, right? And you hire salespeople and you start doing paid marketing and things like that. So different tools will guide you throughout this process. So as you're building an MVP and you're about to give it to the first group of customers, install Google Analytics, install Amplitude. Google Analytics will tell you who's coming from the internet to your website.
40:55 Speaker 03 And then Amplitude will tell you which features are they using? How engaged are they with that feature set? Unless you're able to stand over the shoulders of all of your users 100% of the time, analytics is the next best alternative for that. We also install live chat on the page. So either Slack with your customers, or if you can't do that, then maybe have a live chat available. In the beginning of segment, customers would ping us day and night. And that's where we got the most valuable feedback from them. So just as many open channels of communication as possible. Next, data warehouse. This is something that we recommend. It used to be expensive, is no longer expensive today. Basically, if you have a non-technical co-founder on your team, they'll want to ask questions around the data. And they'll always ask the technical co-founder, who will have to provide the answers.
41:40 Speaker 03 So data warehouse kind of democratizes the data, not only for the non-technical co-founders, but for everyone else in the company that you hire after. Company dashboards, obviously, we should probably move that to the left. Email and push tools. So as soon as users sign up, you want to send them an email. I'll talk about that in a second. And then a help desk. So at some point, you'll have so much support tickets if you start feeling product market fit. And if they're all going to your Gmail, one founder will just get overwhelmed and not be able to answer them. So you want to have a shared inbox where multiple founders can respond. Okay, this is my recommendation of what tools are the best for the MVP process. I'll walk you through them right now.
42:21 Speaker 03 So, Google Analytics is for understanding what users are coming to your website. Amplitude is for feature analytics. Google BigQuery is to democratize data access with a data warehouse, which is just a database of data. Mode is the tool you use on top of that to ask questions on top of BigQuery. Intercom is like a list of all of your customers. They're really good CRM for early-stage folks. Fullstory is for improving product usability. And Customer IO is for emailing your customers. Once you get big enough, you can start using Google Ads, Facebook Ads, to actually do the paid acquisition.
42:56 Speaker 00 All right, thanks for listening. So, as always, you can find the transcript and the video at blog.ycombinator.com. And if you have a second, it would be awesome to give us a rating and review wherever you find your podcasts. See you next time.
Previous #142 - Startup School Week 1 Recap: Kevin Hale and Eric Migicovsky Next #144 - Startup School Week 3 Recap - Anu Hariharan and Adora Cheung Page updated October 26, 2026 · transcript and summary are machine-generated