20VC Roundtable: Spotify, Adobe & Linkedin CPOs on How AI Changes The Future of Product, Why AI is Now the Product, How TikTok Changed Product, Why Cost is the Biggest Barrier to LLM Usage & Why Incumbents Can Adopt AI Faster Than Any Prior Innovation Cyc

Gustav Söderström is the Co-President, CPO & CTO at Spotify. Gustav has been instrumental in taking Spotify from a 30-person operation in Sweden when he joined to being the global leader of the space. Scott Belsky is Adobe’s Chief Product Officer and Executive Vice President, Creative Cloud. Scott oversees all of product and engineering for Creative Cloud, as well as design for Adobe. Tomer Cohen is the Chief Product Officer @ Linkedin where he is responsible for setting and executing the global product strategy at LinkedIn. In Today's Episode on How AI Changes The Future of Product and Design We Discuss: 1. Why AI Is Now the Product that UI Serves: Why does Gustav believe that AI is now the product? How has the importance of UI changed with the rise of AI? How did TikTok change the product paradigm over the last few years? 2. What Matters More Models or Data: What is more important the size of the model or the amount of data a company has? Will companies use many models at the same time? Why will companies using many models at once create a huge opportunity for startups? Will every company have their own model? What will be the decision-making framework of whether to have your own model or leverage another? How does the rise of AI change how companies approach data acquisition, collection and cleaning? 3. The Workforce Needs to Change with AI: How do product leaders and teams need to change in an AI-first world? What do designers need to do to stay up to date in an AI-first world? What does it mean to be good at prompting? How can people get good at prompting? Why will AI kill companies that charge by the hour? Why will seat pricing die in a world of AI? What will be the business model for AI? 4. Incumbents vs Startups: Who Wins: Do incumbents win in a world of AI or do startups? Why is AI primed for incumbents to win and move fast in a way they could not in prior technology cycles? What are the biggest hurdles and challenges incumbents have to face that startups do not? What are the biggest barriers that startups have to win in a world of AI that incumbents do not have?

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Now AI is the product and the UI is there to to help the AI.

• 0:03Speaker 01

  When you think in an AI-first principled kind of way, you're really unleashing the idea of control, in my opinion. You don't control the experience anymore.

• 0:11Speaker 00

  Designers, they need to understand basically GPT-4 as well as they understand a user. Even though the models are there to generate great voice, the problem is actually doing it cost efficiently.

• 0:20Speaker 01

  I can't underestimate how much intelligence even art goes into building the the right prompts.

• 0:26Speaker 00

  The big changes come when that technology enables a new business model somehow.

• 0:30Speaker 02

  The functions of so many businesses are collapsing now. And so the idea of selling seats function by function, I mean, is such an old antiquated way of building a business to some degree.

• 0:39Harry Stebbings?

  This is 20VC with me, Harry Stebbings. And the roundtable shows that we've done so far have been the most popular that we do. And so today, we have three of the world's best chief product officers discussing how AI changes the future of product and design. So in the lineup today, we have Gustav Soderstrom, CPO, CTO and co-president at Spotify, Tomer Cohen, CPO at LinkedIn, and then Scott Belsky, CPO at Adobe. We could not have a better lineup. This was so much fun to do. It's a total masterclass. Let me know what you think of this format of show on Twitter at Harry Stebbings. But before we dive into the show today, HMC, or Harvard Management Company, they're constantly seeking out the next generation of truly great investors and entrepreneurs.

• 1:22Harry Stebbings?

  HMC has managed Harvard University's endowment for nearly 50 years and was one of the first institutional investors in venture capital. Their experience and long-term investment horizon make them ideal partners to get world-changing ideas on a path to viability and success. They work as a true partner, providing insightful perspectives to help managers succeed. I personally have had the pleasure of working with the HMC team and can say that they're truly exceptional partners and just great, great people to work with. Whether you're launching your first fund or your fifth, HMC welcomes the opportunity to partner with both developing and established managers. Have an idea you want to share with the team? Just send it to [email protected].

• 2:03Speaker 04

  You have now arrived at your destination.

• 2:06Harry Stebbings?

  Alrighty, I am so excited for this. I've wanted to do this one for a while. So we're going to do a couple of intros first so everyone can get familiar with each other's voices. And so we're going to start with you, Scott, then Gustav and then Tomma. Who are you and what are you most well-known for?

• 2:20Speaker 02

  I was the founder of a company called Behance back in the day. We were acquired by Adobe. For the last six years or so, I served as Chief Product Officer for about five years overseeing the Creative Cloud business and then have recently taken over strategy, corporate development, design and emerging products for the company in this kind of chief strategy, emerging products officer type role over the last year.

• 2:40Harry Stebbings?

  It's quite an all encompassing role. Gustav, your turn.

• 2:43Speaker 00

  So I'm Gustav Sarduschtam and similar to Scott, I started and sold a few companies and then I've worked at Spotify for almost 15 years now as CPO and CTO and now recently as co-president together with Alex Nordstrom. So we run the company together.

• 2:59Harry Stebbings?

  Toma, your turn.

• 3:00Speaker 01

  Hey everyone, Chief Product Officer for LinkedIn, responsible for basically what we build from the company's strategy to overseeing the teams building it. Been at LinkedIn for 12 years now, joined right at the cuff of shifting from desktop to mobile as a company. So that was fun. And then I started my career as an engineer. So I was doing anything from semiconductors, chip on design, algorithms to embedded systems, all the way to the internet right now in AI.

• 3:24Harry Stebbings?

  Okay, so now everyone knows individual voices. We're actually also very lucky because there are three very distinct voices. I didn't realize quite how apt the selection of this roundtable was in that perspective. I want to start with probably kind of the most broad, but also kind of important, which is fundamentally, we're all told that AI changes everything that we do. When we apply that to product, what do we think are the most significant ways that AI will change the product development process, specifically on the product development side? What will change specifically with AI?

• 3:52Speaker 02

  Well, I mean, I'm sure we're all experimenting with GitHub Copilot in our engineering teams and there will certainly be an improvement in both the productivity of developing code and products and also testing and identifying and reconciling bugs and that sort of thing. But the only other thing I would throw in there is that when you're developing the interface of a product, you are oftentimes trying a few different approaches to find the one that works best. And that just takes a lot of time to have three or four different options and explore, but then stop going down that path and go down another path. And then fast forward now into today, into the future. We're in a world where AI will suggest alternative scenarios.

• 4:34Speaker 02

  You can try variations and just look and, you know, with the mistake of the eye, find a better solution or at least something you want to A B test. And so I think that the product development process, the products will become much better, frankly, with fewer cycles, we'll find better solutions.

• 4:49Speaker 00

  I have some thoughts there. I mean, one way to maybe think about it holistically is that, like, when I started and then Tomer just referenced this, when, when it was a transition to, to mobile and even before then, at least I kind of thought of the UI as the product. And then machine learning came along and the machine learning was there to help the UI. And that shifted completely. Now AI is the product and the UI is there to, to help the AI to capture better signal. And I think, I think most people agree that the real transition there was TikTok, which is like almost no UI. It's just the video and the UI is just trying to make the export export algorithm be more efficient than anything before it. And, and, you know, most people in tech agree that it wasn't really the algorithm of TikTok. It was the UI that maximized to what was pretty traditional export export algorithm, like since the day so hot or not.

• 5:34Speaker 00

  So I think that's a big change, like the AI is the product and the UI is there to help the, the AI these days. And maybe it always was. I mean, users never came to Spotify to click buttons. It was always about your music or something. It's just more clear now.

• 5:47Harry Stebbings?

  I've had so many guests on the show before say about the stupidity of UI and how AI will make UIs redundant. Is that wrong?

• 5:54Speaker 00

  Go, you go.

• 5:55Speaker 02

  I mean, UI is here to help us understand the object model of a, like, you know, find what we want, you know, and, and with as little friction as possible engaged. And so, you know, UI in some cases should disappear, but there's new UI these days in the form of the tone of a product. And the way that something is, um, is the inflections that are used, the persona. I'm like thinking about UI design in the future as in some ways like persona design. When I'm interacting with, uh, with Spotify's DJ and the DJ is like cracking jokes and like being casual with me, that's a UI decision to some degree of a product that's driven by AI as an example.

• 6:34Speaker 01

  One of the biggest implications for me when building product is the understanding that AI is the product. But for many product builders, AI is something they delegate. I think that's like really delegated to the AI team or the engineering team. I think AI strategy starts from the CEO and makes its way down. And with many companies right now, AI, that's something that the team is doing. Reminds me of like, early on with the mobile team, there was a mobile team building it. And it was a whole different, the rest of the company was building something else. Like the analogy I give usually is like, imagine like a rerafting, a river rafting boat. The guy on the back with the two massive pedals, and that's AI today in product. Then everybody else on the side, they had accuracy, they had some power, but they're not as significant as the guy on the back with the pedals.

• 7:16Speaker 01

  And one of the biggest changes I've seen with AI is we moved from desktop to mobile. Not everybody made it for the transition because it required you to unlearn how to build a little bit. It wasn't about hedging, I'm gonna put all these things there and what's the kind of hit map. It was about making a decision about how do you build a relevance experience. When you think in an AI-first principled kind of way, you're really unleashing the idea of control, in my opinion. So what happens basically in AI is you don't control the experience anymore. It's not deterministic anymore. For many product leaders I know, it's really hard to let go. It's like you're a chef at the restaurant and all you dictate is the ingredients, maybe the knobs a little bit, but you don't control the output. I can't tell you how many people just flop on that.

• 7:58Speaker 01

  They just cannot comprehend the fact that they don't control the experience.

• 8:02Speaker 00

  I like that.

• 8:03Speaker 01

  Massive formation of AI-first mentality.

• 8:06Speaker 00

  Probabilistic experiences rather than deterministic, right?

• 8:09Speaker 01

  Hundred percent. It's not deterministic.

• 8:11Harry Stebbings?

  Do you guys not find that inherently concerning? Every company on this call is a public company. When you don't have the control of the outcome and the outcome could be a hallucination. Is that not concerning to you when you don't control the outcomes?

• 8:25Speaker 02

  Well, it's funny. I mean, for some practices hallucination is a bug, but in some areas it's also a feature. If I'm trying to discover some cool new music, I mean, I can imagine that might be a feature as opposed to a bug. You know, when you're trying to do generative fill in Photoshop and imagine what's behind an object or, you know, extending the frame of a photo, hallucination is actually a feature. And if you get the wrong answer or something you don't like, you can just run it again. So I think it is a bigger problem, though, in applications that really, you know, are mission critical around, you know, writing NDAs, you know, for legal purposes, or where hallucination actually could get you in trouble.

• 9:04Speaker 00

  I want to build on something Scott said there because I think it's the definition of what is UI changes. So you took the example of the AI DJ we did, and you're exactly right. It was the design team that user tested. What is this? Is this a utility like Google? Is it an AI person like Alexa? Or is the UI that we digitized a real person with a real personality that exists? And we chose the latter. That is design and it is brand. So I think designers need to think holistically about the experience in that sense, and that is still design. It is the user experience. And I also think something that designers need to get good at in this world is to understand the capabilities they have. They need to understand basically GPT-4 as well as they understand a user.

• 9:47Speaker 00

  What it needs and how it works. So an example that I think is a good example is Midjourney. So Midjourney, the designers there clearly understood the performance of the model when they built the Midjourney experience, right? It took like two minutes to generate an image. It was wrong one out of four times. They could have built a horrible experience. We waited for two minutes and got disappointed 75% of the time. Instead, because the designers there and product people understood the capabilities, they built a fault tolerant UI. They gave you, because they also understood how diffusion works, so you can do it in stages. They gave you four low-res shots at the same time in 20 seconds. Is that, is any of these good enough, right? So that's what I think design needs to do. Designers and product people need to understand the models and the performance very, very deeply so that they can make sure the experience matches the current level of performance.

• 10:32Speaker 00

  It's the same for us. If we have a one in 10 shot in A good song, we probably need to recommend ten on the screen at the same time to get like a good chance of a hit, right?

• 10:41Harry Stebbings?

  Gustav, you said there about kind of understanding the models very deeply, but again, I am basically an absorber of knowledge from smart people. And then I try and kind of amalgamate it together. Everyone tells me that actually every company or the best companies will use multiple models at the same time and transition between them. How will product people be able to have eight different models in use at the same time, know all of them really well to utilize them effectively? How does that look?

• 11:06Speaker 01

  I don't think it's product people. I think this is where you build like a platform that basically enables you to understand the task you're trying to accomplish. Cost, we should talk about cost at one point because this is a very costly software and costly technology. And then that's actually what you're being mass people with, deciding what kind of model you use for what purpose really starts at the application layer. But the decision is not made at the application layer. That's actually, if I think of like why I would expect to see some massive innovation and startups to show up. It's in this tier to really allow people to leverage multiple models at multiple cost centers and resources and efficiencies. And then completely mask it from the developer, from the designers or the product folks.

• 11:47Speaker 02

  I think it's actually common belief that there are going to be a few mega models in the future that are going to do everything for every company in the cloud. And I think what we're saying is actually it will probably be the opposite, right? There will be many, many, many long tail models, some learning locally on people's machines or in applications that they install, some in the cloud, some open source, some not open source. And there needs to be like logic around the routing, you know, not only to the model that's the best prepared or, you know, the most specialized model for a given query, but also the most cost efficient. And a lot of companies like ours, you know, in market with AI products today are actually, you know, counting on the reduced costs over time, you know, from some of those technologies.

• 12:30Speaker 02

  So I think that's a great point. And I actually, you know, there are some startups now that are kind of defining themselves as like router, you know, startup.

• 12:37Speaker 01

  There's like two levels, I think, which are kind of fun to think about. One is at the application layer and one is at the kind of more of the kind of meteor layer, which is like the dispatcher analogy. And there is a dispatcher at the kind of application layer from the application. Like imagine the idea of agents, right? You come in, we all know by now, building one agent to rule them all is not the right design. You want to have multiple types of agents. Like I take a complication like LinkedIn, there's like a seller agent, a knowledge agent, a job seeker, there's so many agents you want to build. But you want to build as a dispatcher around that, that almost like a team coach that knows like how to get that team to work together. Same at the meteor level, when you want to start building some kind of dispatcher or router to Scott's point, around which technology to best use.

• 13:19Speaker 01

  That's where I could imagine tremendous innovation happening.

• 13:23Harry Stebbings?

  When we think about the multiple models that we'll use, you mentioned there that we'll have actually kind of open, closed, very specialized. When we think about like cost, the more models we have, the more costly it is also, and margins are impacted. How do we feel about cost and implementation when thinking about model adoption?

• 13:40Speaker 00

  From Spotify's point of view, most of the stuff we've built in-house, to Tomer's point, is actually mostly about cost. For example, as we talked about, generating a lot of voice. You want to generate like two minutes of voice per day for half a billion people. It's a billion minutes that can ruin you. Like cost is actually already like the biggest factor in if you can deliver products. Even though the models are there to generate great voice, the problem is actually doing it cost efficiently. So I think that's completely right. A lot of the innovation and technology has to go towards routing, often actually for cost purposes. I do think I have a little bit of a different view on, there is some chance that per company, I think you will want to embed the entire user history in one model. All right, so today, most companies, including Spotify, many separate systems that are optimized

• 14:24Speaker 00

  for different purposes, and they sort of collaborate in semi predictable ways. But it's not one system. But if you look at some of the papers coming out of Amazon and so forth, they are starting to look at, you literally tokenize all of it. Basically, the log, the how you scroll, how you click, what you listen to, what was in the content that you listened to, you just do a token prediction based on that, based on all of this, what is going to happen next. So we may still see like one large model. I think the world is going to go towards you embed your entire user history, everything they did into one space. I still think to your point, Scott and Tomer, you will have specialized models for, for example, for voice, for video, for different purposes. So it depends on if you're talking about the user data or about generic capabilities, like rendering and voice and so forth.

• 15:07Speaker 02

  Yeah, and in some cases, I guess you could take the user data at the router level and kind of, you know, go to different models for different purposes.

• 15:13Speaker 00

  Exactly. You can embed that.

• 15:15Speaker 02

  Right.

• 15:16Speaker 00

  You know, as a user history.

• 15:17Speaker 02

  I think Gusav's point about the cost efficiency, the good news is that the desire to have more performative models and the desire to have more cost efficient models are often like parallel efforts, you know, towards the same outcome. And so, you know, the margins will get better as these models get better also.

• 15:35Harry Stebbings?

  Do you think this follows standard Moore's law theory in terms of development of technology and the cost reduction that we see there?

• 15:42Speaker 00

  I think it will. You know, some people say Moore's law is starting to come to an end in terms of just more transistors, but we barely started on neural hardware. So I'm sure we're going to see the same effect, even if it's not more transistors per square inch necessarily, for a good while. I also think we'll start, we will continue to see bigger and bigger models, but there is the opposite pressure as well. Every month, there's a much smaller model that did what the bigger model did a month ago better. So both things are happening at the same time. So I do think we'll see that progress for some time.

• 16:09Speaker 01

  No, coming from embedded systems, you can already see verticalization of software with chips dedicated where, you know, Microsoft announced they're going to start building this. You can actually see like the idea of how do I get that efficiency and gains in resources, in power and costs. You know, when I was building semiconductors a long time ago, it was like every chip was about how do I get, you know, in half the power, double the capacity and like lower the cost. So that was always the kind of mindset. And I think you can start seeing that verticalization. This goes to open versus closed. There's so many implications of that. But if you're going to that closed, highly resourced optimized system, you'll start seeing that from companies and Apple and Microsoft and so on.

• 16:47Harry Stebbings?

  Each of you have incredible amounts of data, LinkedIn, Adobe, Spotify, insane amounts of user data. Everyone puts the question forward of what comes first, the size of the model or the quality and size of the data. How do you think about that and the importance of data versus model?

• 17:03Speaker 01

  I think the size of the model matters, but it really depends on what you're trying to do. If you want to have this like amazing kind of personal assistant, like in the movie Her, then like you want to build like a massive model. But the model size is really the number of parameters you have. That's really pretty much it. But on the flip side, and this is where like, this is where like it depends on what you're trying to solve. And the volume of your training data also matters. If you have a large model that is under trained, it will underperform a small model which is really well trained. So you're just wasting resources and you're going to get like less efficient results. And then there's always the examples right now that like when you build specialized model for media analogies, like an athlete that you transform into a weightlifter or like a long distance runner,

• 17:46Speaker 01

  they perform much better. I can't talk on behalf of Scott and Gustav, but if we're trying to build, I'm trying to build a job seeker coach guide, I'd be better build that agent specifically for that role versus some kind of like lifelong coach that will be very hard for me to build. Those will perform much, much better.

• 18:02Harry Stebbings?

  What's the hardest thing about model implementation? When you look at current product and current tech stack, when thinking through model selection and then model implementation, what's the hardest thing?

• 18:12Speaker 02

  The little secret is that there's a lot of final mile tuning and work that probably we're all doing under the hood. That is work that only we could do because we know our customer really well. We know our product and experience really well. And you know, it's those little finesse moments. I mean, it's the same playbook all along, right? What makes a customer love using a product? Because the product is empathetic to their problems and if the interface, you know, meets them where they are. And I think we get lost sometimes in the technology and forget that it's not the technology that makes us successful. It's the user's experience of the technology that makes us successful. And that's why I think the role designers is more important, not less important in this modern world.

• 18:52Speaker 02

  And you know, I've been thinking a lot about the pace of change that we're all dealing with these days, you know, in our respective companies. I'm sure like me, you're waking up and every morning be like, oh my gosh, what new breakthrough do I have to figure out today, you know, in terms of how it impacts our business? You know, and in this like newsletter exercise I force myself to do every month called implications, like I'm calling it surfing waves in the Cambrian explosion. This notion of like, you pick a wave and then you're like, oh wait, I'm on the wrong wave. Like, how do you transition to the next, that wave over there, but then wait, this wave over there? And it's almost like this frenetic motion that we're all in. And the only like solution I can come to is just like doubling down on empathy with the customer.

• 19:35Speaker 02

  There's so many new technologies and possibilities being thrown at us. But if we just kind of like take the pulse of where the customer is and where they're likely Going even more so like maybe we can help make the right decisions. Otherwise, it's like wild.

• 19:47Speaker 00

  Yeah, I agree with Scott. And I think on your question of what was the hardest thing? That depends, as Tom said, sort of like, what are you asking about and when? Initially, it was very hard to find talent on the technology side. That's now getting easier. And so the problem keeps moving up the stack. And as we talked about, I actually think the hardest thing for us now has been to sort of retool the company to think about what the model needs to serve the user, to rethink design, to rethink products. For a while, there was a lot of work around making sure your data was useful. So it keeps changing what the problem is.

• 20:19Harry Stebbings?

  What does retooling the company mean, Gustav? Is it like, again, this is kind of my question.

• 20:24Speaker 00

  It's what I spoke about. We have a bets board where we kind of stack crank what we're doing. We literally had a bet called AI is the product for like two years to really emphasize the shift. And so getting people in that mindset, educating them to understand how this works, to start to use these things and get empathy, not just for users, but for the models. Because people first underestimate the technology and then they're excited and they overestimate what it can do. So they create products that are not mature yet. And so there's a lot of work in getting people realistic about where these things are right now. That was probably the hardest thing to get everyone sort of on the same level of knowledge and understanding.

• 21:01Speaker 01

  Yeah, one fun one, and I'm curious if Scott and Gustav are doing this as well. But I remember when we shifted from desktop to mobile, we literally had to force people to bring mobile designs. Because again, they were like, we had to force them, literally said like, there's not going to be a jam session unless you bring a mobile design. Now I want to see your prompt. Because ultimately, that's the interface that people are using to generate the results. I want to learn how you're building your prompt and I want to critique your prompt. I want to understand your prompt really, really well. I can't underestimate how much intelligence and even art goes into building the right prompts.

• 21:35Speaker 00

  For sure. I never imagined doing product jams and asking to see people's prompts going into the fashion and now we're doing this on a regular basis. I think this is like a very democratic thing because it used to be if you were like a product person or designer and you had an idea, you kind of needed to convince engineering to build a prototype. In this space, what is happening more and more is like the designer or the product person come with the prompt and the idea ready. It just generates something. And then you can, you know, that's very empowering, I think.

• 22:00Speaker 02

  It's so true. And, you know, we've all been talking about design-driven, you know, innovation and product and whatever for a decade or more. But I think this point about how you have these core teams that are building APIs that are ultimately representing the capabilities of the model, and you can actually directly empower designers to start to explore interfaces and, you know, and to the point about the intonations and the persona and all those other decisions that these designers are now making. They thought they went to school for graphic design or product design, but now they're actually, you know, thinking about the conversation inflections and all these other nuances of design. But that's what we did too. You know, we had our design team actually build a team within that did all of the early development of Firefly. And it was a very like design-driven exercise

• 22:44Speaker 02

  to figure out the interfaces and how they integrated into the products.

• 22:47Speaker 00

  I wanted to just quickly go back to one question you asked there about sort of size of models and data. So I think it's interesting to sort of think about. My bet is that size of models will keep being very important for some time. Actually now, you know, the Chinchilla paper, it's very predictable how it's going to scale. It's unclear why it wouldn't scale, even if it's slightly diminishing for a while more. We're not even at the level of the brain yet in terms of connections. So that seems likely. My hunch would be though, that in the longer term, you're going to be able to do most of what you want with reasonable sized models and like having a much bigger model doesn't have that much. So I would still bet that having lots of, lots of user data and lots of high fidelity user data, basically great user understanding is going to be important

• 23:32Speaker 00

  in the long term. To your question of like, does data really matter? One argument is it doesn't matter. It's all going to be in the model. The other argument is like, no, the models will be very powerful. To be able to ask them good questions, you need a lot of data about that user. And that's kind of the bet that I would make. There was this paper called MedPrompt that came out recently where, you know, GPT-4, like a general model, if you prompt it the right way, it actually beats a fine-tuned model that was fine-tuned on medical data through this very, very clever prompting. But it sort of still proves that in order to do this prompt, you needed to embed a lot of user data. So I still think, long-term, data is going to matter, to have proprietary or a great user understanding of the specific user.

• 24:14Harry Stebbings?

  Why doesn't Spotify create its own models, Gustav?

• 24:17Speaker 00

  So we have our own models as well. We do both. We have lots of our own models. But we don't have a GPT-4 that we haven't told the world about. I'm sorry to reveal that. But it's also what I said before, like, our goals are different. OpenAI's stated goal is AGI. That will drive different incentives. It will not optimize for delivering two minutes of audio to have a billion users per day, right? Not necessarily going to optimize for cost efficiency and building pragmatic products. That's why, you know, there is no point in trying to compete with them for us because we don't even have the same goal. So we're trying to build exactly the things that we don't think they will build. And we would buy the things that they will build or someone else will build. Obviously, we're working with Google and GCP, so that's where we do most of our work.

• 24:55Harry Stebbings?

  Guys, how do you think about the decision to build your own models? I'm just intrigued on like, you know, I asked Scott at Lassie in that question. He was like, no, it's not our call. We don't need to.

• 25:04Speaker 02

  Actually, I think that's that's sort of the right answer is you should build models if you're the best company in the world to build them. And so, you know, for us, it was like, who's going to build a better imaging model? We've got the advantage in terms of understanding the customer's use cases. We have the data. No one should build a better imaging model than Adobe. And so we're like, okay, we need to set out and make sure we hold ourselves to that level. But when it comes to like building a massive LLM, to Gustav's point, like, let's partner with the LLMs of the world that do this, you know, as a business. And I think it's as simple as that. So when we're looking through all the different vectors, you know, we want to build the models that we believe our competitive advantage is to is to be able to build and be best at, and then those others we should partner with.

• 25:47Speaker 01

  There is a junction there, at least for us, given just the amounts of data and nodes and the graph, is that what we're building is an integration model with the models to be really powerful. So today, many companies will just push all the information to the prompt, more of like, I would say, like an interim solution, until you have your own integration with the model on the first serve. So imagine the prompt already knows to reach the dispatcher or the router, and that will basically grab both of the... There will be conflation at the tier level that the prompt can also be masked from. And I think that's actually really powerful. I don't know if I want to compete with the next developer conference from OpenAI, despite us being some siblings to an extent. From a Microsoft standpoint, that's what they'll do best.

• 26:30Speaker 01

  But what we do best is understanding how it works for LinkedIn. But Gustav does amazingly well, is understanding the idea of audio and media. What Scott will do really well is the idea of design and visuals. Building that asset, for me, is a massive differentiation. Competing for, which I think would be also super hard to compete, and you're competing for technology. I don't think you're competing for really solving problems. I don't think is the right solution. I don't see anybody winning there, really.

• 26:56Harry Stebbings?

  Does it change how you do anything with regards to data collection, data treatment, data cleansing?

• 27:02Speaker 01

  I love that. Okay, I have to take this one. Okay, I'm gonna start.

• 27:07Speaker 01

  He was keen for that one, wasn't he? Because this is a big one for me. It's been a pet peeve of mine for a long time. So I've been preaching AI-first mentality for a long time, and really trying to push both design and product folks, and even engineers, to think about data collection and quality data for a long time. Again, this has been like a thought, it's almost like somebody else will do it, data science will figure out how to just collect it from the signals. And you don't understand, this is oxygen. This is what feeds... If AI is at the center, what basically brings it to life is data. I remember, like, early on at LinkedIn, I used to literally spend so much time filtering by myself data, so I can basically say, "This is good food. This is good diet. This is what the algorithm should eat."

• 27:50Speaker 01

  But somehow people gravitate towards the UI or they gravitate towards the specifically the visual elements of it, not understanding that actually fits the algorithm is the data. My opinion, as a product leader, whether you're a designer or an engineer or product manager, data is literally your second most important job in your role. It's understanding how do you basically start to infuse data collection and make sure it's high quality. The first most important job is understanding the objective of the algorithm, which most people also. People outsource this stuff. It drives me insane. They outsource what is the objective of the algorithm to do. They kind of create some kind of spec, nowhere in the spec is it ever written. What is the algorithm supposed to do in a nuanced way? And they outsource this stuff.

• 28:32Speaker 01

  But data is a massive, massive pet peeve of mine because most people just disregard it or they just assume it will be there. Somebody's going to create it and collect it.

• 28:40Speaker 02

  We just always have to be very careful because we're so focused on for the purposes of generating content, the commercially viable and safe nature of the model matters a lot. We're always trying to learn how customers use our tools, but we don't ever train our models off of the customer's creations. So like what the customer makes, unless they submit it to stock and wanted to train our models. You know, it's interesting, like when we first launched our models, like there were there were some people who went in and, you know, typed in Spider-Man doing something and said, oh my God, this sucks. Like, you know, Firefly doesn't even know who Spider-Man is and we're, we had to explain like that's actually, that's a feature, you know? It shows that we didn't train on copyrighted content and you can trust the way that these models are trained as opposed to some other models.

• 29:23Speaker 02

  So the data policies matter, matter a lot. But one thing I should say, you know, Adobe's always been two businesses really. It's like the digital media business and the digital experience business, which is a set of tools that big companies use for their marketing. And whether you go to Nike or any of these other brands, like a lot of them will use our tools to manage their customer experiences at scale. And I've always felt like in some ways Adobe was two different companies sometimes. And in the last year, the suddenly it makes so much sense because the digital experiences you're delivering, like the marketing, the content, you know, your experience on the Nike website or whatever can be informed by the company's relationship with you. And the sizes of shoes that you've purchased before and you know and where you live and things like that.

• 30:06Speaker 02

  So for marketing purposes, you know, companies are going to leverage their own data to dramatically personalize digital experiences in ways we can't even fathom right now. That's really cool, you know, for the future of digital.

• 30:17Speaker 00

  From our point of view, we had the same journey that Tomer had about data. We did realize the value of data early on with all the playlists we had, which was the basis for our first sort of simple embeddings. For us, obviously, the the usage data, how you playlist, how you listen is is very important. We had that journey as well. I think the best example is actually Scott and Firefly, where you're solving an IP problem. You're also solving a technical problem of I want painting and so forth, but you're also solving a business problem. Buy the licenses and the rights you have to the data. That's a very important angle that you may forget. You can solve many problems for a consumer or a or a user. In general, you know, we've seen the first wave of AI now.

• 30:59Speaker 00

  Where the technology is very impressive, it's starting to do useful things. But we still usually the big changes come when that technology enables a new business model somehow. And that hasn't really happened yet. I'm absolutely certain it's going to happen, but so far it's like mostly sustaining innovation.

• 31:14Harry Stebbings?

  What do you think that business model looks like, Gustav?

• 31:16Speaker 00

  I don't know yet. I think I think one example is is Firefly, who is trying to solve a business problem as well. But most of these things are not yet trying to think about if business models can change completely and and they probably could. But I think before that happens, this will be sustaining innovation that largely accrues to bigger players. Usually when the business model is challenged, someone counterpositions like a big business model, that's when things really change. And that hasn't really happened yet.

• 31:40Harry Stebbings?

  You said innovation and the value accruing to bigger players there. I think that that's kind of what everyone's asking, which is like, who wins? Is it incumbents or startups? And all startups say, well, you know, the incumbents, they can't move very fast. And I mean, respectfully, all three of you have moved really fast. My question is like, is it kind of old BS that incumbents can't move fast? How have you guys been able to move so fast to embrace it? Scott, you talked about kind of the Cambrian explosions or Cambrian waves and jumping. How do you move so fast at scale? And is it BS that incumbents can't?

• 32:12Speaker 02

  Well, here's the thing. I mean, every platform shift is not the same. They're all different. And if you're going from on-premise software to delivering in the cloud, that is a major, major transition that I can imagine many incumbents would struggle to do in a way as fast as a startup could do. And maybe that's why startups totally led the pack in terms of cloud alternatives to on-premise software in the past. And then you, mobile was probably the same sort of thing. Different stack of developers, totally different go-to-market in some cases, different relationships with Apple app stores and like, you know, there were so many differences. But with AI, I'm not actually sure that as a platform shift, which it certainly is,

• 32:54Speaker 02

  there are as many differences in how a company facilitates, you know, and leverages that platform shift. Because, you know, when someone comes into Photoshop and they have an image that they want to change, right? If I can just give them a default bar that just shows up where they can just articulate what they want instead of using all the knobs and bells and whistles of Photoshop that have accumulated over the last forty years, right? That is like a very incredible way of disrupting Photoshop to some degree. I don't have to build a net new company and a net new product from zero to one in order to reimagine the capabilities of Photoshop using AI in this instance, right? Now, people might not want to even download Photoshop, in which case that's why we brought these capabilities to Firefly as a standalone app as well.

• 33:39Speaker 02

  On behalf of Adobe, I'd love to take the credit of like moving super fast, you know, and I do think we executed well, but I also think that there are some nuances of this platform shift in particular that do favor those companies that know their customer well and already reached them.

• 33:53Speaker 00

  It's always easier to analyze other people's companies. I think like the risk for Adobe was before they moved to subscription model. Like the subscription model works with this as well, so that's why it's not disruptive. I think what has to happen somehow, someone has to figure out a business model that is disruptive to someone. Some people think that it will be disruptive to Google's business model. You know, that the ad model won't hold up in a conversational world. Not so sure that's actually what's gonna happen, but something like that would need to happen, I think. For Spotify, for example, when we started, it was the business Business model of streaming, that was the disruptive thing. The technology was a bit of peer to peer and stuff, but it was a business model that took Apple eight years of collateral damage to follow. That's kind of what needs to happen, I think, and we haven't really seen that yet. It could also be that this is just more productivity for the existing players.

• 34:38Speaker 01

  I appreciate the push on the business model because I don't think we often talk about it. But you can play the the regular like SaaS model based on seats. And if your technology really makes the organization much more successful, then your seats volume model does not work anymore.

• 34:53Speaker 00

  Yeah, that's a good point.

• 34:54Speaker 01

  Is it really just price that you increase? It becomes like a really big conversation to have around what's the business model? When your product makes the organization more efficient, you're cannibalizing your own seats model.

• 35:03Harry Stebbings?

  There's a brilliant piece by Sarah Tavlet at Benchmark, actually, which is talks about selling the work and not the seats.

• 35:09Speaker 01

  Exactly. You actually still need to sell because the idea of like you're hiring, we talk about hiring and we immediately imagine hiring people, but you're really hiring task completion.

• 35:19Speaker 02

  I love this. This question about what business models will be disrupted by AI is something we probably don't talk about enough. I mean, think about how many people in the world get paid on an hourly basis for what can be achieved in an hour. I mean, even lawyers charged by the hour still, designers charged by the hour. And now you see these new technologies that are readily available. It's like, how does it even work? You know, you're really paying for, of course, right, is like the the depth of experience and judgment that one applies to their work. You know, your lawyer's ability to know based on their like, you know, 30 years in practice, like what's likely to happen to advise you in the right decision. If that's made in three minutes, like, how do you how do you compensate them? And I think the same goes for the seats part as well. The functions of so many businesses are collapsing now.

• 36:03Speaker 02

  And so the idea of selling seats function by function, like how many people in procurement? How many people in financial planning? I mean, I mean, it's such an old antiquated way of building a business to some degree in the age of AI. So I'm excited about this new value based innovation vector for business modeling.

• 36:19Harry Stebbings?

  Is that challenging for you buying software then? When you think about buying new tools today, does it change how you think about buying tools? We're all huge buyers of tools internally as well. Like you say antiquated Scott, I mean it nicely, but I'm sure like we all buy a huge amount of seat based tools.

• 36:34Speaker 02

  Well, we do and at some point, probably sooner than later, again, we're early in the days of AI. But at some point, when there are more tools in the organization that do many different things. You know, I think that the idea of buying seats function by function will just evolve. You know, I, but you're, it's a good question, Harry. Like, when is that really going to be material and how is it going to manifest?

• 36:56Speaker 01

  Yeah, Harry, I think, I think your next podcast is with a bunch of CPOs and how they're making. Because this is actually, this is where.

• 37:04Harry Stebbings?

  Come on, give me a break, okay?

• 37:06Speaker 01

  Yeah, this is going to be the next productivity layer. And I think like for many companies, I don't think coming and saying, I want to deploy this, you know, this program for 20,000 employees is going to work anymore. People are going to try and understand like, wait, wait, what's more productive? How much output are you going to get? Those are phenomenal questions we never used to ask a year and a half ago. That's like an innovation catalyst.

• 37:24Harry Stebbings?

  Again, speak to many people and everyone says that AI development progression technology stellar 10 out of 10. We're doing a great job. Enterprise adoption, about a negative four out of 10. Not good. Like, you know, I think as a stat, 32% of European corporates do not know what Slack is. That's concerning. How do you think about enterprise adoption, keeping pace in any way with AI development?

• 37:49Speaker 02

  I think it's probably good news that we're so early in the adoption curve. Cause that means that there is a lot of potential ahead for these companies. And I also, it doesn't happen linearly. I think that the adoption happens in moments where there's like a step function, you know, breakthrough, or people just suddenly realize that. I mean, a lot of enterprise selling also happens because of who else is buying it. When I'm trying to work with customers who are thinking about, are we, are we ready to change the way we do this? It's much easier when they hear about that company that has already changed the way they do this and how, you know, they're saving money and, and they're, and they're producing better output. It's like, oh, okay. But, you know, I think that the part of, part of this is always about getting small teams. In big companies to start to play with something.

• 38:32Speaker 02

  You know, that's where I think enterprise sales can fall flat sometimes. And you have to have design partnerships with early customers as opposed to traditional sells. You know, to go into early customers and say, hey, I'm not even trying to sell you on this. I just want a small team that can start to play with some of our technology. We have this tune your own model capabilities now for Firefly where brands can use some of their own IP and make a version of the model for them. And we've been going to some specific companies and saying, hey, be our partner and let's play with it. And then, of course, like that play becomes utility. And then that utility becomes a reference point. And eventually, you know, when we're ready to sell it more broadly.

• 39:10Speaker 00

  So I agree with Scott, both that it is early. Yeah, maybe it's our job to start using these things ourselves internally first and get them to work. But I also agree with Scott that I think it's going to go much faster than moving to cloud. I know we actually were on prem companies that old. It wasn't possible to just also be on the cloud and try a little bit. It's incredibly hard. I don't think this will be that hard. I think you can try these things in parallel. It's going to be an S curve, but I think it's going to be a sharper S curve once it happens.

• 39:36Harry Stebbings?

  We've spoken about kind of the seismic change needed in product and designer minds. If you were to sit down with young designers, young product people, and we have like 700,000 that listen to funny product. What would you say to them in terms of what they can, should do to equip themselves for the changes that we're seeing? So they're best placed in their careers.

• 39:57Speaker 01

  I'll get into, I'll start with a quick, just a stat from, we're basically tracking very closely at LinkedIn. They're just looking at talent overall for the world. Like the job is changing on you, whether you like it or not. So like, I think this one mindset for us, I think everybody here on the call and for every generation to come, like your job is going to change much faster than you think. But it might even change titles, the tasks will change. So it's really about how you morph for that. We looked at like the last five years, obviously AI has been playing a big role. But the skills that necessary to do a job changed by 25% in the last five to six years. By 2030, they're going to change by at least 65%. Whether you want it or not, your job is changing.

• 40:36Speaker 01

  So the necessity to learn and to really start shaping your understanding of the task you're trying to do, how the task is morphing is critical. For me, I would put it under the umbrella of growth mindset, just the ability to learn. Then there becomes like more of a question around how should I think about beyond being AI literate and beyond being focused in an AI-first mindset, but should I do? I think there's two aspects that come to mind to me. One is like a more of a T-shaped perspective where I would assume you should have more broad skills than you had before, the ability to activate. Or to that angle, I would still build expertise around a specific domain or specialty. We talked about disruption before between incumbents and startups. Usually we talk about tech.

• 41:17Speaker 01

  There's so many industries yet to be disrupted that nobody talks about, but those could be remarkable areas to focus on. And lastly, I think it's innately what makes us human, things like soft skills, interpersonal dynamics, and imagination, like just focusing on those. Those are not going anywhere.

• 41:34Harry Stebbings?

  Guys, hit me. Go granular. What do I actually do? My job's changing.

• 41:38Speaker 02

  Treating yourself as a business is something also everyone should always do. It's how you organize your ideas and information, how you track your own spreadsheet for your own budget. We all have our own business called ourselves that we also manage, part of being a human being. So embrace these tools. A lot of folks used Google Apps for themselves before they introduced them to their companies. A lot of us used Evernote or Notion or other things, and then we said, wait, I want to use this with my team. Have some embracing. Novelty precedes utility. Playing with something gives you ideas of how you could maybe pilot it with something you do in your day job. And then if you have a pilot, that's how you learn as to whether or not it's relevant for actual practice.

• 42:23Speaker 02

  So that's how we stay on the edge. And it's interesting. We all know people who are those early adopters who love playing with new tools. And then we all know people who are the total pragmatists, who are just like, I don't use it until I'm told I have to change my tool. And you can rip this old tool out of my hands at the last minute that it's supported within the company. If you're listening and you're early in your career. You can be the person on your team that introduces new practices. That's your advantage in a company, especially working with a lot of people that have been around a lot longer, is you can be the one who pioneers new practices.

• 42:58Speaker 00

  And I think in general, like the one way to think about big shifts is that's actually when the opportunity arises. To Scott's point, a lot of people are slow to change. So it usually is the case that one of the advantages of coming straight out of school is what the disadvantage is you have no experience. But the advantage is usually you have the latest and greatest knowledge. Whereas someone who worked in a company for 10 years, they're using the old tools. So I think that's still true. You can have that advantage. But even if you're in a company, the only advice is to educate yourself. And as we said before, in this instance, it's actually simpler than ever. You don't have to ask for permission to learn how to prompt and how to use these things. You know, education is on the internet is basically free now. It's very cheap. I think it's easier than ever, more democratic than ever. To Tamer's point as well, it's just going to keep changing.

• 43:43Speaker 00

  There's actually the risk is if you're in a company that you don't develop fast enough. I think if you're if you're fresh, you're going to have those latest skills.

• 43:50Harry Stebbings?

  Listen, I want to dive into a quick fire. So I say a short statement, you give me your immediate thoughts. Scott, I like your newsletter. What's your biggest lesson from running applied to product?

• 43:59Speaker 02

  Oh, from running. There are a lot of lessons that I have abstracted from running. I'll tell you one of them, which is when I'm running, I often have ideas. And I'm obsessed with capturing ideas and I'm always worried about forgetting ideas. But when you're running, I just like force myself to keep running because I don't want to give myself an excuse to stop running to capture an idea. And that period that I am forcing myself to keep thinking about this idea as opposed to writing it down, it always gets better. I think about that now in my everyday work. I'm so impulsive, I'm always like trying to be decisive. I feel like that's one of the best practices of being an executive is decisiveness. But if you can like sit with something a little longer than comfortable, oftentimes you end up with a better approach to how to say it,

• 44:41Speaker 02

  you know, to how to give the feedback to the person, you know, how to like solve that problem with the product. I have this like mantra of wait for it that I've learned from running.

• 44:50Speaker 00

  The cadence of running, actually, I think is almost like a process. You're forced to iterate the idea. It's almost like in a, you know, in a trance. I totally agree. Having to be forced to be thinking of something, you have this clock going tick, tick, tick for an hour. It certainly helps the idea.

• 45:04Harry Stebbings?

  Toma, if you could change one thing about the LinkedIn product today, what would you change?

• 45:08Speaker 01

  What's interesting right now, we talked about the idea of UI and complexity. There's this great law that I'm, I really like. It was really helpful early in my career to think about how to build products, which is the conservation of complexity. The idea that every product has an inherent amount of complexity. The idea is can you solve it in the product development side, or do you actually put it on the user to solve by themselves? And I think we're in this phase right now where there's so many use cases and audiences that people come to the LinkedIn main app for. And that just had inherent complexity because there was so many, you know, you come in the morning, you're trying to see what's happening in the world. Later in the afternoon, you're interviewing somebody, you want to check out their profile. Then, you know, your boss upset at you later in the day and you want to see what's out there for you. This is all three use cases in the same like 24 hours.

• 45:52Speaker 01

  Now with AI, and I would claim probably this would be more generalized statement, the more complex your product, the more impact AI could have in terms of simplifying it for your user base. So one thing that we're already underway for us is truly take away the complexity and solve it with the idea of bringing in more of this new large language models and in general, this new brain that can like sit on top of it. So instead of adding more features, it's the same experience, it just morphs towards your need. So more TBD on that one.

• 46:20Harry Stebbings?

  I think you should call it a name like Einstein. Don't you love Einstein as a name for Salesforce?

• 46:25Speaker 00

  We call it fighting the entropy in Spotify.

• 46:28Harry Stebbings?

  Tell me, Scott, what have you changed your mind on in the last 12 months?

• 46:31Speaker 02

  Well, I think within a big company, you know, I've, I've changed my mind on the, the need for centralization. Where things should and shouldn't be centralized. Like I always go back and forth honestly on this. You know, there have been periods of time where I felt like design needed to be in different organizations because people had to prioritize and properly resource design for their business. And then recently I re-centralized design in one organization because our strategy, you know, required it. And I guess one of the things I'm learning is that sometimes, you know, you make a completely opposite decision at different times in the same business because the playbook is different. And we typically don't think that way. We like, when we make a huge decision and it's like, this is the way it's going to be, we just stick to it because we almost think like it's first principles.

• 47:16Speaker 02

  But you just have to be willing to discard the playbook consistently.

• 47:20Harry Stebbings?

  Okay, let's go for you, Toma. Can product leaders not be trained in AI today? Can you be a CPO and not have really spent hard yards in AI?

• 47:30Speaker 01

  I would not hire you to my org if you don't have the willingness and the aptitude to go deep. Again, I don't need you to be the engineer working on the model, but I need you to understand, one, the objective you're trying to build and how to build it. Otherwise, what are you here for? And two, all the underlying principles that come with it. The idea of it's not being deterministic. So how do you build the knobs so elegantly that the experience is great in any shape or form? Two, how do you think of data collection in a way that's responsible, but really fuels what you're trying to build? And three, it's that velocity we just talked about. If you don't have this velocity of learning, I don't think you'll last.

• 48:07Harry Stebbings?

  Gustav, final one for you. What are you most excited about when you look at AI in the coming years and what it can do for everyone, but also for Spotify? Just what excites you most?

• 48:16Speaker 00

  For me, we experienced a shift to mobile. It was very scary. Our business model had to change, could have disrupted the company. It was also by far the most exciting time I had. And to be honest, it was a little bit too stable for a few years there. And I feel like we're back to that kind of change. I'm secretly hoping for that business model challenge and those things happening, that everything changes again. I don't know if you agree, Scott and Tomer, but if you're a designer product person, those are the exciting times. We're right there now. I don't think we're at the end of it. I think we're at the beginning of it. So I'm kind of back to like more excited than maybe in the last seven, eight years. It's unclear now what's going to happen. We were on a little bit of an iteration track for a while there. It's not an iteration anymore at all. So that's what I'm excited about.

• 49:00Harry Stebbings?

  Thank you so much for this. This has been fantastic and I so appreciate you putting up with my questions.

• 49:06Speaker 02

  Thanks for having us and Emmerich Gustav, good to see you guys.

• 49:09Speaker 01

  You too.

• 49:09Speaker 00

  Great to see you.

• 49:13Harry Stebbings?

  I cannot state enough how much I love doing that show. If you want to let me know on Twitter at Harry Stebbings what you thought and feedback on the format itself of the show, I'd love to hear your thoughts there. Likewise, you can check it out on YouTube by searching for 20VC. Love to hear your thoughts on that format also. But before we leave you today, HMC or Harvard Management Company, they're constantly seeking out the next generation of truly great investors and entrepreneurs. HMC has managed Harvard University's endowment for nearly Fifty years and was one of the first institutional investors in venture capital. Their experience and long-term investment horizon make them ideal partners to get world-changing ideas on a path to viability and success. They work as a true partner, providing insightful perspectives to help managers succeed.

• 49:57Harry Stebbings?

  I personally have had the pleasure of working with the HMC team and can say that they're truly exceptional partners and just great, great people to work with. Whether you're launching your first fund or your fifth, HMC welcomes the opportunity to partner with both developing and established managers. Have an idea you want to share with the team? Just send it to [email protected]. As always, I so appreciate all your support and stay tuned for a fantastic episode this coming Friday.