How Startups Can Crush Enterprise AI Sales | E2050

This Week in Startups is brought to you by… LinkedIn Ads. To redeem a $100 LinkedIn ad credit and launch your first campaign, go to http://www.linkedin.com/thisweekinstartups Zendesk. A service-first CRM company with support, sales, and customer engagement products designed to improve customer relationships. Qualifying startups can join the Zendesk for Startups Program and get six free months of Zendesk products. Visit https://zendesk.com/startups today to get started. Beehiiv. Power your newsletters with AI tools, referral programs, and ad network features—all in one platform. Get 30 days free and 20% off your first 3 months at https://www.beehiiv.com/twist * Todays show: Alex is joined by Joff Redfern and Derek Xiao of Menlo Ventures to discuss the significant role AI plays in modern workflows. We look behind the numbers of their recent report: "2024: The State of Generative AI in the Enterprise" while exploring AI spending trends in enterprises, with a focus on its impact on the healthcare and legal sectors. The conversation highlights popular AI use cases, the role of large language models in content creation, and workflow automation. We delve into advances in generative AI applications, enterprise spending, strategies for AI startups, training AI for market verticals, and the future of artificial general intelligence. * Timestamps: (0:00) Joff Redfern and Derek Xiao join Alex (4:08) Yahoo's evolution and Anthropic's recent funding (6:23) AI spending trends in enterprises (10:14) LinkedIn Ads - Get a $100 LinkedIn ad credit at http://www.linkedin.com/thisweekinstartups (12:20) AI's impact on healthcare and legal sectors and its rapid adoption (19:25) Popular AI use cases in enterprises (20:56) Zendesk - Get six months free at https://www.zendesk.com/startups (23:17) The role of LLMs in content creation and workflow automation (27:04) Advances in generative AI applications and agentic systems (30:43) Beehiiv - Get 30 days free and 20% off your first 3 months at https://www.beehiiv.com/twist (32:14) Enterprise generative AI spend, ROI, and budget expansion (38:45) Selecting generative AI tools and AI-first startup opportunities (47:43) Competition and strategies for AI startups (50:10) Training AI for market verticals and the future of AGI (53:03) Recap of enterprise AI spend and trends * Want to host or join the next Founder Fridays? Check out https://www.founderfridays.tech Subscribe to the TWiST500 newsletter: https://ticker.thisweekinstartups.com Check out the TWIST500: https://www.twist500.com Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp * Links from the show: Check out Menlo Ventures: https://menlovc.com/ Menlo's report "2024: The State of Generative AI in the Enterprise": https://menlovc.com/2024-the-state-of-generative-ai-in-the-enterprise/ * Follow Joff: X: https://x.com/mejoff LinkedIn: https://www.linkedin.com/in/mejoff/ * Follow Derek: X: https://x.com/derekgxiao LinkedIn: https://www.linkedin.com/in/derekgxiao/ * Follow Alex: X: https://x.com/alex LinkedIn: ⁠https://www.linkedin.com/in/alexwilhelm * Thank you to our partners: (10:14) LinkedIn Ads - Get a $100 LinkedIn ad credit at http://www.linkedin.com/thisweekinstartups (20:56) Zendesk - Get six months free at https://www.zendesk.com/startups (30:43) Beehiiv - Get 30 days free and 20% off your first 3 months at https://www.beehiiv.com/twist * Great TWIST interviews: Will Guidara, Eoghan McCabe, Steve Huffman, Brian Chesky, Bob Moesta, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland * Check out Jason's suite of newsletters: https://substack.com/@calacanis * Follow TWiST: Twitter: https://twitter.com/TWiStartups YouTube: https://www.youtube.com/thisweekin Instagram: https://www.instagram.com/thisweekinstartups TikTok: https://www.tiktok.com/@thisweekinstartups Substack: https://twistartups.substack.com * Subscribe to the Founder University Podcast: https://www.youtube.com/@founderuniversity1916

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  A startup should be looking at like, well, what's different? And it's like, well, part of what's different is we have reasoning as a utility. Think of every role in the economy as a bundle of tasks. And those tasks, some of them fall to AI and some of them fall to to humans. So I I do think what startups should be looking at is like, how can I change? The way like, if you look at workflow, the way that it's been built over the last 10 years, it's been built with humans at the center. Yeah. And that's the actual workflow. So if you were to zoom all the way out and say, oh, okay, well, I'm going to have a workflow that's going to be shared. Some of it's going to be through an algorithm and some of it's going to be through a human. What would that allow me to do?

• 0:42Speaker 01

  This Week in Startups is brought to you by LinkedIn Ads. To redeem a $100 LinkedIn ad credit and launch your first campaign, go to LinkedIn.com/thisweekinstartups. Zendesk, the best customer experiences are built with Zendesk. Qualifying startups can join their startup program and get Zendesk products for free for six months. Visit Zendesk.com/twist today to get started. And Beehive, power your newsletters with AI tools, referral programs, and ad network features all in one platform. Get 30 days free and 20% off your first three months at beehiiv.com/twist.

• 1:22Speaker 02

  Hey, everybody. Welcome back to This Week in Startups. This is Alex and I have a special show for you today. There's a theme going around the world of startups, which is AI hype, AI fundraising, AI applications, AI agents, AI chatbots, AI job loss, AI this, AI that. What I've always been very curious about is, behind the headlines, how is enterprise generative AI spend going? And thankfully for me, a venture capital firm that I've known for a long time, Minlo, dropped a very long and detailed report digging into precisely that. How is generative AI doing in the enterprise? So I decided to have a couple of the authors of that report come on the show to answer my questions. And then, in a stroke of fortuitous luck,

• 2:04Speaker 02

  Minlo is perhaps best known in the AI game for being one of the backers of Anthropic. And this morning, the company announced that it raised $4 billion more from Amazon. So now we have the VCs from the report trapped under our thumb. So we can hit them with all sorts of questions as we would like. But just to welcome them to the show, first of all, we have Joff Redfern. Joff, how are you?

• 2:24Speaker 00

  I'm fantastic, Alex. Thanks for having me on the show.

• 2:27Speaker 02

  Absolutely. And what I like the most about you, apart from the fact that you're part of this report, is that you were at Yahoo back from 2003 to 2009, right?

• 2:38Speaker 00

  I was, I was, I was early Yahoo prior to, I've been a product leader my entire career. Prior to joining Benlow about a year ago, I was the chief product officer at Lassian.

• 2:49Speaker 02

  Yes.

• 2:49Speaker 00

  Was there for six years. And then before that, I was an early vice president of products at Livingston, when it was a tiny company, pre-IPO, and then helped grow that up into 10,000 employees. So I was there for seven years. Yeah, I love building things. Started companies, sold the company, helped take a company public. That's a bit about me, yeah.

• 3:11Speaker 02

  Yeah, but the coolest thing is that you worked at Yahoo, where I've also worked. And it's really-

• 3:17Speaker 00

  Yeah, I was like, of all the things, Alex, that we can point out, it's like, I don't know if Yahoo was the one. But yeah, no, Yahoo in the early days, and you'll remember this, was just super fun. It was like, it was really the Google of the day as the internet was emerging back in the late 90s. So it's, it's always the tricky one because I think, you know, younger folks that are listening will really know Yahoo as the amazing company that we were a part of in the early days. It'd be like if an amazing institute like Harvard was all of a sudden, you know, not Harvard, and it was at a different tier. So yeah, it's changed quite a bit over the years.

• 4:01Speaker 02

  It's like if Harvard became San Jose State, was purchased by private equity.

• 4:05Speaker 00

  Yeah, I was gonna, I was gonna go with Cape Cod Community College, but-

• 4:08Speaker 02

  Cape Cod Community College. Um, yeah, that's a deep cut. I'm in Rhode Island, which is why all Northeast jokes work. But we also have Derek Shaw with us. Derek, hey, you, my friend, I was prepping for the show. And the thing about your background that I love the most is that you were president of the Harvard Crimson.

• 4:25Speaker 03

  That's right. I started in journalism.

• 4:26Speaker 02

  I know.

• 4:28Speaker 03

  A lot of overlap, I think, with venture capital, but trying to, yeah, it's an interesting start and a lot of admire a lot of the work that you do and excited to be on.

• 4:37Speaker 02

  I appreciate that. So also, Derek, on your Menlo page, you are tagged in on the Anthropic side of things. Now, I know it's Matt Murphy, who's the lead partner over at Menlo for the Anthropic relationship. But today, the company announced that it raised four billion more from Amazon, bringing it to a total of eight billion dollars. And I guess the question that I want to know is why, why is it still so expensive for these AI companies? Because that's a lot more money not that far afterwards. And to me, sitting here where I am, apart from the numbers and so forth, it feels a little terrifying, but I was hoping you could have swaged my, my, oh, my God, at that number.

• 5:16Speaker 03

  Yeah, I mean, I think Anthropic was, has a special relationship with Amazon. I feel like they have a very close partnership, and this is just a furtherance of that. But the other side of the coin is that Anthropic is one of the foundation models, right? This is when we made our first investment back in 2023, the thesis was that this was going to be one of the, you know, companies that will matter the most in the AI revolution. And we've seen that play out. That's why we have been getting closer and closer to the company. And, you know, the announcement today, it's probably just a furtherance of that and deepening of the relationship with Amazon.

• 5:51Speaker 02

  So, Joff, you guys took part in the Series C, and then if I recall correctly, led the Series D, is that right?

• 5:58Speaker 00

  Yeah, we, through an SPV, we were lead on 500 million of the billion dollar raise that happened on the D side.

• 6:06Speaker 02

  And I take it now feeling pretty smart, given how AI has gone since that deal was put together?

• 6:14Speaker 00

  Yeah, well, we're, we're excited to share with you some of the findings that we've, we've learned on the LLM side as it pertains to the enterprise, right? There's obviously two large markets for LLMs. We have the consumer-based market, and then we have the enterprise-based market, and it, it's, you know, it's starting to shake out that these are different markets and, and how competition is.

• 6:38Speaker 00

  Approaching and attacking those are quite different. So very, very happy to see some of the data behind Anthropic's progress on the, on the, on the enterprise side.

• 6:49Speaker 02

  I know that I'm slightly harassing you with the Anthropic news when I asked you to come on to talk about the actual enterprise AI report. But I'm just curious, one more question about that. Does having a company like Anthropic in the broader, you know, family portfolio, does that really bring in a lot of information to the investment team that you guys can then, I don't know, learn from and apply directly to making new investment decisions? Or is that information segregated from the firm and so you guys can't go fishing in that pond to get learnings for?

• 7:16Speaker 00

  We are, we are very, you know, church and state when it comes to that. We're, we're Anthropic and in both Menlo are, you know, we're not in there deep looking at people's usage metrics or anything like that. But I will say one benefit of the many that comes from it is just being able to see what actually is coming down the, the pipeline when it comes to new models. We forged a relationship, a special relationship with Anthropic five months ago. We announced our anthology fund. That's a hundred billion dollar fund. And that's really looking for who are some of the most pioneering AI founders out there?

• 7:59Speaker 00

  And being part of that fund gives, gives founders a number of benefit, early access to models, twenty five thousand dollars in Anthropic credits, access to some of the DevRel teams over there and some of the expertise. You'd be part of a network of fellow founders and builders in the AI space. And then every so often, once or twice a year, we, we run a builder's day. Our first builder day with Anthropic was the first of November. And that was just, that was super fun. So it was at the Anthropic offices. We had a bunch of the experts from Anthropic coming out, meeting with companies that were building on the, building on the platform.

• 8:41Speaker 00

  And yeah, so we were able to have a really successful builder day. So I would say, you know, that part of the relationship. Relationship is really helpful for both of us. So we're, we're really cherish the, uh, the team over there and they're, they're just amazing. I mean, the, the, the team and the, the caliber of the talent that has been brought to Anthropic. I just, I, I admire every day. It's, it's really quite talented.

• 9:07Speaker 02

  Well, with four billion dollars more, they can certainly keep hiring. And I know one of your predictions in the report was a continuance of the AI talent drop. So we'll get to that in a minute. Let's start with what everyone wants to know, which is the, the high level numbers. So you guys wrote in this 2024, the state of generative AI in the enterprise, that AI spending surged to 13.8 billion this year, up six X from 2023. Now, before we get into categories, Derek, did that match your expectations? Is it a faster pace of growth than anticipated? To me, big number, big jump. No idea if it was bullish or bearish compared to your projections.

• 9:44Speaker 03

  Yeah. I mean, I think that when we did this report last year, we thought that this would take a little while to, uh, ramp up. And that's what you saw in previous technological transformations like this, right? You benchmark to cloud, you'd benchmark to mobile and these are all, you know, trillion dollar kind of, uh, markets now that took a little while to get started. And so I think when one of our predictions from last year was that this would similarly take a little longer to, um, to ramp up despite all the spending that's going to it.

• 10:15Jason Calacanis?

  Have you built something awesome? Are you ready to bring it to market? Well, that's great, but you're probably asking yourself, how do I get to my target customers? Well, here's an easy solution for you. It's called LinkedIn ads. And with LinkedIn, you can precisely target the professionals most likely to care about your product. You can use LinkedIn to reach people by job title, industry, company, location, and more. You know, all the stuff that you have on your profile page? Imagine you're looking to build your B two B audience. Maybe you want mid-sized companies. Maybe you want large ones. Maybe you want emerging small businesses, right? Small and medium sized businesses, SMBs. Well, you gotta build those key relationships and you want to drive results. Well, you just do that with LinkedIn ads. And you're going to gain access to over a billion members of LinkedIn.

• 10:58Jason Calacanis?

  Of which 13%, 130 million of them are decision makers. That means they can pull the trigger on buying your product or service. And 10 million of them, 1% of the LinkedIn audience are those elite C-level executives. You know what I'm talking about. Chief Technology Officer, Chief Strategy Officer, Chief Revenue Officer, Chief Executive Officer, Chief Financial Officer, they're all there on LinkedIn. And you're going to get two to five times the return on your ad spend. 79% of B2B marketers saying LinkedIn delivers the best paid media results for a reason. So here's your call to action. Start converting your B2B audience into high-quality leads today. And we're going to give you $100 credit to start your next campaign. LinkedIn.com/thisweekinstartups to claim that credit.

• 11:42Jason Calacanis?

  That's LinkedIn.com/thisweekinstartups. Terms and conditions do apply.

• 11:48Speaker 03

  I think what we've seen this year actually is a surprise in terms of how fast that the revenue is actually materializing. And so you look at areas like healthcare or legal, which are actually traditional laggards in terms of technological adoption. And these are actually the verticals that are leading the AI revolution in many ways on the app layer. There's, you know, $100 million plus companies now in both of these verticals. And I think that that part was surprising is just like how much that app layer has really taken off.

• 12:20Speaker 02

  The reason why I'm actually less shocked by that than I thought I would be is that my spouse works in medicine, so I'm actually viscerally aware of that world and its inefficiencies. But if there are places where you could apply AI to a voluminous amount of, you know, written information, legal and healthcare are, gotta be two of the ripest, you know, apples on that tree. So even if they are historical laggards, doesn't that actually mean that they have more accumulated equivalent of like technical debt in their operations? And therefore they're the best place to deploy. So actually, I can kind of see that, Derek. I guess I'm less surprised by that than I thought that I would be.

• 12:54Speaker 03

  Yeah, absolutely. And the other way that we look at it too is the comparison between the size of the total market versus the size of technology spent there. So if you think about somewhere.

• 13:05Speaker 00

  Yeah.

• 13:05Speaker 03

  I mean, this this goes towards, you know, the Silicon Valley kind of phrase now is services as software. And I think that it rings true. It's a cliche because of a reason, right? Because these are massive markets that tech used to not be able to touch. And now with AI, they can now automate a lot of that.

• 13:21Speaker 00

  You know, one thing that does surprise me, and going back to this comment that Derek made, is just the the steepness of the curve on AI, right? Having lived through a number of these different waves over the years, you know, the internet came and then cloud computing came and then mobile came. They tend to be much slower. You you lose fact that at the enterprise level, it was really, this is the gen AI movement started two years ago, and I would peg that at January 2023. It was January 2023, the ChatGPT came out and said, hey, we got to 100 million Mao faster than any other application out there. But it was a really telling story. I was I was still at Atlassian at that point. I was the chief product officer.

• 14:05Speaker 00

  And in January, like that first earnings call, there was like one mention of AI. But if you looked at all of 2022, there wasn't a single branch of generative AI. So it went from the first earnings call, by the second earning call, there were 18 mentions in the transcript. So like a fun activity is go grab any company, right? And pull all of the transcripts from 2022 to to now and load them up into your favorite LLM and just say, build me a chart of the number of mentions on AI or artificial intelligence. And what you see is it's like, it goes like crickets to, hey,

• 14:50Speaker 00

  something's happening here to all of a sudden, that is the conversation.

• 14:54Speaker 03

  Yes.

• 14:54Speaker 00

  Um, so I would say that is is definitely one of the things that played out. And that's one of the findings that we have in the report. If you look at 2023, that would be the year really of the pilot. So the the CEO gets off the train, the earnings call with the CFO, goes over to the R and D group and says to the CTO and the CPO like, hey, what are we doing with generative AI? And very quickly, what happens is teams get pulled together. It's like, how are we going to use this new technology? And that's really one of the findings from the survey is that last year was a lot about experimentation. This year, it's really about moving stuff into production.

• 15:37Speaker 00

  And that's where we get the the six X increase in the overall uh spend at the enterprise level coming up to close to fourteen billion uh by by our marks.

• 15:48Speaker 02

  I want to talk about uh basically data sourcing for the stuff because I love this particular chart. But to me, when I see this, I go, how confident are you guys in these numbers? Because you could easily categorize things in different pockets. I presume there's some bleed between them. And also, it's a growing industry. So can you just talk me through the how this chart was put together? And if you're listening to the audio, this shows um year over year changes in generative AI spend for foundation models, training and deployment, data, vertical, departmental and horizontal.

• 16:19Speaker 00

  Let me start with a high level and then I'm going to let Derek talk through the how we've actually calculated numbers. In this chart, really, there's two big buckets that you should be focusing on. One is around the LLM and the infrastructure needed to bring AI into the organization. And then the second bucket, these four are actually three, the three bars on the right are really talking about the application layer. So what we can see is that two thirds of the spend that goes on at the enterprise, nine point two ish billion is sitting in the LLMs in the infrastructure bucket. And for those that can't see, by far, the largest spend inside of the enterprise

• 17:03Speaker 00

  is against the foundation models themselves. Six point five billion of that is spent there. So two thirds is happening at the infrastructure layer. And then we have another uh third, which is being spent at the application layer. The main thing about the application layer, the big story is there is that that's an eight X increase from from the prior. Year. So that's sitting at four point six billion. Then what Alex is is asking, one of the questions he's asking is like, you can categorize your application layers a lot of different ways. Um, in this case, we've broken it down into vertical departmental and and horizontal AI. And and certainly we can squabble about like what belongs into what bucket.

• 17:48Speaker 00

  But I would say the higher order story is that applications are coming, they're alive and well. And then for a variety of reasons, we've chosen to break it out this way.

• 17:58Speaker 03

  In terms of data sources, we went out and asked six hundred IT decision makers. Um, so these are, you know, budget owners that have purview over their organizations. Generative AI spend, you know, this is very lengthy survey, but basically, how much are you spending on generative AI? What is that relative to your overall spend? Um, and then very specifically, what tools are you spending it on? Um, I think that one of the motivations behind the survey was that there's just a lack of good data out there. Of like, what are actually IT decision makers really kind of looking at? What are the use cases that they have? Um, and where's their money going towards? And so this is, um, you know, we spoke to six hundred, we didn't get to all, you know, the Fortune two thousand. Um, and so there's obviously, you know, this is directionally our answer.

• 18:43Speaker 03

  Um, but I think that it's pretty informative to see some of the insights of like what folks are actually, you know, not just playing around with anymore, but actually adopting.

• 18:51Speaker 02

  So just to summarize that, um, the four point six billion that we see for the different types of AI applications you guys surveyed on, I should probably pay a little bit more attention to the year of your growth in that number than to sit here and go, why is it four point six and not four point seven billion?

• 19:05Speaker 00

  Yeah, exactly. And then the other thing, you know, the next question you're gonna ask, I was like, okay, great. What are they doing at that application layer? Or, you know, and and the thing that I would say is fascinating about that is that there's a real broad uh, set of Gen A use cases in the enterprise. So sometimes you see things get very concentrated around one department or one use case. But I think it speaks to the the usefulness of the techno- So when we break down, if on average, an enterprise has 10 Gen I use cases amongst it, which ones are most popular?

• 19:44Speaker 02

  Yeah.

• 19:44Speaker 00

  And that would be, that'd be an obvious question. So when we look at it, code generation was number one. Fifty one percent of the folks on the survey said that they're using AI for code gen. They're building software for it.

• 19:57Speaker 02

  Joff, just to be clear, not 51% of the 600 who are paying for generative AI services, but of all the 600, 51% of companies are paying for code completion tools.

• 20:11Speaker 00

  Yes.

• 20:11Speaker 02

  Okay, just wanted to make sure. So this is as big of a number of the 600 as we could imagine. Okay, I appreciate that. Keep going.

• 20:17Speaker 00

  That's not surprising. We've seen GitHub Copilot is one of the fastest growing revenue products out there. And in the application layer of AI, they're north of 300 million in annual revenue. We've seen players like Cognition, Codium, All Hands come about. After code gen is support chatbots. Probably not surprising there. Thirty one percent are saying they're deploying chatbots. We've got products like Sierra, Decagon, Acer, which is from an IT SM, IT Service Management use case is a, is a big one there.

• 20:57Jason Calacanis?

  Hey startups, when you're a business, you gotta treat your customers right. Unreasonable hospitality is the standard today, but you're going to need tools. You're going to need a platform to help you do this. And that platform is the Zendesk suite. The Zendesk suite is going to give your startup all the tools you need to deliver exceptional customer experiences so you can build stronger relationships without growing your headcount. That's key, right? Every dollar matters. You gotta control headcount. You gotta control spend. So use the tool that Shopify, Squarespace, Uber and Instacart all rely on. It's called Zendesk. Let's take a look at another customer, Unity, very famous company.

• 21:36Jason Calacanis?

  They saved $1.3 million with Zendesk automations and self-service and they saw an 83% increase in their first response. Time. These companies love Zendesk because it's so easy to set up and it scales with you as you grow. They'll also give you all the metrics to make your reporting easy, keeping you and your business agile and investor ready. And that's because you're their esteemed customer. And they've created the Zendesk for startups program just for you, where you get unlimited access to all the Zendesk products, expert insights, all the best practices and entry into their amazing community of founders, all at no cost for the first six months. That's right. They want to support you. Zendesk.com/ twist.

• 22:19Jason Calacanis?

  Get ready to scale with the best in customer support with six months free. Nothing to lose.

• 22:25Speaker 00

  Then you move to enterprise search and retrieval 28%. You know, these products like a product like Glean has been around for a while, but with the emergence of Gen AI, what they've been able to actually accomplish on the search front has been dramatically better than maybe what they were able to do in their early years. Another product in there would be something like Sana.

• 22:47Jason Calacanis?

  Yeah.

• 22:47Speaker 00

  Then we move to, you know, things like meaning summarization. You know, so many of us are remote workers and we're so used to seeing that AI agent has been added to either pull a transcript or summarization of our conversation.

• 23:06Jason Calacanis?

  Yeah.

• 23:06Speaker 00

  Things like Firefly and Otter. And then the last one I would mention that kind of round out the top five would be, would be copywriting. And this goes back to the comment that you were making. It's like, you know, LLMs are like calculators for words. They're like really good at authoring content. So you know, you could certainly see a world in which everyone would be using or having a copywriter or an editor, you know, grammar checker, just part of whatever they're communicating to help them be more precise and concise. Products like Writer, Typeface, where we have an investment, Copy AI, they tend to be, you know, some of the leaders in that space.

• 23:48Speaker 00

  But that gives you a feel for like, what is going on at the application layer of the enterprise. So.

• 23:56Speaker 02

  A lot of words is my read of that. I think the calculator for words is a great way to think about this. Co-generation is creating characters on a screen, support chatbots, create characters on a screen. You know, meeting summarization, characters on a screen, copyrighting, characters on a screen. The thing that surprised me is that workflow automation was so low in this chart to date. Because if you go back five years, we're thinking about UI path and, you know, RPA, robotic process automation. And there was a lot of enthusiasm, pre-generative AI, pre-LLMs, that RPA was going to remove a lot of the human drudgery from digital work. Then we got much better tooling, but now that we're, you know, looking back at 2024,

• 24:38Speaker 02

  I'm not seeing as much of that show up as I kind of expected. And this is a long way of saying, how much progress have we actually made on agents? I suppose that can take some of the work from us and do it versus being more assistive in our day-to-day work. And Derek, I see you blinking at me, so go ahead.

• 24:57Speaker 03

  Yeah, I, I, I think that we think about it in terms of waves. And so the first waves of gen AI apps were what we call RAG apps, or Retrieval Augmented Generation based. Um, and so usually, you know, you, you have an external knowledge store and you, uh, you use it for things like synthesis. So Eve is one of our portfolio companies. Uh, it's a legal copilot and what it does is it takes, um, a lot of long, dense legal texts and makes it generates reports. It generates, uh, legal briefings, things like that, so that the lawyer on the other end doesn't have to go through the drudgery of all of that work. And so that's kind of the first generation. And then as you get more advanced, we move into agentic architectures, which, you know, today our survey found that it is a minority.

• 25:40Speaker 03

  But if you ask a year ago, if you look at our previous report, it didn't exist a year ago. And this idea of you had like Baby AGI and AutoGen and some of these like, you know, open source projects back then. But it didn't exist as an enterprise idea of something that, you know, when I think about traditional RPA like UI path, um, the idea of applying it to the enterprise hadn't really existed and now it does.

• 26:04Speaker 02

  So Derek, it sounds like what I was doing was just being impatient and now it is showing up. And so what I expected to happen is I just had my timelines off mentally compared to the market.

• 26:14Speaker 03

  I think the technology is now there. And if you look at things that we're really excited for in 2025, this is one of the things that we think will explode. It's moving from retrieval based architectures to more agents and things that can automate workflows across horizontal areas. So like you think UI path, but also verticals, you know, healthcare, there's solutions like Tenor that are doing kind of ingest automation. And you know, a lot of different domain specific applications, I think you'll also see.

• 26:44Speaker 02

  And I'm literally right now pulling up the OpenAI O1 preview blog post because I forgot the term that I need here. They said time series thinking when models take a little more time before they make a decision or return or prompt. Is that underpinning the improvement in the technology that is making the agentic approach more feasible today, Derek?

• 27:04Speaker 03

  Yeah, test time inference. And so I think that, I think that there's a couple of layers to this, right? So you can have stuff like O1, which is kind of formalizing a design kind of pattern at the model layer. And so as the model gets smarter, it makes fewer errors. I think the problem with agents that you had traditionally is that it's kind of run in a recursive loop. So if you think about it, you know, the agent will, the agent, which is the LLM will think, okay, in order to accomplish this like task that requires 10 steps, what are those 10 steps? And then I'll go out and do it. And then I'll think I'm at step one, okay, check. What is step two? And then if you think about error rates there, right? LLMs hallucinate, that's now well established.

• 27:46Speaker 03

  If you have a 99% error rate or 99% accuracy rate in step one, if you haven't that for all 10 steps, by the time you get to step 10, you know, the error rate is something that is unacceptable for enterprises. And so that's traditionally been the problem. And so LLMs getting smarter is part of the answer. But also when you apply it to specific domains, you have data scaffolding around it. We like to use the term agent on rails, which is basically you need to hard code it or harness. Kind of the domain of like all the actions that the agent can take. You need to kind of set guardrails on it with hard with code in order to point it and get, you know, higher levels of accuracy so that that recursive error rate doesn't compound too much.

• 28:30Speaker 02

  Do those guardrails have to be programmed on a per use case, per industry or per company basis? I'm trying to figure out how hard it is to hard code those because to me that could be incredibly complicated or relatively easy. I just don't know where it lands.

• 28:45Speaker 03

  Yeah, I think people are trying to figure that out, right? And I think that obviously the more specific to a particular use case, the higher accuracy and more robust it is. And but the thing you're trading off really is degrees of freedom, which like, you know, all the way at this very end is AGI, you know, no guardrails, just the model, just like put it in a for loop and it runs. And then on the other end is what we have today, which is computers that are 100% hardcoded. Application logic is determined by the computer or by, you know, first of all, some programmer who sat down and was like, okay, I'm thinking in the shoes of the user, what do I need? And so I think the answer will probably be somewhere in between.

• 29:24Speaker 03

  We're trying to move towards AGI eventually, but I think right today we need stricter guardrails.

• 29:31Speaker 00

  I think, you know, there's, as I'm hearing you talk to Eric, there's a, there's a good example in the software agentic space, right? Okay, so let's take code generation and we can say, well, how much progress are we making in code generation? And one way we can look at progress is we can look at this score called the SWE bench, right? And we can look at that over the last, what's happened there in the last 11 months. So SWE bench, for those that don't know, it's, it's testing real world tasks faced by software developers. And the benchmark was based on things like pull requests and issues from open source GitHub repositories. And I think there's something like 2200 tests in there.

• 30:11Speaker 00

  So if we go back to January of this year, about 4% of those tests were completed by the best software agentic system out there. In March, there was a company called Cognition, which got a lot of, a lot of traction, you might remember that. By March, it was 14%. Now, if you look at it, the number one score on SWE bench belongs to one of our portfolio companies. It's an open source software company called All Hands and they can solve 53% of those cases.

• 30:45Jason Calacanis?

  Hey founders, let's talk about a super weapon that your startup might be missing: Killer newsletters. Newsletters aren't just a great way for you to stay relevant, they're also an incredible growth engine for your startup. Social platforms and search traffic, those things are losing their edge. But your email list, that remains your direct line of communication to your customers. You don't have to pay a middleman, you don't have to get caught up and try to trick the algorithm into getting your message through. But in order to have a successful newsletter, you need a world-class platform. And that's why we moved the This Week in Startups newsletter to Beehive.

• 31:26Jason Calacanis?

  Beehive is a newsletter platform that's built to help you grow your community and turn your newsletter into an absolute powerhouse. It's packed with amazing features like an AI-driven post builder. It's also got this really clever built-in referral program and a top-notch ad network if you want to turn on monetization. Plus, they have really easy-to-use tools if you want to charge access fees like subscriptions. So here's your call to action. Beehive plans start at just $39 a month, super affordable. But we've got an exclusive deal for you here at This Week in Startups, a 30-day free trial followed by 20% off your first three months. That's right, go to beehiiv.com/twist to claim that 20% off.

• 32:11Jason Calacanis?

  beehiiv.com/twist to start growing your newsletter today.

• 32:15Speaker 00

  So if you kind of look over the 11 months, right? It's gone from 4% to 53%, so that's like, you know, 13% improvement from the beginning of the year.

• 32:26Speaker 02

  It's dramatic. And I guess, you know, Joff, how confident are you personally, not speaking for any other company, that that rate of progress can be kept up for the next 12 or 18 months? Because if you run the numbers out long enough, eventually we get damn close to 100%.

• 32:41Speaker 00

  Yeah, look, I think it's going to be invariant. It will depend on what department and what use case that you're moving against. Will it move from 53% to 100%? Next year on the software agentic, you know, I do think it attenuates out. You know, the stuff is fairly unoptimized right now. So I think even with the models that we have, let alone new advancements in the underlying foundation model, I think there's tons that can still be done from an accuracy perspective.

• 33:13Speaker 02

  Okay, so thinking about the categories of enterprise gen AI spend that we started with, discussions about improvements to agentic AI and the approach thereof, and also improvements to existing leading categories. This all sounds very bullish on the technology side. My question is, how much of the revenue that we're talking about is spread out amongst the startups? Because, you know, Joff, you said that, you know, six and a half billion, other thirteen point eight is foundation model spend, OpenAI, Anthropic, a couple of names. When we get to the four point six billion from the app side, the number of companies that are nibbling at that spend is enormous. So is there enough enterprise generative AI spend for apps today

• 33:59Speaker 02

  to support the number of startups that are going after those dollars? Or are we like, is there half as much spend as we need? I just don't know if this is too much bread, not enough butter, I guess.

• 34:10Speaker 00

  You know, it's a merchant. If we look at the software budgets at the enterprise for AI in 2023, they were going towards something called an innovation budget, which a lot of times that's like, hey, we just need to be investing in here. We don't have a permanent budget to pull from. So I do, what I do see happen is you're going to pull from the permanent budgets in time as the ROI improves. The, you know, another point that I would make is that historically, a lot of the budgets that were being spent by both the business unit and IT were around tooling. And back to this comment that Derek had made earlier, it's, you know, now we have services as software.

• 34:52Speaker 00

  So it's not just the tooling budget, but it's the human capital budget that we start to move into as well. So once you-

• 34:59Speaker 02

  Can I explain that to people? Because I think it's a good point, but I want to double click on it. So essentially- If you can reduce headcount and replace that with software spend, you can often get quite a lot of bang for your buck. And so budgets for software for AI that might replace some human activity can be relatively rich because humans are costly and need health insurance and travel stipends and office space.

• 35:22Speaker 00

  And yeah, I don't think it's just about human reduction, right? Like, I do believe that there's there. Companies are also faced with like a lack of having enough people. Like, I don't have enough software engineers, so it allows me to, you know, expand and continue to grow that department, but I can do that in a different way. So it's not just about reducing, it's also about expansion. But but that is part of the story. So I think there's plenty of wood to to chop in there. When in the report itself, you'll see the AI spend by department. And, you know, you'll see departments like the legal department is historically like not spending money on technology.

• 36:04Speaker 00

  They're they're much more of a late adopter and here they are being more of an early adopter. So I think the budgets are very promising and, you know, I think they'll continue to grow as the ROI continues to prove itself out. So from an ROI perspective, last year it was less clear what the ROI was. This year, as we move from our pilots into production, there there's greater clarity. But still some question marks, right? Like, it's not all figured out. The number of folks that were in the survey, I want to say it was something like,

• 36:42Speaker 00

  you know, a third of the survey respondents are saying, well, we're still figuring out our exact implementation on AI strategy.

• 36:50Speaker 02

  But Derek, picking up on that point, when we think about AI, generative AI spend in the enterprise, we're essentially asking, what is the term for that today? And I was just talking about on the show the other day about how Uber, when it was very young, people were comping it against the taxi market. Turns out that was BS because the market got much larger. It seems that if we can unlock spend from departments that didn't have technology budgets to begin with. Like legal, for example, the TAM for AI, generative AI in the enterprise is huge, but also it makes the overall TAM for software itself larger. And I don't think I actually thought that was going to be the case, but it does seem very bullish if I'm understanding this correctly.

• 37:28Speaker 03

  Yeah, I think that generative AI as far as adoption today is more expansive than replacement. And so you have like innovation budgets last year, but it's actually really interesting. We asked, where are you pulling a budget from? And of the permanent budget types, um, a lot of it or over half of the permanent budget for generative AI is coming from new budget. So it is not, you know, I am either, you know, replacing spend for my system of record or some other software. It is, I am creating a new line item for this. And I think that you're seeing that across departments, which is it is both expansive as well as across a lot of different places that used to not spend a lot on technology. And we're seeing, you know, from the startup side, which is where Joff and I spend a lot of our time, you know, we're seeing

• 38:12Speaker 03

  companies pop up all over the map, whether it's verticals, uh, we mentioned healthcare, legal, financial services or departments, sales and marketing, uh, data science, uh, human resources. Like it's, it's really all over the map, which is why we're quite excited about, you know, generative AI as a category.

• 38:29Speaker 02

  Okay. While I would love to just vamp about AI for another like three hours, I want to narrow this down to some startup focused stuff. So you guys had a great chart that's called selection criteria for generative AI tools. And to my surprise, uh, the highest line item wasn't cost, instead it was ROI. And so it seems that when people are approaching AI software, what they want is not to spend as little money as possible, but to have the biggest bang as possible. And so I'm curious, what should startups take away from this particular chart, Joff? Uh, as they approach the market so that way they can see the most success.

• 39:04Speaker 00

  Build things that are incredibly useful.

• 39:06Speaker 02

  No, no one's ever said that before. There's no accelerator that says build things people want.

• 39:11Speaker 00

  Yeah. You know, it will, it's interesting and just sticking with like, what would be good advice on the startup front? I, I do find that, um, you know, you, you want to look at what can you do now that you couldn't do yet? Yesterday, and that's really a thing to focus on. I remember when I was at LinkedIn, one of the things that I did was I started the mobile group there. A lot of my peers were saying, hey, look, we've got 2,000 pages on the website. Let's like jam it into this mobile device. And my point was like, no, that's exactly what we shouldn't do. Like, the mobile device has something that is unique and it's special. And therefore, what we should be doing is focusing on highlighting those use cases. So that it, you know, in time, I believe that LinkedIn would become a mobile company. And indeed, it

• 39:57Speaker 00

  it did wind up coming to become a mobile company. And the product that we built was quite different. There was only 17 screens compared to the the 2,000 that existed on on the website. So I, I think in this moment, you know, you have to look at, a startup should be looking at like, well, what's different? And it's like, well, part of what's different is we have reasoning as a utility. So, you know, if I look at, um, there's a, there's a guy, Daniel Rock, who's a professor at UPenn, he, I've had a number of conversations with him and he was telling me, he's like, look, think of every role in the economy as a bundle of tasks. And those tasks, some of them fall to AI and some of them fall to, to humans. So I, I do think

• 40:40Speaker 00

  what startups should be looking at is like, how can I change? The way, like, if you look at workflow, the way that it's been built over the last 10 years, it's been built with humans at the center. And that's the actual workflow. So if you were to zoom all the way out and say, oh, okay, well, I'm going to have a workflow that's going to be shared. Some of it's going to be through an algorithm and some of it's going to be through a human. What would that allow me to do? Because a lot of these workflows that we see at the enterprise level are very complicated and very tool rich. You know, if you look at a, an average sales team, they have like 12 different tools hanging off of Salesforce or HubSpot. If you look at software development,

• 41:25Speaker 00

  there's over 12 different tools that are hanging off of the software development flow. So, you know, you have an opportunity to reimagine what that workflow should. Be so you're you're either going to enter in and say, hey, I'm just going to try to AI if I the existing workflow. Or I'm going to think about it at a more first principles level. And think about what could be possible, given what we know now. And and that would probably be my number one. Uh, my number one suggestion is really get back to the first principles of it.

• 41:56Speaker 02

  Okay, so I want to pick up on that because you guys wrote, uh, last year incumbents dominated the enterprise market with bolt on strategies. That layered Gen AI capabilities onto existing products and Joff, it sounds much more like you're saying, look, don't do that. Rip the page out, start blank and build from the ground up. So Derek, I think we call it's an AI first approach, just to to building software. Um, what what fraction of software that exists today needs to be ripped out and started over again, because people joke like, I don't know what Salesforce does. We all kind of do, but not really. And, you know, not to make fun of, um, Alasian, but Jira is hell. So, um, I'm curious, Derek, you know, how much I I will not, sir.

• 42:38Speaker 02

  I have filed tickets and I you owe me lunch for the pain you put me through. Don't worry, people, people who may concur owe me a house, so it's fine. I'm just curious, Derek, like, uh, how much of software needs to be rethought in this way?

• 42:51Speaker 03

  Yeah, I don't know that I have a very clean answer for, you know, X percent of software needs to be completely rethought, but it's very across the board. I think that one of the things that we realized this year versus last year is that it's actually not that easy to build AI networks. You know, a lot of people thought last year, like, oh, if I'm a Salesforce, if I'm an Adobe, I can just tack on something AI on top of my existing system of record. And that will be my AI solution.

• 43:16Speaker 02

  Because you have the data already, because if you want a system of record, you have the bucket of data to make your own tuned models with.

• 43:21Speaker 03

  The data, the distribution, the trust with customers, all of that, like, and that is why a lot of people thought, like, is AI really a net new category? Or is it just a feature on top of existing software? And I think what we've realized is actually it is its own independent category. And why it's really exciting for us is because that gives the advantage to startups rather than, you know, Salesforce coming out with agent force. Everybody just being like, this is the greatest thing on earth. And so everybody will adopt it. There is opportunity for startups because people, enterprises we talk to try out features like agent force and realize, wait, this is not what was promised. And so there's opportunities through domain specific workflows, you know, AI native approach to actually make that promise work.

• 44:06Speaker 02

  But if we think about the companies that exist today that might struggle to move from, you know, their kind of legacy software offerings that make them all their money to an AI future. That means that there is trillions of dollars in market cap out there for startups that are being born now with the blank sheet of paper and the modern tools that we're discussing to go after. So in a sense, I kind of like, think we should short the NASDAQ and like double our investment in DC. That would be the best way to get kind of both sides of the bet.

• 44:36Speaker 03

  You know, it's different domain by domain, right? Like it's hard to predict and that's what makes investing fun. Like Adobe, for example, they started as an on-prem company. And then when the cloud revolution came, they actually took a tremendous bet. It's actually really interesting. If you look at their quarterly revenue, when they decided to go from on-prem to subscription, they were able to convert that. And you know, you could, Adobe is one of the companies that I personally admire a lot. Will I bet for them or against them in this gen AI revolution? I personally think Adobe is a really interesting company. Firefly is a great product, but not all companies will do that, right? And so when you look at the startups, both the startups tackling it and what did they offer? Why are they different from the incumbent?

• 45:19Speaker 03

  As well as who is the incumbent in that solution? Are they well positioned to move with the currents versus against them? You really have to take it domain by domain.

• 45:28Speaker 00

  And it is like, it's a very visceral thing when you see a company that gets caught up and left behind in that, right? You can look at a company like Chegg, eighty-five percent of their market cap is disappeared with AI or Stack Overflow, a site that I used to use quite a bit, like their traffic is halved as people go directly into the LLM to get their coding guidance.

• 45:52Speaker 02

  Yeah. I wonder how many of those we're gonna have by this time next year because Chegg. Made money off helping people cheat on homework and they're going to get mad at me about that. So PR team, please don't email me again. That's what we're using check for. We all know it. Now people use OpenAI to cheat. It's great. I wish I had when I was in middle school. What a crushed chemistry. I did not because I didn't. But I wonder how many other companies are kind of on that list and if that will be a good barometer for how fast AI first companies can kind of uproot legacy companies that are, I guess, cloud and SaaS are now legacy and kind of, they almost feel outmoded. Like, do you guys remember when Salesforce invented SaaS and we were like, oh, my God, this is the future. It's weird now to be seen here going, oh, my God, is that the past?

• 46:37Speaker 00

  Well, you know, it's, it's interesting because I, I feel like these are all things that were building blocks that were needed to get us to where we are today. We actually needed the internet to get the world's information digitized. We needed to have cloud computing in order to unleash vast, vast amounts of, of GPUs and CPUs to do training and inference and things like that. So it, it feels like it was more of a lineage building up to the moment that we are today.

• 47:07Speaker 02

  Yeah. But how much credit do we give Yahoo for inventing the internet portal? None today. We don't, we don't even think about it.

• 47:14Speaker 00

  Yeah. Well, what's that saying?

• 47:17Speaker 02

  Xerox in the GUI, right? I mean,

• 47:18Speaker 00

  It's like your, you know, your, uh, my company was built on the shoulders of giants or AI is built on the shoulders of giants. I mean, there, there's a lot of things that, that needed to happen to get us to the state that we're in today.

• 47:31Speaker 03

  Yeah, I think there's building blocks and there's value capture. And I think that a lot of the new value to be captured will be by startups and new companies. But obviously building on the, the shoulders of the giants.

• 47:43Speaker 02

  So glad we ended up here because I have a question about this. I know we're talking about how startups can disrupt incumbents, but there are some incumbent AI companies. And I'm thinking about very clearly OpenAI and Anthropic being two of the largest players in this kind of Neo space. And OpenAI recently put out a, a search product, which is frankly, pretty good. I use it on a regular basis as a testing tool. And I also think that perplexity will see some of its momentum cut because there's now a competing product from a larger, better financed company. And so when you guys are talking to startup founders, how do you help them navigate building something that won't get stomped on by a foundation model company, releasing something that could be considered to be competing?

• 48:27Speaker 03

  I think one of the things to think about is what is on their, what is on their near term roadmap, right? And for a company like OpenAI and Anthropic, their ultimate goal is to achieve AGI, for Anthropic to achieve AGI safely. And what are the things that are going to be, you know, the things that they'll tackle next? And so Anthropic has already mentioned, you know, we won't pursue, you know, side quests such as image generation or things like that, because it is not necessarily the thing that will bring them to AGI. Better reasoning, on the other hand, or using tools like web browsers and computer use on the other hand.

• 49:01Speaker 02

  Computer use, yeah.

• 49:02Speaker 03

  Those are, right. And so a lot of the, you know, when you are an AI app today, there are two things that you really need to get, right? One is, can I make the base technology work? And a lot of the things that actually have happened to date are just, how do I make the LLM reliable? How do I connect it into enterprise systems? How do I give it a tool such as a web browser? And how do I, you know, when I was talking about the agents before, how do I make sure that the reasoning is reliable? But you can think about these as data scaffolding things that, you know, actually are needed today to help act as crutches for the LLM to actually work in an enterprise application. Those, I think, will one by one fall away over time as Anthropic makes advancements in the intelligence of the base LLM.

• 49:47Speaker 03

  But what won't change and what Anthropic won't get to, because quite frankly, it's not important for AGI, is how do I apply this to my, you know, healthcare domain-specific workflow? If I need to, you know, do like clinical documentation integrity, if I need to do RCM on the backend, revenue cycle management. Anthropic's not really interested in that, and so that's an opportunity for app layer startups.

• 50:10Speaker 02

  Okay, but here's my question, because all of that tracks with me, Derek. I agree with you wholeheartedly. But as we get closer to AGI, and as we actually maybe reach it in the near-ish future, doesn't that obviate a lot of the work that's being done to make AI apply to specific categories or market verticals? Because as the AI brain gets smarter, it probably needs less help to do more? And so I wonder if we'll see a dilution of the power of going vertical in AI as we get closer or reach AGI?

• 50:41Speaker 03

  I think the the thing that maybe that discounts a little bit or takes for granted a little bit is the training needed to actually make the solution work. You know, if you let's say that you have a super intelligent PhD level human, right? If you have to, if you take that person and apply them to, let's say, the healthcare example of like, how do you do RCM and work with payers to make sure that your claims get paid? You still have to teach them how to do it. You still have to, you know, there's a learning curve as they get used to the workflows there. And, you know, obviously, if you take that person versus somebody, you know, who may be less intelligent, there's a shorter learning curve there, but there's still a learning curve there and how you actually get that intelligence and apply it to the specific

• 51:25Speaker 03

  workflow on the other end. That is the area where application layer companies can add value.

• 51:30Speaker 02

  Okay, but you're making a a a a jump there that I I wouldn't make, which is you're saying that if you took a PhD level AGI, my hope is that by the time you reach AGI, we're no longer using postgraduate benchmarks to determine intelligence or expertise. We're so far above that that those those analogies don't hold. And then we won't have to have so much verticalization guardrails built and so forth, because in theory, this should be able to be a bit like a magic box. I'm hoping.

• 52:02Speaker 00

  I suppose there's a question too, if there's like, well, one algo to rule them all, or if that algo is a collection of millions and millions of algos, right? You know, think about my phone, like, the power of my phone is actually sitting in the ecosystem. It's that there's an app for that. So there's millions and millions of apps that make that product more productive. So, you know, one version of the world is like, I got an algo and it does everything. Another version might be that I have an orchestration algo that knows when to ask other algos, like, for their expertise in a given area or science. So part of that is still unknown as we move into the future.

• 52:43Speaker 02

  Yeah, I think we we joke about how everything is bundling and unbundling, but I kind of wonder if when we get to AGI, if we're going to have AGIs orchestrating AGIs in a bundle or kind of a singular brain.

• 52:55Speaker 00

  Mother of all bundles.

• 52:56Speaker 02

  Right. Yeah, kind of like if you got, you know, Netflix and ESPN, the same thing. Can you imagine? That would be crazy.

• 53:03Speaker 00

  That would be nuts, Alex.

• 53:04Speaker 02

  I know, I know. So to summarize, though, just kind of taking all this in one, in one bucket. Enterprise generative AI spend grew very quickly and perhaps even faster than expected. We are seeing increased spend on foundation models, but also on the application side. And you guys in the mid low perspective is that the upstarts are going to knock off more incumbents. And probably we're going to see more agentic AI progress next year. That's going to be very exciting. Is there anything else at the, the highest level from this report that I have not brought up? Cause I want to make sure we get all the, all the key bits of meat out, if that makes sense, Jeff.

• 53:42Speaker 00

  Yeah, I think, you know, there, there's one point that I found really fascinating that we didn't punch into too much is that. We went into the enterprise and we said, well, what is, and this is at the infrastructure layer. We said, well, what is the LLM that you're using? What they came back and said is that they're leveraging multi-models. So I, I thought that they would try to like get behind a single model. Whether it's for hedging or for performance or cost reasons, they're, they're typically on average using three different models, which I thought was really fascinating. And then the other thing is, um, you know, just the shift in the enterprise around what is, who is using what?

• 54:23Speaker 00

  From a foundation model perspective, OpenAI clearly had the first mover advantage in the enterprise. But what we saw on the survey is that they moved from 50% down to 34%. So they lost 16, yeah, 16 points here every year. You can see that anthropic doubling coming up. Closed source models are clearly, you know, the majority of what's being used. Meta was flat, Mistral, uh, you know, off a little bit, um, but you know.

• 54:52Speaker 02

  Poor Europe. Look at that. Look at their one AI champion, like fourth in line. We gotta, maybe everyone should spend 10% of their AI spend on, on Mistral just to help France. Because, you know, new administration, Ukraine. Okay. Just me. All right. Keep going, Jeff.

• 55:08Speaker 00

  Man, we're investors over here.

• 55:11Speaker 02

  I love how uncomfortable he got right there.

• 55:13Speaker 00

  Yeah, exactly.

• 55:14Speaker 02

  No, don't touch.

• 55:15Speaker 00

  Exactly. It's like, stand clear, stand down. You're, this is where like that PR training comes in play. It's like, wait, is Alex baiting us? Is this gonna be the one quote that comes out? After all this like wonderful, great conversation? You know, Alex and team are gonna pick on that one time that we blinked.

• 55:35Speaker 02

  Joff Redfern just endorsed all Trump administration policies. I'm kidding, I'm kidding, I'm kidding, I'm kidding.

• 55:40Speaker 00

  Exactly.

• 55:40Speaker 02

  But I, I am, I think, just very net optimistic. And that's a good place to end a year. I think a year of so much change. It does all feel to me very exciting, like we are really making progress. And, you know, the computer use stuff from Anthropic, not to give your portfolio company extra ups. But like, when that dropped, I was as excited about that as I was about ChatGPT the first time I used it. You know? And that, that's a great place to be going into a new year of a lot of investment and progress. So it's all very encouraging. Now, just before I let you guys go, I do have one lightning round-ish question. So sorry for this, but I, I can't help myself. So Derek, we'll start with you. Um, what percentage of your net worth is in crypto or Bitcoin?

• 56:20Speaker 03

  I am actually, outside of venture, a very unsophisticated investor, so, uh, next to none, fortunately.

• 56:26Speaker 02

  Uh, so a little bit via ETFs, sounds like?

• 56:29Speaker 03

  All index funds, um, you know, bet on the US economy.

• 56:32Speaker 02

  Bro, you should see my family's portfolio, exactly the same. And now 0.1% Bitcoin via Fidelity ETFs. Joff, over to you, same cue. Curious how well crypto exposed you are going into 25.

• 56:43Speaker 00

  Zero.

• 56:43Speaker 02

  Zero?

• 56:45Speaker 00

  Zero. Yeah.

• 56:46Speaker 02

  This was supposed to be the freebie, not the, not the, Alex does pick out the quote at the very end and make that into the headline. I mean, Jesus.

• 56:53Speaker 00

  Yeah, no, I, um, you know, I, I focus on things where I think I have like better understanding or a competitive advantage. So similar to Derek, it's like passive on the economy. And then I am way over indexed on, uh, on, on venture investing through a variety of different funds and angel investing. So it's like.

• 57:15Speaker 02

  Yeah.

• 57:16Speaker 00

  If you and if you look at that, it's like, why is he doing that? It's because I feel like I have a like, that's all I've done. I built software my entire life, like startups. It's like, that's the thing I know super well. It has no, and it's like, not a reflection at all on crypto or my beliefs there. It's mostly just about what I I mean, like, I have so much interest in in startup land.

• 57:39Speaker 02

  I appreciate the honesty there because I feel like there's always a pressure whenever crypto does crypto things to appear sophisticated about it for 18 months until it goes away for two years again.

• 57:52Speaker 00

  Yeah.

• 57:52Speaker 02

  But it has been quite loud and the the crypto fans have been making noise. But anyways, guys, thank you so much. And we'll come back and do this again next year for the 2025 generative AI and enterprise reports. I'm sure by then we'll have even bigger numbers to dig through. But before you go, drop your Twitter handles and then we'll say goodbye.

• 58:08Speaker 00

  Yeah, I might use my LinkedIn handle. Go to go to LinkedIn, man. That's where I'm posting. That's my activity. That's the thing I built. So I stick with it.

• 58:18Speaker 02

  Fair enough. Derek, what is your preferred social media handle? I apologize.

• 58:21Speaker 03

  I am on I am on both, but on Twitter, I'm at Derek G. Shaw.

• 58:25Speaker 02

  All right. Thank you guys very both very much. And Twist is back next week for a couple of new shows and then it's Thanksgiving time, y'all. We'll see you then. Bye.

• 58:32Speaker 00

  Thanks, Alex. Super fun.