Ep 37: Co-Founder and CEO of Fireflies.ai Krish Ramineni on How AI Catalyzed Fireflies to 16M Users

According to Ramp’s (ramp.com) quarterly spend report, Fireflies was the 4th highest AI platform by spend. With over 300,000 customers worldwide and 16M users, Fireflies.ai is operating at some of the largest scale amongst AI companies today. This week on Unsupervised Learning we had Krish Ramineni, Co-founder & CEO of Fireflies on to talk about how he sees AI changing the way we work and conduct meetings, and share his biggest learnings around AI in building Fireflies.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Jacob Effron?

  This is Unsupervised Learning, Red Points AI Podcast. And today, we have on Krish Ramineni, the co-founder and CEO of Fireflies.ai. Fireflies is an AI-powered voice assistant designed to record, transcribe, and analyze meetings. They currently have massive scale, over 300,000 customers, over 16 million users, and 75% of the Fortune 500 using them. Ramp called it a top AI vendor by corporate spending, and I wanted to have Krish on to talk about a number of things, from operating at true scale, how Fireflies has evolved from its inception in 2016 through the release of GPT-3 and beyond, and a problem that many early-stage AI founders face today, how to win in the market with huge incumbents in Microsoft Teams, Zoom, and Google Meet. One of the most insightful conversations from a founder that's truly operating at scale. Now here's Krish. Well, Krish, thanks so much for coming on the pod.

• 0:44Speaker 01

  I'm excited.

• 0:45Jacob Effron?

  Been looking forward to this one for a while. I think we're trying to tick our way through all the companies on Ramp's top AI spend chart that they show, and so I feel like you're one of the last ones for us.

• 0:56Speaker 01

  I appreciate it. I was surprised to see our name on that list as well.

• 1:00Jacob Effron?

  No, it's awesome. I mean, I guess I'd love to start just like zooming out. So maybe 10 years from now, let's say we have a meeting as a full team. What do you think that looks like both before the meeting, during the meeting, after the meeting? Like, how is AI changing the way that that works today?

• 1:16Speaker 01

  So before the meeting, having prep come in and being able to know exactly who I'm talking to, what I'm going to be talking about, like even right now, if I had my EA, like give me a debrief on, hey, this is who the person is. Here's some of the topics. Here's what you talked about last time. So those sort of things still, like that you have a human do, you can technically have Fireflies do. So that pre-meeting debrief. And then during the meeting, as I say all of these things, it can be a little annoying because the real work starts after a meeting ends. The moment I hit end on that Zoom link, I have to do all of this other stuff. As a knowledge worker, I have to go fill out my CRM. If I'm in sales, if I have to create tasks in my project management system,

• 2:00Speaker 01

  write documentation if I'm a PM. So there's like endless work that happens after a meeting. But what if you can have your AI assistant that's there? Already doing that work for you as you're talking, like turning your voice into action. And then afterwards, you're having all of these conversations. How do you remember them? The AI is just nudging you and reminding you of, hey, these are the some of the things that you should prioritize or don't forget to do this. So I think that's not going to take 10 years. Maybe in the next couple of years, we'll see that. Maybe 10 years from now, what we'll see is I'll have my agent go talk to your Firefly's agent.

• 2:39Speaker 01

  And then you're going to figure things out. I think the CEO of Bumble said something about, uh, we're going to have AI agents that are going to like date on our behalf. So.

• 2:48Jacob Effron?

  I think that was a Black Mirror episode.

• 2:50Speaker 01

  It wasn't. Okay. Yeah. Surprised I have never seen Black Mirror, but some of the things that I say to my team, they say like, did you get this out of Black Mirror? Or this is so like, this is so crazy. So I need to go watch that.

• 3:01Jacob Effron?

  What will we have to do face to face versus like, you know, letting our agents go off and, uh, and figure things out together?

• 3:07Speaker 01

  Meetings should be used to decide, debate and discuss. So a lot of times, even within our own org, we try to practice what we speak. Any sort of presentation, any knowledge, debrief, I think it should happen before a meeting. And so usually just send me a loom or give it to me over writing. So if a lot of that prep work can happen beforehand, meetings can be super productive in terms of making important decisions and discussing the minute details.

• 3:37Jacob Effron?

  Yeah.

• 3:37Speaker 01

  Yeah. So that's what I think like humans are still going to be responsible. Like, I don't know if you want to have an AI CEO just makes all the decisions. I mean, yeah, I'll be out of a job then. Uh, but.

• 3:48Jacob Effron?

  I'll be out of a job long before you are, but.

• 3:51Speaker 01

  But yeah, I do think that the human creativity, decision-making and the human in the loop process can be far more efficient if a lot of that knowledge transfer can be done before meetings.

• 4:02Jacob Effron?

  Yeah. Well, I think that's a nice bridge into just like, you know, where we are today in terms of the state of play with respect to what Firefly is capable of doing today. And maybe, uh, some of the, the near-term horizon. I mean, obviously some of the stuff you talked about, I feel like you already kind of do today in the product. And so maybe give our listeners just a bit of context of like the capabilities with this current generation of models that we're, we're able to do.

• 4:21Speaker 01

  Yeah. So Firefly's today is an AI meeting assistant. It joins your meetings. Uh, it's an AI note taker. You've probably seen it on one of your.

• 4:29Jacob Effron?

  On many of my Zoom calls.

• 4:31Speaker 01

  And wondered like what it is that actually is that word of mouth has helped us grow today to taking notes for millions of people and we haven't spent any money.

• 4:40Jacob Effron?

  What did you say 5000 years of notes or something?

• 4:43Speaker 01

  5000 years of notes and that was last year. So I'm assuming it's

• 4:47Jacob Effron?

  Many more thousands.

• 4:48Speaker 01

  Yeah, we've done that in half the time this year. But yeah, it's just like if a human had to do that work, it would take a really long time. So we were able to like build this assistant that's going to join your meetings, take notes, and then do work for you afterwards. A lot of those capabilities now are evolving from just this AI note taker, which was kind of the wedge into building this work chief of staff. One of the things we recently released was this idea called the feed. And basically, all the chatter that's happening across the organization, if there's important things that I should know about, but I wasn't able to attend. So if we're making an important decision on which LLM to use, Fireflies will surface that to me in my feed in the morning.

• 5:33Speaker 01

  So it's a news feed that self updates and helps me keep track of all the different conversations that are relevant to me that for meetings that I did not attend.

• 5:41Jacob Effron?

  Like, obviously, I feel like a huge part of that is, you know, there's certainly integration into a bunch of parts of a company, but there's also this prioritization, right, of what's important. And I'm sure what's important to different people and different roles might, you know, and across different companies, like, how do you even get the initial product out the door in terms of, you know, figuring out what to prioritize?

• 5:57Speaker 01

  We start with the source of data and our thesis for starting the company was conversations are where work happens and it is the most important source of data inside an organization. And how can you uncover and make sense of that data? The challenge back then before LLMs when we started this company in 2016, 2017 is LLMs were not there. NLP was not that good enough, so you couldn't even do basic sentiment analysis highly in an accurate manner. You couldn't do summarization, which now seems trivial with GPT-4. And even like the open source ones are really good at doing this stuff. So if you can start with that data source and then see like what value can I

• 6:40Speaker 01

  I deliver to customers and transcription notes is the starting point. And then you can do so much more. And you also want to think deeply about the workflow of the user because every meeting is different. Like a VC meeting to determine if you should make an investment is different from should I make a hiring decision? To am I about to lose this deal? And so what we try to do is look at each of our customers' meetings. And like, how can we help them get that value out of those things? And then integrate to a lot of the downstream systems like Salesforce and Asana and Slack.

• 7:18Speaker 01

  And then the other thing that really helped us build a lot of these features is I feel that AI is most valuable. When it's telling me things that I don't know. And it can help me do some of the minute annoying things every single day. So we try to prioritize those sort of features. I know like Rahul was on your podcast and he said, we try to do these like automatic features first. And so we like to use the same phrase like automagical features where like I just open up Fireflies and wow, I have this feed. Or one of the things that it does is it takes all the action items from all of my different meetings. And creates a ready-made task management system.

• 7:57Speaker 01

  So I can check off stuff and we have a pre-meeting debrief feature which we released where if I'm meeting you two months from now. It's going to tell me, hey, this is what you guys talked about. This is what you said you were going to follow up on. So it's going to hold you accountable. So these sort of things that like, oh, wow, like this is what I would want an EA to do. But not all of us can have an EA, so let's just give it to every knowledge worker.

• 8:18Jacob Effron?

  I feel like you've had such an interesting purview, having started the company in 2016. And obviously you've just seen these models get better and better. You know, maybe just a bit of history of like what some of the capabilities that were unlocked, like when, you know, GPT three initially came out, then GPT four. And maybe contextualizing, obviously, I think the, you know, yeah, when you were able to unlock these capabilities. And then also, what are you still kind of waiting for? What are you hoping GPT five, like when GPT five comes out, like what are you going to test like the day it comes out?

• 8:42Speaker 01

  Oh my god, I think that that 2016 to 2020 period was essentially what people would say is you are looking out into the abyss, chewing glass and hoping things will get better. And I was just 21 when I had started Fireflies and I had like a year of experience as a PM at Microsoft. So what did I know about enterprise software? Absolutely nothing. And my co-founder had just graduated college. So there was so much we had to learn about the industry. I kid you not, Sam did not know what Salesforce was when we started Fireflies. That's how naive we were. So we had so much to learn about the industry, the use cases, the customers.

• 9:24Speaker 01

  And then we were also naive enough to believe in a couple trends. And just like LLMs have proven this, back then transcription, there was a lot of questions around that. When we went out to raise our seed round in 2019, a lot of the same questions were asked about transcription. And we took the bet that the cost of transcription was going to go down. It's going to be human level accuracy. No one questions that today, but you know, even two, three years ago, that was a question. So once you're able to capture that data, you can do a lot of, lot of things. So those early days was about unlocking that and just making that really good. And people at that time were essentially just buying Fireflies for transcription because I don't want to hire

• 10:07Speaker 01

  a human to do it because if I had to hire a human to do two hours worth of transcription for a board meeting, that's more expensive than an entire year-long subscription with Fireflies. So now transcription is like commodity and everyone expects like perfect transcription. So now you look at the summaries and the notes and the action items. And there's a lot of work that the LLMs have enabled. Before we were using NLP, it wasn't that great. It would just take chunks of the text and then slice it up. I remember in 2018, Salesforce released a paper on summarization using sequence to sequence learning. And we said, what if like this, this, this stuff, you know, if it plays out and then you had GPT-2 and then GPT-3? Like what if this plays out?

• 10:52Speaker 01

  And we can actually like have human level paraphrasing, not just extraction of text. So that's where we started really, right? Like, I think companies today have a lot of luxury. But yeah, we were starting with like very like basic stuff that just the viability of the product didn't exist. So I think we were a couple of years too early to the market, if I have to be honest.

• 11:15Jacob Effron?

  Well, no, I mean, it seems like it's it's worked out having been there. What is like, what still can't these models do? Like, you know, what are you, you know, what have you guys tried that you really, you know, want to put into the product? But it's just GPT four and, you know, the latest models just aren't good enough yet.

• 11:29Speaker 01

  Yeah. So, you know, once GPT five comes out, like you said, or some of these other models, like the level of intelligence is going to go up, right? The the next model may be as smart as a PhD, the current like GPT four oh. And then Claude three point five are as smart as a, you know, high school student, college student. And then GPT three might have been more like a 10 year old. So like, when you look at these sort of models, the things that are really important to understand for any founder is you're trusting that model to do a whole lot of things. But if it's not repeatable and consistent, that is a problem. We have seen with the newer models, the consistency in the answer has been good. Like you can give it the same input and get a completely different answer.

• 12:14Speaker 01

  So how do you control for that variance was a big challenge we had to focus on.

• 12:18Jacob Effron?

  It's funny, I noticed Sierra the other day, the customer support company published something where they were like, well, people think support's accurate because these these queries are accurate, you know, I don't know, call it 80% of the time. But if you run the same query eight times, the odds that it's run correctly eight times are like 20% or something.

• 12:33Speaker 01

  Yeah.

• 12:33Jacob Effron?

  How have you guys solved that problem?

• 12:35Speaker 01

  I think when these companies release their metrics and scores, that should be definitely one thing to include in in terms of consistency of results. So we had to build our own A B experimentation platform. I don't really believe in fine tuning and I have some reasons for that. But what we have done is we've had to do a lot of work with prompt engineering, like certain constraints, even just like the format, like the JSON format. And the query parameters can completely mess things up. So we've had to like. Basically, keep telling the AI to like work within the confines of the information has and not get too creative. Or so those are like, simple things that we've had to do and a lot of experimentation.

• 13:18Speaker 01

  And we've actually like tested it on like different models like from every single vendor out there. And then we like actually let customers rate the response. And then we have like a stack ranking algorithm. And we realize for like a perfect summary, maybe certain models do certain parts of that really well. One is going to do an overview really well. One is going to give you shorthand notes really well. One's really good at creating action items. So we have to be flexible in terms of using all of the different solutions. But for the problems that we're solving with like summarization, like I think the level of models that are there today are sufficient. And the real things that I'm excited about, let's say, six months from now or 12 months from now is

• 14:02Speaker 01

  how do you take what's coming out of those meetings and then take actions and recommendations and decisions? That part, I think the smarter the model is, like the more stuff that it can do. And also, we're getting to multimodality, right? So I saw something on Twitter where someone put up like their password for their wifi on the screen. And then it did the screen recognition and then like logged into the wifi. It's obviously scary Black Mirror stuff, but it would be great like as I'm talking, it's able to like go do a background check or a reference on a candidate. Maybe run some stuff through perplexity and then Fred is able to like, give me a bunch of information in real time and recommend stuff to me.

• 14:46Speaker 01

  I think when you have these Omni models or multimodal models that are intelligent and the latency gets lower, we can do a lot of crazy things.

• 14:56Jacob Effron?

  That was really interesting. I think a lot of the stuff you said there really resonates. Obviously, there's going to be a set of actions that you're going to want to take. So it's like, okay, we talked about like a next step on this account, like update Salesforce or do, you know, whatever. Or we're trying to resolve a ticket and, you know, update, you know, our ticketing software. But also this idea of, you know, today you could give a very specific ask to a model of like, hey, you know, we're talking about this market, like go figure out the market size. But how do you kind of maybe a broader, a broader question? How do you go get a model to kind of start doing research for you in the background? And that those are, those are really interesting capabilities.

• 15:29Speaker 01

  Yeah.

• 15:30Jacob Effron?

  I'm excited for that to be in your product.

• 15:31Speaker 01

  I would love to like, we just talk about all the tedious things that we have to do. And we dog food our own product every single day. And it's like, oh, it would be nice if Fred could do this. And so we tend to kind of think about it. I also like to believe in this agentic future where it's not just one agent that rules them all, but it will be a bunch of agents talking to each other. So the Fireflies Fred agent for meetings might be talking to Harvey dot AIs like a legal agent to help draft up something that was discussed on a meeting. Or perplexity can just actually go check if someone on the team said some stat that didn't make sense. It's able to go on Google and like fact check.

• 16:11Speaker 01

  So I think the world where these agents are really good at doing certain things, whether it's like answering questions or search or legal stuff. Like then you get all these agents to work together, similar to like APIs.

• 16:26Jacob Effron?

  Totally.

• 16:26Speaker 01

  Uh, I think that's a cool future to look for.

• 16:28Jacob Effron?

  No, and you guys set an interesting position sitting on top of like, as your point most work at stunning conversations. And so sitting on top of those conversations.

• 16:35Speaker 01

  Yeah.

• 16:35Jacob Effron?

  Is really interesting. You said something in your previous answer that was too juicy for me not to follow up on. It's like, you don't believe in fine tuning. I would have thought, you know, someone could look at your business and be like, oh, like they probably have some general models. And then like, once they get really deep in a company, you know, and they learn what's important to that company, they fine tune on it or what's important to, uh, you know, the sales team versus the finance team. Uh, but it sounds like that's not what your guys are doing. So I'm excited to dig into this hot take. Why, uh?

• 16:56Speaker 01

  Yeah.

• 16:57Jacob Effron?

  Why the lack of belief?

• 16:58Speaker 01

  So one thing we don't do is train by default on customer data. CIOs like that fact about Fireflies. And a lot of AI companies are itching to use that data to build general models and, uh, and stuff. So we think one, fine tuning is expensive and the returns are diminishing over time as the model gets a leg up. Also, we're in a market where it's not changing on a yearly or monthly basis. It's changing on a weekly basis. And you have to throw a lot of your assumptions outside, right? Completely out the window. Uh, what we had, uh, maybe when we were at GBD three, where fine tuning made more sense versus now GBD four. Maybe, you know, you look out a year ahead.

• 17:41Speaker 01

  It just makes less and less sense to do because it also slows things down. I think you can do use do use prompt engineering and context from meetings to help a customer, right? Be like that. That context will help the AI itself do really well. But in the long run, we felt like fine tuning was just not optimal in terms of the speed at which the market is changing. I think this is a general trend for a lot of things. So the moment GBD 4.0 comes out with low latency voice to voice interactions like what they showed in the demos. I feel like so many of those contact center companies that have spent two to three years.

• 18:20Jacob Effron?

  Latency, right?

• 18:21Speaker 01

  Yeah, latency, right? And like had spent billions of dollars on R and D to get there. We'll probably say, you know what? Let's just go ahead and adopt this and like throw what we had out the window. I think that's something we've had to be really good about as well, like letting your Legos go and say like, hey, this is not going to work. We gotta like work with a new toolset.

• 18:42Jacob Effron?

  Yeah, so this is one of the questions I was most excited to ask you about because I feel like having done this since 2016, you have seen some step change improvements in these models. And I feel like everyone today is asking, you know, okay, you can, you can take GPT four for law or healthcare or accounting or whatever. And you can solve the gaps that like don't work in the model today. And then there's always this risk that like all the work you've done, GPT five comes out and it's like, okay, cool. Like you get it for free. But at the same time, like in the here and now, there are customers that want a product and there are shortcomings in these models. And so I feel like so many builders are asking themselves this question of like, like, what should I build around these models today? Knowing that things might change, but also knowing that there's a customer need here and now. You guys have experienced this through like four different generations of these models, like teach us.

• 19:25Speaker 01

  Oh, like how to adapt?

• 19:27Jacob Effron?

  Or like, how do you prioritize what's like the right set of stuff to build? Given that it could be obviated, like I'm sure you guys worked on a ton of transcription things that now you get for free in GPT four. But also I'm wondering like in retrospect, was that worth it? Was that like, you're like, oh, man, I wish we'd kind of just like gone on the beach for two years and then come back when like GPT three and a half was out.

• 19:46Speaker 01

  Yeah, so we did like spin up our own ASR engine. We even like had an initiative to spin up our own open sourced LLM and use like Llama and these other things. But we realized, you know, as a startup, you have very limited resources and focus is going to be extremely important. Especially when you don't have billions in your war chest. I think even 12 months ago, there were companies that were convincing investors that in order to build their end application, that creating their own LLM from scratch was the right answer. And they poured billions into that and we're seeing now that that was not the right answer and those companies didn't have traction. They struggled.

• 20:31Speaker 01

  They had really expensive AI engineers. They were trying to get data in some ways. So eventually those companies ended up crashing. So I, I felt that it was better to take a bet on where the industry is going. And in fact, I give that credit to my CTO where he was very bullish on let's ride the technology wave. And in Firefly's experience, there were like three factors that really have been transformational. One is the transcription, the cost going down, accuracy going up. The second is, uh, this is more go to market, but like COVID made video conferencing adoption go through the roof. But why do we have more video conferencing today than maybe if COVID had happened in 2011?

• 21:16Speaker 01

  I don't think the internet connectivity, uh, the infrastructure and speeds would have been good enough for companies to be like fully remote. Like I, I don't know. Um, the third piece was like these LLMs that brought intelligence and the cost of intelligence has gone down. So if you're trying to build in this market, the only thing that is really going to be defensible, whether you're competing against large incumbents or whether you're trying to compete against what a LLM is going to do at the base level, is what problem are you solving end to end for a customer? And like, how deep are you going to go in that workflow? Cause even in our market, there's going to be Like Teams and Zoom that are going to say, okay, we're going to give transcription.

• 22:00Speaker 01

  Okay, general note taking. So there's a difference between providing a note taking solution versus saying, I'm going to help you make an important hiring decision, or I'm going to help you close a really big deal based on the conversation that happened, or I'm going to fill out all your ERP systems. So I think that it's better if you're an application layer company to think about your defensible moat being how deep can you go into that customer workflow? And I think that is more important than trying to build a moat around like the very basic stuff that these LLMs can do.

• 22:38Jacob Effron?

  And you think it's worth it, even if, you know, maybe two years from now, you get it for free in the model, like to solve the here and now problem for the customer?

• 22:44Speaker 01

  Yeah. And I think this will also impact how people think about pricing and it'll impact like what people give away for free. So there's all this debate around utility versus seat-based pricing. I think the solution for Fireflies is a hybrid version of this. But as these models become cheaper, our general belief is we're a very self-service, PLG-oriented company. We're going to give all of those benefits down to the end user. And if commoditization happens, we would rather be the first to commoditize it and make it readily accessible to all of our customers. But yeah, I think that it's really important that, because I talked to founders who are raising money and then, you know,

• 23:27Speaker 01

  they came to come for advice. And sometimes you have to sit them down and say like, are you really raising that money? Because you're an AI company and you have the hype to do this? Or are you actually going to use that to do something fundamentally different? Whether it's like with engineering or building models. And a lot of times, they don't have a very clear answer for that. So that's something that I would say is like, sometimes there is beauty in constraints and you should really focus on solving deep customer problems. And it doesn't matter what you're using under the hood. Eventually, the cost will come down, even like GPT-4O, the cost will come down by five. In if not next year, in the next couple of years.

• 24:10Speaker 01

  So you don't want to try to innovate on what is a race to the bottom.

• 24:15Jacob Effron?

  Yeah.

• 24:15Speaker 01

  Yeah.

• 24:15Jacob Effron?

  No, I love this idea that like whatever your most cutting edge features are today might actually be the ones you just give away in a few years as the capabilities continue to improve. Do you think you still charge like a per seat license in the future or do you, you know, as you move more toward this chief of staff, are you kind of moving into the selling the work, you know, type uh thing where you're really giving an extra employee and you price, you know, based on that?

• 24:35Speaker 01

  Yeah, like with our model, I do believe that there is going to be a seat based pricing so that every person gets incredible value at the base plan, right? Unlimited transcription, unlimited note taking, uh being able to like ask Fred questions. And then for work that Fred is going to be able to do for you, uh that are really complex tasks that may require a lot of intelligence. Uh, and it's costly, like from a compute side, we would potentially do that based on like value based or utility based pricing.

• 25:07Speaker 01

  And I think that's where uh you're going to have a combination of both. I'm a big believer in this like Costco, Sam's Club model where, you know, with that subscription, you're giving as much insane value as possible. And then all the other things that you get, like just you using it for one month, you recoup your costs. Like that's the sort of mentality that we have. And also we've run our company like a bootstrapped company. Uh, so you know, I'm happy to say that we're profitable today because of that dynamic and that mindset that we've had. You know, very few companies.

• 25:43Jacob Effron?

  You haven't gone and built your own massive model for.

• 25:45Speaker 01

  Uh, that that helps too. Uh, that helps too. And then, uh, I think even like for uh like companies that are building great products. You know, like different industries have different margins, right? Like e-commerce is going to have low margins. But people are now just getting to that realization, oh, L L M's AI compute, there's real cost to this. But I also really believe that today it may not matter, but eventually you'll need to figure out like, I can't sell something for 90 cents that I'm uh that I uh that cost me a dollar to make.

• 26:20Jacob Effron?

  Are there things that you'd like want to do in the product today, but like the inference costs are actually prohibitive?

• 26:25Speaker 01

  We've been good about really offloading some of that stuff to customers and letting them make that decision, like which model they want to use or some of the tasks that we want to do.

• 26:37Jacob Effron?

  You just charge more for this?

• 26:38Speaker 01

  Yeah, I think the auto magical features, like it does, because if you have to run that across a ton of meetings, it can be. So we have features today that if I, and I think anyone who's an ex consultant might love this, like anyone in McKinsey, it's not going to take over their job, but it is something, because a lot of times it's about like combing through all of this research and all of this like due diligence and then coming up with insights that you present, right? So one of the things that I like to do on a regular basis is, hey, go through all of my team's sales calls for the last, you know, six days,

• 27:17Speaker 01

  seven days, and then tell me which are the most common feature requests and what type of customer is asking that information. So what would take a PM or a user researcher probably like 10 hours to like comb through, it's going to be able to do in a matter of minutes.

• 27:37Jacob Effron?

  Totally.

• 27:38Speaker 01

  So that is an expensive query. It may cost us today a couple of dollars. Maybe it'll go down in the future. It should go down in the future, but those are the sort of things that where we want to really unlock the knowledge on top of this like work data for you, you can do some incredible things and you can give insights like instantly that will help you run your business in a very, very data-driven way.

• 28:01Jacob Effron?

  Yeah.

• 28:02Speaker 01

  So that's just like an example of something I use today, but like, yeah, if we had to put that into production, that's going to be expensive. We have to figure out how to do that.

• 28:07Jacob Effron?

  Totally. Also, and I'm sure, yeah, if you wanted to automatically, you know, there's a version where you ask the question and say, okay, we know it's valuable, but also like maybe there's a feature version of the product where you just are constantly surfacing like here's the, the most recent updates from like, you know, these three sales calls on, on product and, you know, running that in the background constantly, I imagine could get expensive.

• 28:25Speaker 01

  Yeah, we've also, because we have the video from the meetings and you probably go on social media, you see all of these like reels from different podcasts and stuff.

• 28:35Jacob Effron?

  I mostly just watch our Redpoint TikTok.

• 28:38Speaker 01

  So we have like a feature that automatically creates soundbites from the most action-packed moments of the meeting. And then it creates like this like highlight reel. So you can go to these clips, which are like very engaging. It's all AI generated. Yeah. So, and I've used to do it manually. And then I said, Hmm, let's just put LLM into this and like do it. So you can make automatic clips. So those are the sort of things we want to keep delivering to customers where anything that you have to do manually, can we help augment and assist you? Yeah.

• 29:09Jacob Effron?

  Yeah, no, that makes a bunch of sense. It's interesting, that example you just gave of of sales. You know, I wonder how you think about the extent to which like a horizontal product makes sense here versus one that's like a tat, like both job specific or industry specific. Obviously, like you have Gong and Chorus and folks that have, you know, just focused on sales calls and probably wants to build lots of workflows around that. And then I imagine like, there'll be some companies that are like Fireflies is great, but like law meetings are so different, or like doctor-patient meetings are really different, or I don't know, there's some set of meetings. Like, how do you think about, yeah, when a horizontal tool makes sense and whether we'll see like, you know, once you guys are, you know, as you guys continue to be so successful, are there going to be like 20, we're just like on Fireflies for, you know, X, Y, or Z?

• 29:49Speaker 01

  That's a great question. My controversial opinion at the very start was that vertical SaaS is extremely difficult. And usually, vertical SaaS is incredibly overpriced because they have a smaller set of customers to go after. It's more sales-led, and then you have to build a really good narrative around it and then justify the pricing for it. So in a world where general intelligence gets better, I'm not sure vertical SaaS is necessarily defensible. Like, I'm talking specifically like, hey, we are CRM, but just for finance people. Yeah. Like, technically, if that is like a different overlay

• 30:33Speaker 01

  or UI, couldn't like the LLM get customized for you? This is why I think companies like Monday.com and Notion are doing so well, is because they take this general horizontal

• 30:48Speaker 01

  And then let you customize it in however way you want, right? So monday.com can be your CRM, it can be your project management system. And it can be for different departments and so forth. So I think we're seeing a rise in more of those. And that has nothing to do with AI, but just in terms of people are getting more savvy. But I think if you couple that with AI, I genuinely believe that like you are going to be able to do a lot of the work. Maybe some of the deeper end workflows, we won't be able to solve fully. But that's why like companies like Slack have app stores. And that's why we're building our own AI apps. So if you're in real estate and you want to do something unique for real estate, based on your meetings, you're going to have a bunch of people that are going to build real estate apps on top of Fireflies.

• 31:32Speaker 01

  At least that's the long-term goal. My other take here is that for most knowledge workers, notes, tasks and contacts, like who you're working with, who you're talking to, is 70 to 80% of your day-to-day work, right? Like what work do I need to do? What are the important topics that are being discussed? What are the notes for it? And then who am I talking to, right? So that for us was like the building blocks that we wanted to build for for users. And so if someone wants to use Fireflies in that manner, like we're going to just make it super customizable. And you just tell Fred that, hey, I am a person in pharma.

• 32:17Speaker 01

  And these are the things that matter to me. So surf as these sort of insights and then these sort of recommendations. And I think that's, that's what we're going for.

• 32:24Jacob Effron?

  It's so much, I love this point, like obviously traditionally you had horizontal software that's cheap and broadly, you know, like broadly applicable and then vertical SaaS that's, you know, high priced, but so industry tailored. And you could justify that because the, you know, there was so much industry specific stuff you had to build. If AI makes horizontal software way better and provides like an easy UI to customize it for your vertical, you have the end user do it, like your vertical SaaS product better be pretty damn good. If you're going to try it, if you're going to have to charge, we need to charge to distribute it that way.

• 32:52Speaker 01

  I think this is going to take many years, maybe multiple decades to get there. But just to see where the technology is, and if you just go on a first principles basis, like, why not, right? Like, so if you have GPT-4, that's really good at analyzing your earnings reports. But then it can also take your lab work and give you a pretty solid answer on like what those lab report data looks like. So what's stopping general intelligence? I'm not even talking about AGI, but like general intelligence in terms of like SaaS, like from doing those sort of things and threading the needle, right? So I do think that the world that we're kind of moving towards

• 33:35Speaker 01

  is where you will be able to kind of bundle up software in an interesting way.

• 33:40Jacob Effron?

  Yeah, I mean, maybe there'll have to be these like, you know, to your earlier point about agents working together within a corporate context. Maybe someone is still building the pharma-specific agent that really understands, you know, pharma sales or something. But like, they're not accessing it through, maybe they're accessing it through you guys. And it's just like, they never even know that thing exists, but someone else has built this agent that has that industry context.

• 33:59Speaker 01

  Yeah.

• 34:00Jacob Effron?

  That's really interesting. I guess, you know, another thing that you mentioned earlier, that I imagine like every time you pitch this business, the first question people ask you is like, well, what about like Zoom or Google Meet or Microsoft? And I think you made the point, which makes a ton of sense, that, look, there's just a level of depth of building out workflows and solving customer problems. That plus I imagine people use all of those tools. And so it's nice to have one solution that sits across them. So I totally get, you know, the argument you make against like, hey, the incumbents just win everything. Are there some instances though where you like, well, someone will bring you an idea or something and you're like, eh, I feel like the incumbents are probably just gonna, gonna, gonna win that category. Like, how do you think about that?

• 34:34Speaker 01

  Oh, wow. That is a question that you'll get from a lot of investors. I have some thoughts on investors and then I-

• 34:42Jacob Effron?

  Well, definitely start there.

• 34:43Speaker 01

  We can start there. I think when we started the company, it took a long time to figure out and raise our initial amount of capital. And we were so exhausted at that point in time. We both, my co-founder and I, said that you never want to be in a position where you have to- Depend on others for capital. So we need to do things so that we control our own destiny. So we made it really important to be efficient, to be profitable. So we raised total today only about $20 million in funding, considering other AI companies. A small seed. Yeah, it's a it's a seed for AI. So across our seed and series A.

• 35:23Speaker 01

  And our last funding round was in 2020, end of 2020, 2021, early 2021, when we announced it. So since then, like, we have to make a lot of like decisions in terms of, uh, okay, how are we going to go about doing this? And I think investors are going to have 20 reasons why you shouldn't do something. And you may only have one reason for why you want to start the company. And that might be enough. That might be enough to to do what you want to do.

• 35:53Speaker 01

  I think when it comes to competing with incumbents, they have incredible distribution. So there's huge advantages there. So you do have to just execute better on the core stuff. And you have to go deeper on the things that for them are just checklist features. Like AI could just be a checklist feature for many of the larger companies. Because it looks good, like on their earning reports and their stock price and stuff. The incentives are different. Whereas if you're a startup, you shouldn't be discouraged if you are building AI first. So everything you're going to look from this like new lens of how products are built. You don't have a lot of this baggage. And then the one thing I've always underestimated is like corporate bureaucracy.

• 36:37Speaker 01

  So you have the advantage to be lean, be AI first. And just you have to out innovate your through a lot of the problems. So that's the only thing we said. And a lot of times that's not enough to tell a VC saying, yeah, we're just going to like execute better. And we're going to go deeper. We're going to stay in our lane. We're not going to spread ourselves too thin. But a lot of times that is the answer. And you have to just show that through execution. But I will also say like for folks that are fundraising today. AI is insanely like overhyped when it comes to fundraising. I'm sure you see it all the time, too. And then there's startups that are chasing valuations, which they will not be able to justify.

• 37:20Speaker 01

  I think foundation model companies get a pass because you do need the resources to build those. You're going to have a handful of that are going to be successful, like OpenAI, Anthropic, Mistral. But a lot of folks, I think, who tend to be disciplined investors, but because a deal is getting very hot, will overspend on an overly valued company. And then later, they may miss out on some great companies because then they start saying public multiples and market market interest rates and all of those sort of stuff. So I think it makes sense to be pragmatic. There's going to be some category leaders for sure.

• 38:02Speaker 01

  But yeah, we've been doing our own little thing outside the Silicon Valley bubble.

• 38:06Jacob Effron?

  No, it's super impressive. And one thing also that I thought would, I imagine, was difficult in the early days competing with these incumbents is if you think about sitting on top of someone's meeting data, it's about as sensitive as it gets. So I imagine, like, even if Microsoft doesn't have the greatest product, they're like, we're Microsoft. And people are like, okay, like, you know, you're probably going to be fine with our data. And then you guys pop up and, you know, people are like, obviously now you're much more popular. But in the early days, I'm sure, how have you like gained that trust? Or like, what have you learned about doing that? Because I feel like a lot of startups are facing that where people just know the incumbents and kind of figure they'll take care of their data.

• 38:39Speaker 01

  Yeah, I think credit has to be given to the incumbents in terms of how fast they've been moving within this AI world. Microsoft is not going to make the same mistake that happened on mobile. They've been incredible in doing this work with AI.

• 38:58Speaker 01

  It's a funny like journey for me because when I was younger, I did a school report on personal computing and then wrote some letter to Bill Gates about how this is going to change the world, like, like everyone's going to have a computer. We're going to do so many cool things. Funny enough, I got like a letter back from Microsoft. It was like some fan mail thing saying, hey, we encourage kids in STEM. I think my mom had it, you know, till we moved. So that was a really interesting thing.

• 39:27Jacob Effron?

  They should have hired you right then. That would really help.

• 39:29Speaker 01

  So a decade later, I worked at Microsoft and it was a very like interesting journey, like kind of moving. And I had never imagined that's what would happen. And then just a few weeks ago, I'm sitting in front of Bill Gates, like talking about like how Fireflies is working, also knowing in my mind, they're working on Copilot stuff. And he said, okay, I see how you guys are taking a lot of this separate data from all of these like different meeting platforms and different stuff. And then you're trying to do these sort of things. So like, it's not just Microsoft, which is going to do it for Teams and their sort of things. Also, Microsoft has a much larger user base, so they're going to have to

• 40:11Speaker 01

  support like a lot of the general features for everyone before they go do, you know, specific workflow related stuff. So yeah, it's just really interesting that, you know, I have deep admiration and respect for Microsoft and they're incredible in terms of what they've been doing. And yeah, like I don't see us as like competing directly with like Copilot, but like being able to realize that, just like how every company was a cloud company at some point in time. Every company is going to be AI enabled. They're going to have some of these like AI features. You just don't want to remain as a feature and you want to build like the platform and the ecosystem around it. Yeah.

• 40:51Jacob Effron?

  Totally. I guess to that effect, like what are the metrics that matter most to us business? I mean, obviously I imagine there's some single player stuff, there's multiplayer, there's probably ways you want people to use the product. Like what's kind of top of mind for you in terms of managing the business today?

• 41:05Speaker 01

  For us, it's about the downstream value we can generate for customers. So we're entering all these systems of communication on your behalf. And then how do we go into these systems of records and make them better for you? Is Notion fundamentally better for you because you're using Fireflies?

• 41:24Jacob Effron?

  Is that like a metric you use to measure that or?

• 41:26Speaker 01

  Yeah, we just look at like the amount of data routing that we're doing. So. Peter Reinhardt from Segment is an angel investor in Fireflies. And I loved the value that Segment creates and Zapier creates, where they are this connector, this piping that runs your company. And it helps you, like, it's hard to rip and replace once it's in. And in the same way, we want to do that for conversational data. So we're building this conversational infrastructure is how we think about it. And then we're taking all of this incredible knowledge that's coming out of all of your different conversational systems and then just making all the downstream systems better.

• 42:08Jacob Effron?

  It's super interesting that the North Star has to be the way it interacts with the rest of your tools and organization, not just like, oh, cool, I have my Fireflies thing and it sent me a few bullets from that meeting. It has to actually flow into other products in the company. You also have, like, on your website, you have a bunch of specific use cases for how folks should use Fireflies. And I'm wondering, I feel like a lot of AI companies, you probably don't have this as much, but there's this blank canvas problem where it's like, oh, it's a really cool tool. And it's like, there's a ton of things it could do. And I'm sure most people add it to a meeting once and then get, like, how have you thought about teaching people, I guess, because there's so many different ways to use it and you have all these different things the product can do? What have you learned about teaching folks?

• 42:44Speaker 01

  Wow.

• 42:46Speaker 01

  So the blank canvas problem is real, especially when you're trying to teach people how to talk to AI. What we've learned is sometimes to start with a recommendation and then to be able to kind of like a decision tree, just branch off from there. Perplexity does this really well, right? When you do an initial query, it'll give you like five other questions that are related. So you want to nudge users and, like I said, like, give them things that they might not be thinking about, but are highly relevant to the work that you're doing. AI is going to be incredible at recommending stuff. Like, I genuinely believe that the incredible recommendation algorithms that you see the

• 43:28Speaker 01

  big companies like Facebook and Google do, more small players are going to be able to do that, thanks to this general intelligence from LLMs. For us, what we found is that everyone has meetings, everyone wants notes, and everyone wants to. To be organized and have all that information documented. And they're trying to reduce clutter, find the needle in the haystack in terms of the knowledge. So things like search are really important for us in terms of searching across your meetings. Being able to find actions that you need to do, things that you promised. So like for us, like that still applies whether I'm in sales, whether I'm in recruiting, whether I'm, you know, in finance. So those are the sort of things that we start with.

• 44:13Speaker 01

  And once people get good with those, we then want to build out some capabilities saying, hey, given that you're in finance, these are like five, six different applications that you can run on top of your meetings that is going to be really helpful. Or, hey.

• 44:30Jacob Effron?

  I love that you get them comfortable first before, like you don't just like bombard them in the beginning like, oh, you're in finance.

• 44:34Speaker 01

  You should, yeah.

• 44:35Jacob Effron?

  The 10 things to do.

• 44:36Speaker 01

  Yeah, I actually like learned about, you know, like some of the best companies in the past. Like it's important to build for your power users, no doubt. But a lot of times like, and we try to like, listen to every piece of feedback that customers say. And you don't want to get too complicated. And then this is where you will have companies that'll come in and then do that basic stuff really well. And then like start to take your market. So you want to still be simple enough for people that are signing up for the first time and wanting to use it for the first time. Cause every product gets feature creep and it gets more complicated. And then a new company comes around that says, hey, we're like them, but just a simpler, easier, cleaner UI version of them.

• 45:21Speaker 01

  And so for us, in order to not be a relic of the past, like we have to be really focused on getting people through that journey.

• 45:27Jacob Effron?

  You mentioned you use a lot of different models. And I think you, you said something actually that beyond it source graph said too, which is that like, one of the best ways to eval these models is just throw them at customers and see what they like better. Like, do you guys have a way you do internal model evaluation and anything learnings around that? Or you mostly just, you know, hey, there's new models. Well, we'll give them to customers and see, you can tell from usage, whether they're finding it valuable.

• 45:46Speaker 01

  Yeah.

• 45:48Speaker 01

  We've actually spent a lot of time building out our own toolkit in terms of rolling out different models, A/B testing, measuring it, getting enough data. So we have an internal framework to do all of these sort of things. We can do an eyeball test and say, this looks great, but ultimately, we believe that customers should be the end judge of it. And because we have a lot of end users, we can quickly get to a strong signal, like positive or negative signal with some of these models. And then there's different ways, like, we've been so granular at testing like two to three words in a prompt, like removing it versus including it.

• 46:30Speaker 01

  And so when you have like enough of that user feedback, you can just continuously like optimize and chip away at the problem.

• 46:36Jacob Effron?

  Huge benefit of having a large, a large user base. I mean, another thing that's clear when you use the product is it's just very fast, right? Like, any tips you can share about like, how have you gotten it to be so fast?

• 46:46Speaker 01

  When we first started, it would take about 30 minutes to process a meeting and then get you the notes and stuff. Then we eventually got it down to about like 15 minutes and then 10 minutes. And there was a very positive feedback loop around, the faster I can get you your notes and all of that information, the more likely you were to look at it, the more likely it would be useful, the more likely you would forward it to other people. So we've continued to like optimize our infrastructure. Also, transcription service is getting better, LLMs are getting better. I think one of the hardest challenges at Fireflies has actually been managing the scale.

• 47:25Speaker 01

  Like, LLMs aside, you know, we're joining millions of meetings and I can be late to a meeting by two to three minutes, but if Fireflies is late...

• 47:35Jacob Effron?

  It's late to a lot of meetings at the same time.

• 47:37Speaker 01

  You're gonna, you're gonna get a lot of support tickets. So we've had to focus a lot on that infrastructure. Actually, that's been the hardest part for us. It hasn't been the AI stuff, it hasn't been the UI stuff, it's been like, how do you build this like infrastructural giant that is going to like process, you know, 70...

• 47:57Speaker 01

  3% of Fortune 500's like conversational volume. And then you build the security measures and all of these other sort of things. All I would say is like, we were so naive in the beginning that we had like a monolith code base. And then we had to just break down each part, getting them better. And then like measuring the latency and performance of each part of the engine, like the recording and the transcription and the notes. And then how they get sent out. Like we've hit incredible like rate limits on just the number of emails that get sent out. So every piece of the puzzle, yeah, we keep like trying to optimize.

• 48:32Jacob Effron?

  Are there infrastructure challenges you've run into on like the AI model side? I mean, I'm curious, there's not that many companies that have the level of scale you have. And so as a VC, shamelessly have to ask, are there like infrastructure things that you're like, oh, man, I really wish this existed? Or, I mean, I definitely take your point that the biggest infrastructure challenges have just been on the non-AI side. But anything in the AI world?

• 48:52Speaker 01

  Yeah, I think like in the last three months, we've exceeded our rate limits like eight times. So that's just like, and I've had like folks at OpenAI like tell me, I don't think I've seen anyone that like need this much like rate limits or like TPM, like tokens per minute. Like your peak volume is insane. I'm like, yeah, like 10 AM PST. Think about how many people are having meetings, right? So, and then meetings have way more data than like if you're processing like a thousand emails. So there's just a scale at which we're doing a lot of these things have been challenging. So yeah, we've been constantly working and OpenAI has been great. Anthropic's been great.

• 49:31Speaker 01

  We're also looking at trying to work with different providers and be able to tell them, hey, like the volume here is insanely high. So just, I hope you can like help us scale with that.

• 49:44Jacob Effron?

  Right. And do you use a bunch of the open source models too?

• 49:46Speaker 01

  We've looked at Llama, we've used a different tool, we've tried Grok, we've Mistral. Like, so there's a lot of different solutions that we've been using. I think that what's been interesting is like every time we were about to make certain switches or decisions. This goes back to like throwing stuff out the window. Like, you're like, oh, it looks like a new model's coming out or it looks like a new thing is coming out. We just like did an announcement earlier about like now providing Anthropic to all of our customers as an option. And then, you know, Sonnet 3.5 comes out. And so like the field is moving so fast that you don't want to be like. Fixed on any one thing.

• 50:28Jacob Effron?

  Yeah.

• 50:28Speaker 01

  And I think that competition is actually good for the downstream startups because these guys are building incredible tools. And because they're competing, uh, the costs are going to go down incredibly fast. And then the intelligence is going to go up. So any AI startup founder, as long as you're just riding that wave, you're going to be okay.

• 50:50Jacob Effron?

  It's been a good way for you guys. I'm curious what you think of some of these, like, I feel like there's this analogous category of like hardware plus recording everything going on, like the Humains and Limitlesses of the world. Like, um, what do you think of that approach? And like, you know, what's the future of like those kinds of tools?

• 51:05Speaker 01

  Yeah. Uh, hardware is insanely difficult. So I've told myself, uh, yeah, we're not going to touch hardware anytime soon. We'd love to partner with different, uh, solutions. And we're actually talking to a few companies right now to partner and bring Fireflies onto those, um, services. I think it's really hard to get people to use something that's different from your phone. And so a lot of times if that convincing factor is not strong enough and the technology for its part could be great. But if it's slightly less than, wow, I could just do this with my phone. It's very hard to convince, uh, people. I also like to focus on things where we can get the max amount of distribution.

• 51:50Speaker 01

  So we intentionally like focused on video conferencing and meetings. And we've had this like maniacal focus on like B two B and teams because that's where knowledge gets shared around a lot more. So we, we did that. And then later we introduced like in-person recording through the phone.

• 52:07Jacob Effron?

  Yeah.

• 52:07Speaker 01

  We have integrations to like dialers, like Ring Central and Air Call. So we're trying to just pick the places where we're going to have the most data. I think the, there are some security challenges with having a hardware device that's always like recording. I don't know if you can build an entire business just saying you're going to do hardware. For Fireflies, we believe like we should capture like all of your, the knowledge from all the different sources. So whether it's in person, whether it's over the phone. Whether it's over video, but those are the things where we're willing to partner rather than do it ourselves.

• 52:42Jacob Effron?

  And do you think you eventually like sit on top of everything that's on my desktop, like to get full context on everything I'm doing?

• 52:48Speaker 01

  I think it's too early to say, but I do think we're building out those like integrations and building out those workflows. First, we want to deliver value and then can the chief of staff have more context to do more helpful things for you?

• 53:01Jacob Effron?

  Yeah, I mean, I feel like going after the chief of staff is like the heart of enterprise AI. Like, are there other enterprise AI spaces that you pay attention to that are like kind of adjacent? Or like, what else are you like, you know, you keeping tabs on?

• 53:13Speaker 01

  Yeah, so I'm very fascinated by some of these like design companies that are basically, you know, you give it a prompt and it's going to automatically like generate a UI or design for you. That's going to make the life of a lot of designers and iterating really fast. We talk about like coding, like Copilot and Devon. I also think the soras and the runways of the world, even though that's not enterprise directly, the implications for enterprise is really, really powerful. So you can take a lot of this knowledge and then like imagine being able to tell a really convincing story. You know, like a presentation for investor deck or like for a salesperson.

• 53:57Speaker 01

  So I think visual content, anyone that's working on that, this is a great time because we're very visual learners. Like if you look at your IG reels or even like LinkedIn introducing like reels, like everyone's gotten so used to those like 10, 20 second form factor. So like if you can have like this sort of stuff, so I'm really bullish on the visual content that's being worked on.

• 54:21Jacob Effron?

  That's awesome. Well, I feel like it's been a fascinating conversation. We always like to end with a quickfire round where we ask you a standard set of questions that probably should be their own standalone questions, but we lump them into the quickfire. So I guess to start, would love your thoughts on one thing that's overhyped and one thing that's underhyped in AI today.

• 54:36Speaker 01

  I think I've covered the fundraising has been overhyped. I would say that fine tuning and like overtly like focused on like cost reduction has been overhyped. Like if you're a business and you're in the you're in the business of like telling people, hey, we're going to help you like lower your LLM costs when you're competing with the big guys that are already doing that. So I think try not to over focus on what's happening like in the next five to 10 months. But look at like two years from now, will this still be relevant like what I'm doing?

• 55:13Jacob Effron?

  Yeah. And then what's been the biggest surprise in building AI features for Fireflies? And I think like maybe an example of something you were like convinced would work that didn't. And maybe conversely, like something that you were like, this is definitely going to work and it didn't work so well.

• 55:25Speaker 01

  Yeah, I think that when we introduced Ask Fred, like our assistant that you can ask questions about.

• 55:34Jacob Effron?

  How much debate was there about the name? I feel like when you name an assistant like Alexa, obviously, like, you know, a monumental change for people that were named Alexa. Like, how much debate was there around Fred?

• 55:43Speaker 01

  Well, the reason for using Fred was like, why only like stereotype and have only female assistants? Like, let's, let's have a male assistant to just kind of change things up and break the gender norms. And the other thing was about using the letter F like for Fireflies and Fred. So that's why we call our thing the Fireflies Feed. So we try to use like the F word as much as as possible. That's been really like the the thing for us. But like, yeah, when we introduced Ask Fred, like, we thought people were just going to be really smart about asking certain queries and like be very thoughtful.

• 56:22Speaker 01

  But then you have to realize like, a lot of our customers may not even know what an LLM stands for, right? Like, there's going to be a lot of folks that have used GPT-4, but they don't know like the LLMs and stuff that are under it or like how to talk to AI. So it's a new skill set. Over time, more people are going to be familiarized with it. So we realized like we had to do a lot more handholding, a lot more nudges and suggestions. So it took a lot of iteration before it became this breakout feature. So it took like six months of iteration for sure.

• 56:53Jacob Effron?

  Yeah. Yeah. And then has there been like a wild success in a product that you're like, I had no idea that was going to work so well?

• 56:58Speaker 01

  So we recently released tasks and so far the early data has been very promising. And the NPS reports and the feedback, people basically saying like, wow, this task management way for me to cross off tasks that were for meetings, assign it to other people. Fireflies will automatically assign tasks after something was talked about. It just brings a lot of structure to their life. And many of them say like, I don't even need to go back to Asana. Like, so that's been really effective. We actually, when we first started Fireflies, before we even did meetings, it was, we had this Chrome extension, this again, pre LLM era, that would follow me across email, LinkedIn messages, Slack messages.

• 57:42Speaker 01

  Anytime I like said, hey, I'll send this report to you next Friday, it would create a task. And so it'd create like a to do list for you. So now we basically brought back that concept.

• 57:50Jacob Effron?

  Comes full circle.

• 57:50Speaker 01

  Yeah.

• 57:53Jacob Effron?

  What's something you've like changed your mind on since building Fireflies?

• 57:56Speaker 01

  I've always felt that I probably could have done better with a few more years of corporate experience before doing enterprise SaaS. So I felt that in the early days. But going back, I think I would still do exactly this and do it this way. I would be okay with that first three to four years of struggle and uncertainty. It builds a lot of character. As long as you have enough runway, like you're living, like my, my co-founder was like living off Domino's pizza and soylent. So that was, he has a blog post on this, which to this day gets us a lot of traffic. Yeah, that like, as long as you can survive, you know, and get to raw and profitability,

• 58:37Speaker 01

  like it's fine to like learn from those, those experiences. So I think I value like grit and persistence way more than just like, okay, this person has a decade of experience doing a certain thing in a certain industry. Even when we hire, we look for more of that grit and like ability to learn. Yeah.

• 59:00Jacob Effron?

  I know you mentioned the design space already, but like maybe a different, like which AI startup outside of you guys are you kind of most excited about right now?

• 59:07Speaker 01

  Yeah, so I am a regular user of perplexity now, so I'm convinced. So I use it very regularly. I know they've raised a ton of capital and they're looking to raise more, more money. So it's going to be a very interesting battle. I think the team is great.

• 59:27Speaker 01

  You know, you're going up against Google. It's no joke. So I'm just curious to see how that will play out in the long run. But just on search, on the answers, the quality of the responses, like they've basically proved to all these other people that are taking on incumbents, that if you can just do the core thing really well, you know, it will give you a leg up. So I really do appreciate like the work that they're doing.

• 59:54Jacob Effron?

  Totally. I think to your point earlier, it's like everybody wants this like clean answer of like, why are you going to win in this space? Or, you know, every investor is always looking for that. And it turns out for a lot of these horizontal SaaS, you know, application companies, it's never like one clear thing of like, well, we are, you know, better at building X models. To your point, doesn't even make sense to do. It's much more of like the thousand little things that just make the product being so deep in a workflow and also just make the product such a delight to use.

• 1:00:17Speaker 01

  Yeah. I mean, you look at hardware and look at where Nvidia is today. But when it started, I'm sure it had a lot of dependencies on Intel. Right? So like, that is like the biggest thing. Because if Intel decides to say, hey, we're not gonna, we're gonna do our own little graphics card version of what you're doing. Like, how do you compete in that market? So and then you have the giant that is Nvidia today. So, yeah.

• 1:00:42Jacob Effron?

  Largest company in the world. Well, this has been a fascinating conversation. I'm sure folks will want to pull on all sorts of threads. I can't imagine anyone hasn't tried Fireflies yet. But if they haven't, they should. But I'll leave the last word to you. Like, what, where can folks go to learn more about you and what you guys are building?

• 1:00:56Speaker 01

  Yeah, fireflies.ai to get started and use Fireflies. You can follow us more on LinkedIn. It's Chris Ramineni. Typically, we've been very low key in terms of media and PR and all of that stuff. But yeah, stay tuned for a lot of exciting product updates that are coming out this year.

• 1:01:16Jacob Effron?

  Amazing. Well, Chris, thanks so much. This was fascinating.

• 1:01:18Speaker 01

  Thank you so much.

• 1:01:19Jacob Effron?

  That was awesome. I think Fireflies is such an interesting company. And I feel like one that we all know because it joins many Zooms that we're all on. But because, you know, to Chris's point, they've been profitable, they've been really capital efficient, they haven't had the same like splashy, you know, we're raised $200 million headline. Other AI companies have had, but it's, I think, you know, probably as that ramp survey would say, one of the, you know, top five used AI apps in the enterprise today. And so it really just totally all of it.

• 1:01:48Speaker 02

  Yeah, I thought he was super interesting. The company story was super interesting and the founder's story is super interesting. Like, one of the things I wrote down is that he was, he said he was 21 years old when he founded the company. And like LLMs weren't really a thing, or at least not nearly to the scale that they are today. And so you had like five years of like practice running the company.

• 1:02:06Jacob Effron?

  I was reminded of so many things. I mean, there's a common refrain, I feel like in the startup world sometimes that like a lot of it's just about surviving. And it's like, the fact that they survived as a company, probably learned a lot, built really good product, to the point where these transcription and summarization models got really good. There was a whole wave of companies that started around 2015, 2016, that just didn't make it to the time when GPT-3 quality models were out there. And so it's incredibly impressive, you know, he's obviously very humble about it, but to have navigated the company that efficiently and to have built really good product when they're, when these models weren't as good. It feels like, oh, it's so lucky you're in this perfect position. But obviously you make your own luck by, by having kind of been able to be in that space.

• 1:02:45Speaker 02

  I mean, you guys talked about his, his big hot take at some point, but I wanted to unpack it more during the debrief, which is like him saying that he doesn't believe in fine tuning. Were you surprised by that?

• 1:02:56Jacob Effron?

  It's interesting. I mean, from the outside, you always wonder like, okay, this is such a general purpose tool. Like, how do you make it, you know, the needs that the finance team's going to have versus the sales team? Or a meeting that's about like hiring someone versus a meeting about a sales call. Like, you'd think intuitively, like, okay, those, those meetings, you'll need different outputs from them. Maybe a salesperson cares about different things than a finance person. And there's different ways to accomplish that. And so I guess before the interview, I was wondering, maybe, maybe they fine tune their model for, you know, Slack has different needs than like an HVAC company, or I don't know, maybe there's some way to tailor the model to an enterprise specifically. It feels like they do a lot of that via the prompts themselves. But I thought what he said, which was, which was really interesting, was these models get better so fast.

• 1:03:39Jacob Effron?

  Like, what a kind of waste to fine tune a last generation one. You'll probably get your, your GPT-5 not fine tuned. Might just be better than your GPT-4 fine-tuned. And so why spend all the effort and time to to fine-tune? Which I think makes total sense.

• 1:03:54Speaker 02

  I think another interesting theme throughout the conversation was also just about him talking about sort of like what not to focus on if you're an AI founder today and all like the deep stuff that's getting better. You know, the big players are taking care of that, so don't spend your time there. But then also just going super, super deep into a customer workflow. And when I saw this came come on the radar, or even when you think about a company like Firefly's and most new companies that are pitched today, there's probably an incumbent that you can look at and be like, you know, why wouldn't name your huge company go build this? And this feels very true for this one, like Microsoft, Google, Zoom, whatever. Could all go in theory, build some like meeting notes capabilities.

• 1:04:36Speaker 02

  So I'm curious, like from an investor perspective, when you're looking at a new company, what is it that you see that makes you like think even though there's incumbents are in theory so well positioned to win? How does a fire, a company like Firefly still like do anything?

• 1:04:53Jacob Effron?

  And this is the question right now, and I think, you know, it's hard because you traditionally you'd want to hang your hat on like, well, there's just some like technical capability that like this team has that like good luck, you know, Zoom trying to replicate. Like this is the best team at building, you know, low latency summarization models or something. But to I think to Chris's point, like that's actually a misdirection. Like these, I think over the last year, it's become clear that, you know, you benefit so much from these general purpose models. Like, why would you go build your own model for it? It's just not going to be a huge underlying source of advantage. And so I think what becomes difficult about it from an investing perspective is a lot of times you're what you're betting on. And what really makes a startup work is it's the thousand little things that make a product better. And any one of them sounds like ridiculous to be like, I'm betting the whole house on like the fact that this like

• 1:05:36Jacob Effron?

  individual workflow is so much better, but together they create this kind of experience that that is really magical. And I feel like perplexity has become this example that a lot of people point to where it's like, Google has the best AI people in the world. Like there's like, there's no like reason that if you said, here's what perplexity does on first principles, could Google do this? And it's like, Presumably, yes, they should be able to do it. But there's so many decisions and, you know, I think what startups have going for them is there's those thousand little things, there's really deeply understanding, you know, whatever a specific workflow is. I think to Chris's point, maybe there's things they can focus on early on that for Microsoft and everything has to be billions and billions of dollars, like it's not going to make sense to go super deep on like a finance workflow. And then the other thing I think is just the velocity, right? I mean, that ultimately

• 1:06:20Jacob Effron?

  is one of, is the biggest advantage startups have. The space changes so fast. It's, you know, he was, he mentioned in passing that they already support like the latest Claude model, like that came out like three days ago. And so like there, there is just an incredible speed. If, if things are changing so fast, that ends up being such a huge advantage.

• 1:06:35Speaker 02

  Yeah, he touched on the incentive structure too. That's where my brain goes is like I've, you know, having operated at like huge companies like Google and seeing how decisions get made there, like that to me is also a big.

• 1:06:47Jacob Effron?

  But they just have much more, you know, I think with Google and perplexity, there was this famous example of, of the glue pizza, right? Where I think someone was trying to figure out how to get cheese to stay on a pizza. And they were like, you should glue it. And it showed up in a Google search because it was like from a Reddit thread and, you know, became this whole like hoopla. And the reality is like, if that happened on perplexity, it's almost like, who cares? Like, but because it's Google, everyone is going to try to like hack it to death. And so, you know, Fireflies may have much more allowances to, to not be perfect. Whereas, you know, when Microsoft ships something, it has to set up, it has to live up to this like, you know, 100% standard. That was a really fun one, Rashad. I feel like there's only so many companies that are really operating at just massive scale today. And it's funny that it's been a theme now on, on multiple podcasts

• 1:07:31Jacob Effron?

  we've had that when you have that kind of scale, it's just, it's such an advantage in terms of, you know, we talk all the time about how do people evaluate models? And I love, people with scale are like, we just throw them at our users. Like they tell us what's good and what's not. And I think that, uh, there'll be so many more AI companies in the next year or two that are at that scale. And it's amazing to learn from this first batch that's really gotten there.

• 1:07:48Speaker 02

  I'm excited to have to do less work if we start using Fireflies and it starts making tasks for me and, you know, filling up our CRMs or whatever.

• 1:07:56Jacob Effron?

  Yeah. Sometimes with our interactions, I assume there was an AI model behind the scenes, but hopefully for your sake, the models get a little better.

• 1:08:02Speaker 02

  100%.

• 1:08:04Jacob Effron?

  That was a really fun conversation with Krish. Definitely make sure to join us next time. We'll have another AI founder operating at really large scale. We'll look forward to seeing folks then.

• 1:08:33Jacob Effron?

  Chris.