40 Trillion Tokens a Day (Yes, More Than OpenAI) | Lin Qiao, CEO of Fireworks

Should American companies be worried about Chinese open-source AI models? Lin Qiao https://www.linkedin.com/in/lin-qiao-22248b4/, CEO of Fireworks https://www.linkedin.com/company/fireworks-ai/, doesn't think so. In this episode, she joins Lukas Biewald https://www.linkedin.com/in/lbiewald/ to talk about why she believes the industry is at a turning point, one that calls for more open intelligence, not less. They cover how Fireworks now processes more tokens a day than OpenAI's API, why she thinks the future belongs to specialized models built on private company data rather than general-purpose ones, and why she believes OpenAI and Anthropic should be open-sourcing their own models too. Connect with us here: Lin Qiao https://www.linkedin.com/in/lin-qiao-22248b4/Fireworks https://www.linkedin.com/company/fireworks-ai/Lukas Biewald https://www.linkedin.com/in/lbiewald/Weights and Biases https://www.linkedin.com/company/wandb/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Lukas Biewald?

  Do you think American companies should be concerned about using Chinese models?

• 0:03Speaker 01

  I do believe we are in these crucifying moments that across the industry, we should encourage more open intelligence. We truly believe in open development, open intelligence.

• 0:13Lukas Biewald?

  I think of Fireworks as a company that does a really good job running open source models.

• 0:18Speaker 01

  We process more than 40 trillion tokens a day. It's bigger than OpenAI's API and Gemini's API. We do not believe the world will be dominated by a few models from frontier labs. If you think about before AI, data is one of the biggest innovation. There's so many open source project about data. And I believe that's kind of the fundamental reason the data field is able to move so fast across the whole entire industry. To me, it kind of doesn't make sense. Intelligence is in the hands of a dole.

• 0:46Lukas Biewald?

  Do you think OpenAI should just open source? And Anthropic should open source their models?

• 0:49Speaker 01

  I would really think.

• 0:53Lukas Biewald?

  You're listening to Gradient Dissent, a show about making machine learning work in the real world. And I'm your host, Lucas Biwol. All right, I'm here talking with Lin Chow, an old friend of mine. I've always admired her as an entrepreneur. I've always wanted to get her on the podcast. I finally got it today. I get to ask her questions around the technical aspects of how her product works, what motivates her as a founder and how she got started in the space. I hope you enjoy this episode.

• 1:20Lukas Biewald?

  So thanks for joining us. I've been wanting to actually have you on this podcast for years, back to when you were a big part of the PyTorch team and working on that. And we never quite made it happen. So one thing I was really surprised by researching you is you have a college-age daughter, which made me realize that you're older than I thought, I think.

• 1:40Speaker 01

  Yes.

• 1:41Lukas Biewald?

  So you kind of came to being a founder a little later than a lot of people, or at least the Silicon Valley ideal. Did you always want to be a founder and now you're doing your dream? Or how did that happen?

• 1:56Speaker 01

  I talk with people who are college dropout who are founders. So I'm more than their age, twice of their age. So first of all, I think there's no good timing. Only you know when's a good timing. And that's just a gut feeling.

• 2:16Speaker 01

  I think to me, about 10 years ago, it started to come to me. I want to be a founder.

• 2:26Speaker 01

  I want to fund a technology company. And I actually started doing that. I started to kind of contemplate because 2015 is the year I've done everything across data, starting from being a researcher, build the first generation in memory, super fast. And then warehouse to build many data product from offline to online data tools. And I think, hey, I know everything in that space. And there are many companies during that time, so I want to start one. But I don't feel I'm ready from people's point of view.

• 3:09Speaker 01

  Because I know kind of building a company is all about people at the end. So all about kind of organizing experts across different fronts to be able to build it up. I don't think I have that skill. And that's part of the reason I, that's the main reason I joined Facebook.

• 3:28Lukas Biewald?

  Oh, really?

• 3:29Speaker 01

  To kind of learn what is kind of, at that time, they're, it's post-IPO, but they are kind of one of the fastest-growing star in Silicon Valley.

• 3:41Speaker 01

  My main motivation is to learn what this culture look like? Why this company is so unique? Why, what is so unique about their people? And I'm so curious to kind of figure it out.

• 3:53Lukas Biewald?

  What did you take from that?

• 3:55Speaker 01

  So believe it or not, when I moved to Facebook from LinkedIn, right? LinkedIn is another social network, a professional social network. And Facebook is kind of commercial, like a consumer social network.

• 4:06Speaker 01

  At the beginning, I know this company is outstanding. But on the other hand, I'm like, hmm, you know, I probably have seen law. So I, you know, it probably will not be that different, but I need to learn what I need to learn. And I was shocked when I joined the company. What is, what is really surprised me is what I carry on to Fireworks is the extreme sense of ownership. Because in the other company, I don't feel that. I feel like the other company does always, hey, you're an employee. Here's your box, you do your job, you'll be evaluated. You know, that's kind of how things work.

• 4:42Speaker 01

  And when I joined Facebook first, it's just everyone cares so deeply about company. They do whatever it takes.

• 4:50Speaker 01

  You can change any code. It doesn't matter whether you write, it's your code or not. If you find this a bug, fix it. No problem is other people's problem. From small things, hey, the dashboard in the reception where every visitor's seat is broken, go report it, fix it. So it almost feel like this is my family. This is my company. Although I have tiny ownership of the company, it doesn't matter. But I do feel that way. Everyone feels that way. And it's magical. It's magical, like just bring like extreme, like everything best out of person to build for the company.

• 5:31Speaker 01

  So that's, I think that's the secret sauce. That's the kind of the magic power. And at Fireworks, we strongly reward extreme ownership. And those people carry the sense of extreme ownership. They go, they rise up. They rise up without you asking them to do anything. And they just kind of by themselves figure things out.

• 5:54Lukas Biewald?

  So what was your core motivation for starting a company? Like you've talked about how it's okay. It seems like you're okay with the competition, being successful, like it's maybe not to beat the competition. What made you want to start Fireworks?

• 6:11Speaker 01

  I think it's the same with my other co-founders. We want to, we are very gravitate towards impact.

• 6:22Speaker 01

  We're almost like impact machine.

• 6:25Speaker 01

  And before we start, it's kind of a little bit rooted in our journey in PyTorch. Obviously, PyTorch has a great, a lot of great people moving forward, including Sumit, Joe Spisak. They're all kind of great people I worked with. And because of PyTorch, we work a lot with open source community, with other companies. It become very clear to us, the whole entire industry is moving from mobile first, law data being generated, AI first. And we actually moved back five years. We hit the same problem at Meta. Mobile first, law data generated, AI first.

• 7:11Speaker 01

  And at that time, there was no AI hardware. Everything is CPU-based, tiny machine learning algorithms, linear regression. There was no AI software. Very rudimentary, deep learning just got started. There was no AI team. And at that time, around 2017, we all joined during similar range of time, build AI infrastructure from ground up and build PyTorch in Google's community. And then five years later, the industry is hitting the same problem. And that makes it clear, there's a whole entire industry movement towards AI. And they hit the same problem as we started this journey at Facebook. And we know how to help them.

• 7:57Speaker 01

  And as a matter of fact, they keep coming to PyTorch team saying, can you build the training platform for us? Can you build a serving platform for us? Can you tell us how to even build this AI team? And it just kind of, there's a lot of desire, you know, for clear demand for industry, to have industry-wide impact. That's the reason we want to start a company is to kind of help much bigger movement. With regard to competition, I don't view those as competition. I feel like if there are, there are a community build great open source project,

• 8:34Speaker 01

  it's not our game to compete with them. I want to see them doing well. And again, goes back, every single company should have a reason for them to exist. Then our unique value is not in that space. Our unique values to kind of in other layers that we add, where's kind of we find our, our kind of position and anchor on specialized intelligence as we believe that's kind of the biggest area we can move the industry and deliver impact. So yeah, so that's kind of how we think about it.

• 9:06Lukas Biewald?

  I think I saw recently you raised at like a $15 billion valuation. Is that, is that right?

• 9:11Speaker 01

  Yeah, we just finished raising Series D, 1.5 billion cash and 17.5 billion valuation.

• 9:23Lukas Biewald?

  17.5, congratulations. Do you want to brag a little bit about your metrics or your success at Fireworks?

• 9:29Speaker 01

  So if anything like I haven't done very well is to talk about who we are, what we do. We're being kind of really, just want to be the engineer building product. But yes, I would love to talk about Fireworks. So we build specialized intelligence platform. So specialized intelligence is a parallel strategy to general intelligence. And this is a lot of discussion actually happening across the industry. And we're very happy to kind of participate in that debate, but also kind of really charge forward to build the best tool in the specialized intelligence category. So the idea here is there are two strategy to intelligence, right?

• 10:12Speaker 01

  So one is build the intelligence that can solve all problems as a black box API. This is a typical frontier labs, building AGI, very familiar. Everyone's kind of understand that direction. But we believe intelligence is the derivative of data.

• 10:33Speaker 01

  That's our fundamental assumption. Where, but if we look at the world's data, you actually live in the data world, right? So kind of all the, when with in BISI started, you see a lot of data about, about training first and then flowing to the rest of AI. So there are, if we look at the world's data distribution, only a small fraction of data is public internet. That is a primary source plus the label data. Those are the primary source for, for frontier labs to train from scratch. Majority of the data is actually locked inside application, locked inside enterprise.

• 11:14Speaker 01

  And in my opinion, this data should never be shared with anyone else because those are the alpha of those company. There's a much deeper, deeper thinking behind that. And we believe the new frontier of intelligence should be specialized intelligence, as in turn those private data into, into a customized model that is uniquely owned by the company designed their product in a unique way. And also this process shouldn't be one time. It, it should be continuous because we all know application keeps evolving. Product engineer makes application constantly better. Base model keeps improve, improving before like we can say every month, there's a new base model popping up.

• 12:00Speaker 01

  And now every few days, there's a new model. It's just a velocity insane popping up in top leaderboard in a, in, in a really good like forward looking way. So then your, your customization, specialization process should be continuous as in you should be doing that every, every week or possibly every day or every few hours. We do have customer have different cadence and, and then we take a big step back.

• 12:29Speaker 01

  We believe the specialized intelligence in general, like intelligence, it, these two strategy will coexist in the future. They will have their own usage and so on. But we do not believe the world will be dominated by a few models from frontier labs. The world will not be a duopoly. The world will be millions of specialized model, one per application per use case. That's how, that's what kind of we're, we're, we're seeing the worldview and, and we're building our platform to, towards like, hey, giving the control, giving the control of

• 13:09Speaker 01

  IP and giving control of cost to every single company because those company exist for a reason.

• 13:16Lukas Biewald?

  Mm hmm. That's interesting. I'm, I think of fireworks as a company that does a really good job running open source models, but specialized intelligence is a little bit of a different way of looking at it. Are most of your customers actually modifying the open source models before they run them?

• 13:32Speaker 01

  That's a really good question. So today we process more than forty trillion tokens a day. So this is actually fun fact, it's, it's bigger, based on what we know, it's bigger than OpenAI's API and Gemini's API. So, but more interestingly, ninety-five percent of that traffic is from customized model and customized inference deployment. It's not off the shelf.

• 14:03Speaker 01

  So in from what we see with our customer engagement and the demand coming to us, we're just at the beginning of this S curve.

• 14:13Lukas Biewald?

  Mm hmm. So you're claiming that you, you think that you ingest more tokens than OpenAI's API?

• 14:22Speaker 01

  We process, so by process, it means really the prompt and generation combined.

• 14:26Lukas Biewald?

  I see. So it's more input tokens and output tokens in total than OpenAI's API or Gemini's API?

• 14:32Speaker 01

  Yeah. So actually, we don't know how each company accounting for this number. I mean, they may have their own philosophy, but just by numbers, that's what I'm seeing.

• 14:42Lukas Biewald?

  Wow. And what are the biggest use cases?

• 14:47Speaker 01

  So that's also evolving. Last year, it's, it's all coding and we have all the coding company built on top of us. And because of coding, it actually unlock the next wave. Because of the coding advancement, I think one thing has changed significantly in the past one year is software development and application development. The velocity is insane. And before it would take tens of very strong product engineers and PMs from ideation to implementation to production scale over multiple quarters. Even multiple quarter may be fast. And now one person, a few weeks, knowing nothing about the writing code can do that.

• 15:33Speaker 01

  And because of the velocity, then we start to see, I think it's a dependency here, a sequence here. We start to see a vibrant usage, especially in the co-work space. And we have, we have many application, they are solving general purpose co-work problems. For example, professional deep research or slide generation or kind of the tools we use day to day general. And then we are also seeing development of a wide diversified vertical specific co-works across legal, finance, recruiting, marketing, sales, customer support, even within each bucket, there are like fine granular product being built. So, so that's what we see this year.

• 16:17Speaker 01

  And in addition, we're seeing the consumer facing market start to think about how to use, how to use logical reasoning capability for LMS to change how we do search, how we do recommendation, anything. So could be possibly next year is, is the, is the consumer facing unblockier for GNI. So, so yeah, so that's where we see the trend is. And but nonetheless, one thing that is clear is just the variety of creativity, innovation built on top of GNI, bringing from experimentation to production is, is exploding right now.

• 17:01Lukas Biewald?

  Hmm.

• 17:03Lukas Biewald?

  And I mean, OpenAI had a fine tuning API in, in various iterations, but I don't think it was very popular. Why do you think RL and fine tuning is more popular on open source models?

• 17:15Speaker 01

  I think fundamentally, the general intelligence company, if you think about the unit of economics, right? It doesn't align to support a fine tuning product. The fundamental reason is training from scratch is very expensive. It's massive R and D investment. And, and the end result of this model would, like companies would want to create monetization as, and scale as fast as possible. The way to scale as possible, possible is just package that in as API. And, and the underlying infrastructure will be most efficient if it's just a few models, right?

• 17:58Speaker 01

  If it's support millions of model, it's completely different unit of economics and that's massive, it almost like it's a completely different business. I think it kind of makes sense for, for the frontier labs to focus on scaling the model as is and go like really fast and wide. And creating a separate business is, is a strategic change and pivot. So I think that's a fundamental reason. It's not like having a fine tuning service that's on high market is kind of focus of a company is, it doesn't align.

• 18:32Lukas Biewald?

  Mm hmm. I've been a little surprised, honestly, over the last few years that fine tuning with RL hasn't been more popular. It does seem like it can be fiddly to get it right and you kind of need the data in a good format or you need to create a simulation environment, which can be an expensive process. Do you help your customers with that?

• 18:50Speaker 01

  We have different level of engagement. And the interesting thing is our customer has wide diversity right now in terms of a product is only built for this, the needs of the customer, right? Right on the needs is, here's the spectrum. We have customer, they are very deep. They're, they have researchers, they have researchers from frontier labs and they want to control every single knob. For example, we work with Cursor, they build their composer models on our, on our training stack, especially focus on RL.

• 19:29Speaker 01

  And where they, they want to control every possible parameter to tune and we help them like connect with their trainer and we manage our rollout inference. So, so that's one level of engagement is very extremely low level, give you all the controls as you want. There is a, a different tier where I think it's bigger in terms of number of developers who can use this. They are, they have AI experience, but they haven't been the expert in training a model. And they are learning that specialty and, and they want to control something, but not

• 20:14Speaker 01

  everything because it's overwhelming. So then we build the next level API. We have a fireworks training SDK that geared towards, you know, they, they can plug in, they can plug in a loss function, they can tweak which, which algorithm they want to use and a few other parameters. And then they can kind of start to do their experiments.

• 20:39Lukas Biewald?

  Can you talk a little bit about how that would work for like a real world use case where it would work super well?

• 20:44Speaker 01

  So we have been working with cross startups, digital natives and even to my surprise, even enterprises. So they, the training SDK is actually self-serve. So we just, they can just code against the SDK and that SDK is going to talk with our backend API. We have two modes. We have, we have surplus where they don't even know, you know, how many GPUs is required and the kickoff the training job. And then we have telemetry for them to see how things are going.

• 21:24Lukas Biewald?

  I mean, could you give like a specific example?

• 21:26Speaker 01

  Of a use case or?

• 21:28Lukas Biewald?

  A use case, yeah, like a specific customer even maybe if you can talk about it?

• 21:31Speaker 01

  Yeah. So for example, I think we have in, in the healthcare space, we have Docimidy. They are do, they are building, they are basically building deep research for doctors. And they're, they're tuning their model, you know, in medical term, the, it's almost like different language. Um, and that is not, uh, so usually the, and the, the tuning is a, a, a, a pattern is it's a special DSL. Um, whether it's medical or whether it's, um, it's some kind of special programming language.

• 22:11Speaker 01

  Uh, for spreadsheet, um, uh, manipulation for data processing. Um, so, so then, uh, that knowledge is, that logical reasoning is being baked into the base model. But that language is they're not familiar with.

• 22:26Lukas Biewald?

  But let's talk, so what is Doc somebody showing your SDK to fine tune the model?

• 22:31Speaker 01

  Uh, showing meaning?

• 22:32Lukas Biewald?

  Like what, what, how is it teaching the model what it wants?

• 22:35Speaker 01

  Oh, so they have actually a team of researchers. Um, they are using a combination of, uh, so we have SFT, DPO, KTO, uh, RL. We have different flavors of RL. Uh, it's actually up to them to pick and choose and oftentimes to use, uh, a combined algorithm. Um, and, uh, so, you know, usually you, so it's very similar to kind of the, uh, frontier labs training process without pre-training. Right? So usually SFT is like mid-training to infuse the knowledge and you are.

• 23:10Lukas Biewald?

  Maybe like taking a step back, you know, again, for people that maybe aren't quite as familiar with all these, uh, acronyms. Like, you know, so reinforcement learning is another way now in this context to fine tune a model. Like in the past, you know, you would show, um, fine tuning would mean you need to show the model exactly. You know what you want each time and modify. How, how is RL different?

• 23:30Speaker 01

  Yes, R is very different in the sense that, um, so, uh, like just compare SFT with RL. Uh, uh, SFT is basically you tell the ground truth to the model and let the model pick up the ground truth. Uh, so, um, so RL, so it basically is like, hey, here's a textbook, you memorize all the textbook, right? So that's kind of SFT. Um, RL is, um, you try, you try different variation of the model. Uh, and then let the model interact with your product or your simulation. And then get the result back and then you, you rate that result saying it's good or it's actually not good.

• 24:12Speaker 01

  Or there's a range of between zero to one, some kind of, uh, reward, it's called reward. Uh, and then based on reward, the model will know, oh, this direction, this exploration is not that good. So I'm gonna backtrack, I'm gonna do some other exploration until, uh, the reward shows up, it's pretty good. Then, okay, that probably is a good result.

• 24:29Lukas Biewald?

  But that rating is tricky, right? Like, how does Doc Simony do the, the rating of the quality of the results?

• 24:34Speaker 01

  Uh, so that's where, uh, a lot of deep product experience. So, um, this is a very interesting that we see a new emerging. Um, I wouldn't say it's a job profile or it's just kind of the, the type of people doing in the past. Um, pre-GNI, we have product engineers, right? Focus on product, we have researchers focus on building model and they talk with each other to figure it out. And, and now we see, uh, product, um, people who to model has product knowledge. And, uh, and they know how to tweak things. And they either, uh, researchers learn to like build product or product people learn how to, uh, to models.

• 25:18Speaker 01

  Because the judgment, there are a lot of judgment in, hey, what should be the reward look like? Um, and, uh, uh, would this be a, um, a good, um, search result? For a doctor searching this medicine is, is a judgment, it's a part of judgment. It's almost requires domain specific knowledge.

• 25:38Lukas Biewald?

  So are you saying they're using people to actually look at the results and say if they're good or not? Or they're using people to build algorithms to automatically decide if the results are good or not?

• 25:46Speaker 01

  I think there's always a, um, so before anyone to anything, um, the company to build their own eval, right? Is, uh, is the same as if you write software, you need to write unit tests and integration tests to judge, uh, how good this software. It all start from there. Um, and then, uh, once you have that, and that's what one choose to hill climb. Um, and, uh, with the eval, then, and then you start to understand, oh, now I'm gonna, uh, I'm gonna write. How the reward look like, which is different from eval. Um, and this is basically like, it's exactly how, similar to how we human grow.

• 26:29Speaker 01

  And as we born, we come with our, um, IQ is stay the same. So this is a base model, right? Uh, and, and then we learn by going to school, by memorize that's SFT, uh, supervised, uh, fine tuning. And we learn by trying things because deep down, I think, human like homo, homo sapiens, okay? Is defined by our curiosity, um, desire to explore and try new things. And we try new things, we get positive feedback, we'll try more. Uh, we get negative feedback, we'll try less. So this is exactly how, uh, model learn. And the feedbacks come from, uh, from the product.

• 27:10Speaker 01

  Because product could, if you do, if you get the feedback from product directly, then you have signal from product. Um, or from simulation where the simulation will also provide feedback. Uh, so, so then the, the product engineer, product slash researcher, um, that person is the codify. Uh, what that product feedback should be based on, uh, the gen, uh, the, the text generated, uh, and then close the loop.

• 27:36Lukas Biewald?

  I mean, I guess in the past, and you know, as a big part of this at, at CrowdFlower, you'd label lots of examples of exactly what you wanted. And that was expensive, but at least it was very clear, you know, what you're doing with SFT or supervised, um, fine tuning. But now where you actually don't know exactly what you wanted to do. You, you're just looking at results and trying to say if they're, they're good or bad. I feel like there's a much more complicated problem here, right? Like, I mean, famously, the labs pay tons of people to go in and, and just, you know, um, RLHF, right? Like have humans just grade over and over. But I think you're talking about using product signals. I think we also see RLAIF, right? Where the AI looks at the, um, the results themselves.

• 28:21Lukas Biewald?

  I mean, you're seeing all these companies that are building their own evaluation functions or, or reward functions. What, like, what are the trends here? Like, what, what's the best practice? And, and can this really scale to all the different applications that there are out there?

• 28:35Speaker 01

  Yeah, so our, our, um, our thinking is it can. Um, so there are multiple fundamental reasons, uh, it can and it should. Uh, one is, um, so this, this kind of articulation came from, uh, my conversation with Jensen after his GTC keynote. We, we're, we're shooting a video together. Uh, and shooting a video with Jensen is kind of very casual. It's kind of, he just start talking and then we just talk, uh, there, and then done. So, but he casually mentioned one thing, we talk about specialized intelligence. He casually said one sentence, um, there's no specialized general company.

• 29:14Speaker 01

  And, uh, it, it's logical, but actually when I reflect back, it's very profound.

• 29:21Lukas Biewald?

  It's, uh, there's no specialized general company?

• 29:24Speaker 01

  There's no specialized general company.

• 29:26Lukas Biewald?

  What does that mean?

• 29:26Speaker 01

  So that means every company is special. They are solving a special problem using a, a special solution, um, with a special purpose. And that's why we have millions of company or maybe tens of million company in the world. Um, and we, we deliver, we carry a unique design, um, of the solution space for, for the problem we care about. Um, and that's why companies exist. And because of that, the knowledge or the choice or the taste or the judgment to create these companies.

• 30:05Speaker 01

  Are not unified, are not common, are not even commonly shared. Hey, this is standard. Um, and because of that, it's, it's really hard to be captured by a general purpose model. Um, and because of that, we believe every company should own their intelligence because they are the expert. Carrying a taste, that judgment, uh, that unique thinking. And that should be codified into, uh, into the intelligence they own. And have that intelligence for the power of their product, make their product even better and the start to create this flying wheel.

• 30:44Lukas Biewald?

  But I guess there's, there's multiple ways to do it, right? I mean, one, one way to do it is to actually modify the weights of the model itself. I think that sounds powerful, but it, it also seems potentially difficult and complicated, right? There's also, you know, the context window where you could insert something or just using the model in different ways. Like, it's not the, the only, it's not the only way to modify the, the weights. Like, why do you feel like modifying the weights is gonna be the way that companies inject their knowledge into? Or inject their special sauce into these models?

• 31:14Speaker 01

  Yeah, there are actually many different way to define, uh, to create frontier. Um, prompt engineering, context engineering is one way and many company has already been doing that. And only they know how to kind of the best construct the context and so on. Um, and, uh, activate their private data to constantly tune their model and not giving alpha out is another way. Um, and having a, uh, dissemination of routing logic to, um, route. Um, to route the task towards the best model in terms of quality and cost. Um, and have this, uh, network of, um, underlying model supply is another way.

• 31:58Speaker 01

  So there are many different ways to, uh, to push frontier, uh, in a setup. I think they should use all. Uh, I don't think tuning the model, uh, on, uh, no, having the weights is only way. But I will say, uh, I would say having the weights is essential for other reasons. Uh, the reason is it goes back to. Application software development has been disrupted. So the mode before be able to implement and push kind of production scale of an idea is a mode because it's hard. It requires, there's a deep barrier to to get it right. Uh now that barrier becomes very thin. And then what is a mode for application company?

• 32:44Speaker 01

  And the mode is um is something that cannot be copied replicated. And the data collected from your product about the customer intent, customer preferences, why they engage, why not engage? And business logic, those are proprietary. And it will be, um you just leave, uh your, this is your offer and you leave your offer on the table. If this is not integrated into the model you use to power your product. Um, so I will argue this probably is by and large the biggest mode every company should create. Uh to turn your data to, and your data is reflect, a reflection of your proprietary taste and judgment.

• 33:26Speaker 01

  Um, and the reason why you exist as a company, turn that into uh into your own model, which is your own intelligence. And you do not let that proprietary knowledge leave your premise. Um, so, so that's kind of the bigger macro. The second big macro is uh is unit economics. Uh, so that's very interesting. Uh, we all started working on AI pre GNI. Uh, that's to SaaS time, right? And during SaaS time, when um the uh the product market fit and the durable business, they are almost the same thing. It's really, really hard to find a product market fit, but if you find it to scale, scale as fast as you can run.

• 34:08Speaker 01

  Because the the costs of running a business are mostly dominated by people. Um, infrastructure costs, which is mostly CPU and storage, those are commodity. Those usually people don't care about, like those, those were not kind of cost problems. Um, but now in the GNI time, product market fit and a durable business are two separate concept. Um, hit a product market fit doesn't mean you will have automatically have a durable business because operate AI infrastructure is or intelligence is is expensive. Um, and then the unit economics completely changed. We have so many startups, they, they have great product, their customer love them.

• 34:49Speaker 01

  Uh, and also want to pay them, but they can, they just cannot scale the business. Uh, because they are scaling to bankruptcy. Uh, and scaling to bankruptcy is a norm now. Uh, really new kind of um think about, you know, how to, how to even build the cash flow and have a positive gross margin and then it's kind of durable. Um, it's even worse for large company, public companies. Um, the digital media, for example, they were startups um a decade ago and they're the winner, right? They win their market, they win some consumer markets, some personal markets, some developer market. Uh, and the the consequence of winning is they have huge amount of traffic.

• 35:33Speaker 01

  They already have a huge customer base. Um, and for them to roll out a AI feature, cover everyone. Uh, the cost is enormous. Um, and and then they need to answer uh to Wall Street, have their quarterly earning report and explain, hey, why, why, why you suddenly kind of um spend so much on a feature that ROI is to ETPD. Uh, so, so cost control is a really, really big concern. On top of that, obviously, because of the popularity of coding agent, almost every company has a coding harness. Um, and and they are all worried about, hey, the cost of um how we justify spend so much in in coding and

• 36:17Speaker 01

  and and now the industry is shifting from token maxing, I think, beginning of this year into value maxing. We seriously need to talk about ROI, our part of ROI. Um, so, um, so cost control is another big concern. Uh, that's where really um picking the right model for you to activate private data and private knowledge and make the model quality on par or even better than uh than the black box API. Um, and then bring the cost down by five to 10x extremely for to build a durable business. So that's another reason.

• 36:55Lukas Biewald?

  Mm hmm. How much cheaper is it to to run your own model versus, I mean, like, the the frontier labs are also constantly dropping their their token prices at the same time if you use more and more of it, like, what's the the delta?

• 37:07Speaker 01

  Yeah, so um here, when we talk about pricing, it's actually not per token pricing. Uh, because the um different model, they, their velocity is different. Um, open model tend to be a little bit more verbose. So even though, like, if you look at the pricing rise, everything's public, usually they are 10 times cheaper. Uh, but usually that 1.5 to 2X of mobile boats. So then the cost saving is around the 5 to 6X. So per task to solve the same problem. Uh, usually, those are kind of uh the relative uh scale we're seeing.

• 37:45Lukas Biewald?

  What about quality right now? I mean, how do you view the quality trade off between the open source, the best of the open source models and the frontier labs?

• 37:54Speaker 01

  Um, so I think across the board, open and closed model quality has passed the threshold um of solving a lot of day to day problems. And interestingly, a lot of the tasks we work on, especially especially professionals can can be interpreted as a coding problem. And the coding is kind of by and large is kind of solved problem. So, so then it makes a lot of our kind of um professional task very friendly to be tackled by, uh addressed by those uh those models.

• 38:30Lukas Biewald?

  What do you mean, sorry?

• 38:32Speaker 01

  Um, so again, goes back to um, you know, most of the coding. Um, like all these model providers, close, open, they really care about coding because coding is the foundation of solving the next several problems. And then across the board, there are many like cowork problems. For example, uh I think our company is a reflection of um industries, like, within a company, every company has finance department. So our finance use um uh use uh both open and closed models heavily to do finance forecasting, uh manage our books.

• 39:07Lukas Biewald?

  So you actually use closed models inside?

• 39:10Speaker 01

  We do, we do, we do, we use both, right? So again, as I mentioned, um, and another new frontier is how to blend those models intelligently.

• 39:21Lukas Biewald?

  So I guess you would be a huge proponent of open models, right? So what's the cases where you use closed models?

• 39:26Speaker 01

  Uh, so again, as I said, um, I think specialized intelligence and the general intelligence, they will coexist. Um, they will coexist as in uh during the different phase of development. In early phase of development, uh you don't want to think about which model to pick. Just pick, and you also don't care about cost. Uh, none of other later stage concern, production scale concern kicks in. And they just for simplicity, want to pick one and then you can pick the kind of most expensive one. Just kind of see if it's even viable, i- is there even viable solution, right? So that's kind of, and as the development goes into, hey, it's validated, it's a viable solution.

• 40:07Speaker 01

  And you now want to kind of really scale and then you want to kind of start to optimize. So that's one possibility. Um, the other possibility is different model actually uh special, slightly geared towards solving different problems better. Um, I think even across uh even across kind of closed models. Um, of obviously, Anthropic is the best uh tackling um coding agentic. Um, and really focus on instruction following. And before, uh OpenAI has been focused on uh interaction uh base and deep research. And now they are also focused on uh coding. And uh, and Gemini used to focus on multimodality really, really heavily.

• 40:50Speaker 01

  Um, so now they also focus coding, so coding becomes a common denominator, but still uh different labs have their like special uh secret sauce and special focus. Um, and even from there, that just decide your starting point may be different. And same for um, you know, same for uh open models, right? For example, like G L M just doesn't have vision component. Uh, and Quinn has been very good at vision uh from the beginning. And Kimi recently, they are combining, right? So, um, yeah, so I think based on the task, we have the uh, we will have, we will give our customer the guidance. Uh, which could be your best starting point uh to to kick off your job.

• 41:33Lukas Biewald?

  Do you think there's a security issue here as well with the the open source models versus the closed models?

• 41:38Speaker 01

  This is a very deep topic. Um, I think it's a lot of debate now happening um on Twitter and across the industry about security. So, so here's the controversy. Um, security always have two sides, the attack and defend. Uh, so if we, um, the challenge of security is um if there is, if it's asymmetry. If the attack has better tool and defend side, then it's really bad. Um. If the defend has better tool than attack side, that's really good.

• 42:19Speaker 01

  But usually it kind of, it it will kind of get the equi- equivalent that they are on par. That I think that's kind of a healthy situation. So uh not saying kind of we should encourage attacker to have better tools, but they will find all the ways to kind of acquire that. So because of that nature, I feel like open model is a way to strike that balance. Um. And the other benefit of open model is um the encourage a broader community to continue to build on that, to um to increase the defend um like complexity. Um. Because the model can be post-trained.

• 43:02Speaker 01

  Uh. So recently, Hugging Face, our friends, uh uh Clan, uh they they work with OpenAI to resolve that incident that kind of pretty well establishes. They couldn't get um, they couldn't get um um open as model and fail to be able to work. Because it detect, oh, this is a possible uh cybersecurity uh breach and um refuse to uh find solution. And they have to kind of activate GLM five two uh to find the solution quickly and contain the problem. Uh. So that's, that's really about accessibility and and the the defense.

• 43:37Lukas Biewald?

  I'm not sure I heard that part of the story. So so didn't, I thought what happened with Hugging Face was OpenAI had a model that in an eval stage actually kind of hacked Hugging Face. Is that right?

• 43:46Speaker 01

  I my understanding is uh it was using uh it it was testing Export Jam, uh which is um cybersecurity attack benchmark. Uh. And that just starting to um go um go wild.

• 44:01Lukas Biewald?

  But then the models also refused to fix the cybersecurity issue?

• 44:04Speaker 01

  That's my understanding.

• 44:05Lukas Biewald?

  Interesting. What about um China? I mean, it's kind of interesting that all the open source models, the best ones seem to be coming out of China. Do you think American companies should be concerned about using Chinese models?

• 44:18Speaker 01

  Um. So I think we should differentiate the geopolitical debate from open model versus like close ecosystem. Uh. Based on like I've been working on Pedro for a long time. Uh. You're also in that ecosystem, right? So um I think uh we truly believe in open. We truly believe in open development, open um open intelligence. Um. Because we have seen so many good things. Not be even before Pedro, I've been working on many open source project all the way from. Uh. If you think about um before AI, data is the uh one of the biggest innovation. There's so many open source project about data.

• 45:02Speaker 01

  And I believe that's kind of the fundamental reason. The data field is able to move so fast across the whole industry. Uh. And the from there, it derive a huge variety about different kind of data processing. Um. And that power, all sorts of kind of um part of our economy, all the way from um data analytics, this new field, uh to be able to kind of uh for all the business metrics, be able to kind of uh standardized processing, make data-driven decision to uh online ranking recommendations as everywhere in our digital life. Um. That is um that is significant power by a new um that now it's not new, everyone is using that.

• 45:44Speaker 01

  But product analytics is a new practice to do, to build product. Uh. To um self-driving cars is huge amount of data uh from perception. So uh so all this data innovation heavily depends on um all the open source uh project to do all sorts of kind of interesting data processing. Um. And fast forward to now, I do believe we are in this um crucible moments. Um. That across the industry, we should encourage more open intelligence. And to me, it is very, it it kind of doesn't make sense.

• 46:23Speaker 01

  Intelligence is in, it is in the hands of a doably or in the hands of a few people. Um. I have never seen a community evolve um in in in the balanced way, especially we're talking about security, cybersecurity in the balanced way and in in the best way. If if it's, you know, only small, the more people has control over intelligence.

• 46:49Lukas Biewald?

  Why do you think we have this dynamic where Chinese companies are mostly leasing open source models and American companies are mostly closed source?

• 46:58Speaker 01

  Um. I would really think there's no reason American companies shouldn't open source. Our American companies shouldn't open source the best models. I I couldn't find a reason. I I think we should.

• 47:13Lukas Biewald?

  Interesting. Do you think OpenAI should just open source, and Anthropic should open source their models?

• 47:17Speaker 01

  I was strongly hopeful. OpenAI had done that before. And last year was a high moment uh for the community. Well, here is kind of a strong open model coming out. Um. And I would hope they continue to do that. Um. Especially, you know, they are the leaders in the market. And they should set example for the whole entire industry.

• 47:37Lukas Biewald?

  Well, I think what they would say, though, is like, hey, we we spend all this money in R and D. We need to recoup that money by running the model. Do you have some alternate monetization plan for them? Or do you sort of feel like this is like the moral thing to do? Or what do you mean by saying they should, they should open source their models?

• 47:51Speaker 01

  I I think, I I think whether they open source or not, um I think they need to have a reason to do that. Um. And it should, you know, hopefully connect with their monetization uh path. Um. But at the same time, there are many other American companies I, we work closely with. And they are deeply, deeply um passionate about uh open source, the next generation, like really push the US open source quality to the next level. Um. We work with Microsoft, M A I, we work with Nvidia, Nimotron. Um. You know, hopefully Reflection will have something come up soon. And Thinking Machine just launched their model.

• 48:34Speaker 01

  Uh. So I think we're just at the beginning. Um. Again, based on.

• 48:38Lukas Biewald?

  It still feels, honestly, like the best open source models are coming out of China and the most like kind of interesting innovation. And maybe you could add Mistral to the list, but sort of outside of America seems like the the models that are really in use are generally not American, right?

• 48:53Speaker 01

  Right. So I think it's probably the strategy. I I think in the past, our primary strategy for US companies are focusing on whether those are model frontier labs or hyperscalers. Um. They are mostly focused on kind of pushing the closed model because, again,

• 49:15Speaker 01

  it it is fastest to scale one model in terms of business, right? So if you think of business efficiency, that is the fastest. Um. But then it has a limit. Uh. Because you you cannot get into the special part of your business, right? If you are general purpose, solving the common problem, task kind of model. And that's what we believe the next phase of frontier is to get into. The most interesting part is getting to special uh special proprietary uh knowledge of the business, let business own their own intelligence. And that will push the next frontier. And that work just started because um the the base model

• 49:57Speaker 01

  where business can build their specialized intelligence on the quality has a positive threshold. And therefore making that flying well much easier, more accessible, much better result and so on. Um. And before we have been pushing on that, but the the uh quality difference is too big. Uh. So I believe with um now with a lot of interest from American companies are trying to uh build really, really good open model. And the American um gap between close and model, open will close. Um. While like we also have other uh choices of open model across different regions.

• 50:36Speaker 01

  I think the open community, open ecosystem will have way more diversified kind of contribution. Um. And um that make me so so much excited.

• 50:48Lukas Biewald?

  It does seem like there's been a lot of innovation recently in making the models run more efficiently and faster. I feel like a lot of these newer models are kind of designed to run well and and um.

• 51:00Lukas Biewald?

  And related to that, an amazing thing that's happening in our space is when a new model comes out, it's almost immediately supported by fireworks, like same day. It's unbelievable how fast you guys, you know, get these things working well. Are you in conversation with the model building companies about what they're doing? And kind of getting early previews and testing them on your infrastructure before the actual release date?

• 51:23Speaker 01

  Yeah, so it really depends on situation by situation. Sometimes we have early access, sometimes we don't.

• 51:32Speaker 01

  And regardless, I think the day we launch. So by the way, we, you're right, we're very proud of day zero launch. We almost kind of have our reputation. Fun fact, I think early days of Mistral, they, they didn't launch the model. They just toss out the weights, there's no model code. And we were engineers, the model code from their previous launches and launch before they launched API. It's, it was a fun exercise. And, but we care about quality more than day zero launch. When DeepSeek before launched, we hold back the, the launch from our side by three days. And that three days, we didn't sleep at all.

• 52:17Speaker 01

  The reason is the, the release, the, the weights we got and the corresponding code we got has a lot of bugs. And we, it doesn't pass, we have a lot of internal evals and it doesn't pass our threshold.

• 52:34Speaker 01

  And then we work closely with, we actually work with VLM and SG lang, the open source community to fix those bug. We fix those bug and contribute back. So they also can fix those bug with their community. So, so that took us three days and we launched three days later.

• 52:54Speaker 01

  But we just cannot, we cannot deploy a model where we know there is an issue that trumps everything.

• 53:00Lukas Biewald?

  Why is it so important to you to launch so quickly when a new model comes out? I mean, you're talking about specialized intelligence. Nobody's going to be able to take the model and specialize it to their application on, on day zero. So why, why the rush? Why are you telling people to not sleep for three days to, to get something out?

• 53:16Speaker 01

  People, so this is an interesting time because,

• 53:21Speaker 01

  because of velocity of AI development is so fast.

• 53:26Speaker 01

  Our customer always want to get in touch with the latest. And there's no time to wait. So that's kind of the, because they are going to make a decision. Are they, they already tune the model and are they going to change the backbone of the model? This is a big decision. And because the model release cycle is so fast and very close to each other. And they need to be informed quickly. And today, no one really trusts public benchmark anymore. The judgment is really coming from be able to test through their own internal eval and, and make those calls.

• 54:08Speaker 01

  Because if they decide, oh, this actually is a really good base and they want to tune it, they need to act quickly. Because if they don't move quickly, the next model will come up. So it's just the pace of AIs in the whole entire industry so fast.

• 54:21Lukas Biewald?

  I mean, that leads me to another question about kind of the, the shelf life of these models is so low. People always switch to the next great model. And you have this business that's scaling amazingly well, but the metrics are so public for everyone to see. You can see the cost and the performance and there's not much else. Like what is your long-term mode for fireworks?

• 54:46Speaker 01

  So many people ask me, what is fireworks, first of all? And then we can talk about mode based on that. They ask me if I was a, it's an inference company or if I was a, is a new cloud, a new, a new cloud? Or fireworks is a PyTorch cloud and all these fireworks. So we are none of this. So we're a specialized intelligence platform. I think we have built a training platform, the inference platform, co-optimize, co-designed with the goal of maximize quality. So we are very obsessed with quality, but after quality, after we reach quality, then we will.

• 55:26Lukas Biewald?

  When you say quality, what do you mean? Is that the, the quality of the result?

• 55:30Speaker 01

  Quality of the result, yes. And after quality is good, then no, this model will be in the play, right? If the quality, I haven't seen people sacrifice quality significantly for a much cheaper price. Almost like if you don't care about quality, when you build product, then I'm not sure like this is going to help you. So, so then after that, we, we customize our inference deployment for, for the specific application for speed and cost. So, so, okay, let's go to quality, right? We are obsessed as in we, we go actual miles to get quality.

• 56:13Speaker 01

  For example, between training and inference, when there's a transition, there could be a loss of precision because of numerics, numerics difference, different libraries you use. And online numerics is a very hard challenge. So we reach zero KLD across training inference.

• 56:31Lukas Biewald?

  Zero what, sorry?

• 56:32Speaker 01

  Zero, zero KLD.

• 56:33Lukas Biewald?

  What is KLD?

• 56:35Speaker 01

  KLD is a metrics of, of the precision. So what it means is we reach bitwise equivalent. So the, the result in running from the training side, the result from inference side is bitwise equivalent. And why that matters is after you're training to deploy, right? So you don't want to lose precision there. Or for RL, RL actually is a combination of trainer and I roll out inference, that's inference. And the counseling going back and forth and a lot of things. And then if, if the numerics are not, has small arrow and they start to amplify. So this is a very hard to achieve.

• 57:19Speaker 01

  And we just spent all the R and D to kind of achieve that. And second is almost like a principle we operate is we want to make our, we want to make our platform, this tool accessible by all sorts of companies. And very few company can afford 10,000 GPU fully interconnected. There's just not so many, not many of those. And not even many of those available, if you want to spend the money. So we, we implement a very aggressively tiered disaggregated implementation where we can,

• 57:59Speaker 01

  we can basically pull together scattered GPU across all regions globally to do a training run up to, you know, tens of thousands of GPU in one run. So, so this requires a lot of innovation and the innovation is not to kind of make it work. The innovation is make it work without losing quality. Because the more synchronized you have across the, across different clusters, the more possibility there, there's arrows. And then you, you just spend the money, but the result is not there. Those are examples where we're really obsessed about quality. But after quality, we, we continue to obsess about a speed optimization, cost optimization.

• 58:41Lukas Biewald?

  So you don't want to be just in the game of competing with other inference providers on price and speed?

• 58:50Speaker 01

  So we will compete, we will absolutely compete. But I think, I think the, the unique, again, goes back to every company exists for, for a reason. And the reason for us to exist is we are squarely focused on one size fits one. We squarely focus on customization from our belief. Every single company is special and we want to build, deliver the special intelligence for them. And that reflect in, in special quality, special cost and, you know, and speed. And that's all take, we'll do whatever to kind of optimize for that. And that's where we build our platform for.

• 59:31Lukas Biewald?

  So it sounds like you are committing back to open source libraries like, you know, SG lang and VLM. But as far as I know, the, the core of what you do is closed source, isn't it?

• 59:42Speaker 01

  Yes, we both our training and inference engine are proprietary.

• 59:47Lukas Biewald?

  So I could imagine, you know, you're here telling me, hey, OpenAI should open up their models. I could imagine talking to someone at OpenAI and they say, hey, Lin should open source her runtime and training environments. How would you respond to that?

• 59:59Speaker 01

  Yeah, so first of all, we, why we do proprietary, right? So we build the engine before VLM, SG lang, so they don't exist. But also we need a special design because again, goes back to the roots. We want to customize to the extreme by, by, you know, deliver extreme quality, speed and cost. That requires we design the engine in a special way. And we want to maximize the choice we can make

• 1:00:29Speaker 01

  to extract the most amount of optimized result. And that requires us to design the engine in a very modular way. So each module and the modules are, and they, they can interact with each other, fully compatible in terms of the inter, interaction, interface. And then each module will have a few choices and we can compound those choices and kind of, and then it becomes a search problem. So for inference engine myself, we create a search space of more than 100,000 options. And then based on the customer requirements, remember every company is unique, their workload is unique. Their, their trained model and result is unique. Then we search across those options and find one that's best for them.

• 1:01:14Speaker 01

  So because of this unique design, it's hard to manage other engines to deliver the result we want to deliver. That's why we build the proprietary. And, and we do, we do work very closely, again, the DeepSeek case, we do work closely with the open source community. We do work closely with NBDR. We give them a lot of feedback as in kind of what kind of, we also work with MD to get the feedback, what kind of kernel we need. And whatever feedback we give to them is going back to open source. So we feel like this is the most efficient way to engage. Because we also want to have the velocity of moving forward really, really fast. And that's kind of strike the best balance.

• 1:01:59Lukas Biewald?

  So are you saying that you don't want to open source because you don't want to deal with outside contributions? Or you want to protect your IP? Or what is it?

• 1:02:07Speaker 01

  We don't think it will be productive because I've done open source before, right? Multiple rounds.

• 1:02:13Lukas Biewald?

  Of course.

• 1:02:14Speaker 01

  It is a lot of, it requires a lot of people to engage and build a community. And we have done that before. And if I have time, like, for example, PyTorch, it took us seven years to really be kind of broad. And now it doesn't seem like, everything is moving so fast. And the velocity of we changing things is extremely fast. And when you change things a lot faster and you open source, and people get confused. Where did they contribute? And then if they, if their contribution is not being incorporated in time, and they will not be happy.

• 1:02:57Speaker 01

  So I don't think, we actually discussed that internally about open source, our engine.

• 1:03:05Lukas Biewald?

  Do you think, I mean, you now have lots of resources, like surely you could hire people to manage community. Would you ever open source the fireworks engine?

• 1:03:12Speaker 01

  So guess how many people we have?

• 1:03:15Lukas Biewald?

  A hundred?

• 1:03:17Speaker 01

  More than that. We do have more than that. We have more, we have a hundred, a hundred engineers and product people, tech. So total, we have two hundred people.

• 1:03:27Speaker 01

  So it's actually a very small team. The people working on the engine part is around ten people.

• 1:03:36Lukas Biewald?

  I guess this is, you seem so passionate about open source and you're saying, oh, everything should open source. Like, open source always wins. It's kind of just interesting that you're not excited about open sourcing the core thing that you're building.

• 1:03:47Speaker 01

  I would say I would do, I will work on things if we add tremendous amount of value. We already have great open source project, VLM, SG Lang, even including TRTLM. It's great. I'd rather support them to go really big than I build another open source project to compete with them. So I feel like this is kind of the part of open source is, it's, there's no ego. If there's a leader in the open source world, then we will support a leader.

• 1:04:17Lukas Biewald?

  If there's a leader, would you switch to using it?

• 1:04:20Speaker 01

  Oh, we are open minded. We're open minded. Yeah. Again, we are very practical. If it help us to get extra miles of optimization, we'll open minded to use any, any tools.

• 1:04:34Lukas Biewald?

  Do you have a feeling right now between VLM, TensorRT, and SG Lang, which is the leader? Is there one that you favor?

• 1:04:44Speaker 01

  I think they have unique strength.

• 1:04:45Lukas Biewald?

  Oh, man.

• 1:04:46Speaker 01

  They have unique strength.

• 1:04:47Lukas Biewald?

  Come on.

• 1:04:49Speaker 01

  And I admire the effort. I know it's very hard to build an open source project.

• 1:04:57Speaker 01

  I admire they kind of keep, keep focusing there. Although each of them are starting to build companies, I think that's great to, for them to kind of

• 1:05:07Speaker 01

  have a real entity to see into the future. But, but I'm, I think they're doing the right thing for the community.

• 1:05:15Lukas Biewald?

  Do you use strategies like Andrej Karpathy's research? Where an L M kind of iteratively tries new sets of parameters and decides what to try next?

• 1:05:28Speaker 01

  Yeah, to build kernels and so on. So, I mean, it's not surprising we're doing that too, but it's not easy. I don't think it's gonna replace performance engineer anytime soon. The thing is, we haven't find, the model can do things we know, already know how to solve. And the discovery part of finding new ways to write kernel, we haven't seen that. We were trying all different kind of way, we haven't seen that happening yet. Maybe it really depends on the base model quality. Maybe there's another leap into much deeper thinking.

• 1:06:10Speaker 01

  When model goes hit like ten trillion parameter, it may be possible.

• 1:06:16Lukas Biewald?

  Okay, well, switching gears a little bit.

• 1:06:19Speaker 01

  Switching gears is scary.

• 1:06:21Lukas Biewald?

  You should be scared. Actually, I want to say, I've always admired your style as a founder and related to it a little bit in that you have kind of a quiet confidence, but I think unlike a lot of Silicon Valley founders, you're not really out there kind of beating your chest about, you know, hey, I'm so great. I remember when I was running my company, I sometimes got criticized by my board, saying, hey, you know, you should be out there more, you should be like more aggressive. Do you ever feel like you should change your style or be a more aggressive founder? Be maybe more like some other company leaders?

• 1:06:56Speaker 01

  I think I, I think about that every day.

• 1:06:58Lukas Biewald?

  Interesting.

• 1:07:00Speaker 01

  But I cannot change who I am, right? I think the, the way to present ourselves, myself, I need to feel I'm authentic.

• 1:07:10Lukas Biewald?

  Totally.

• 1:07:10Speaker 01

  I cannot be anyone else. I cannot be anyone else as a CEO. I cannot be anyone else as I speak for the company. I cannot be anyone else as I even speak about my personal opinion just for the sake of getting a following. So I think that's part I'm, I'm trying to learn, you know, where's the balance? But be able to talk about company is absolutely important. So on that front, I agree with you. That's, that's the part I've been working on for a long time.

• 1:07:43Lukas Biewald?

  I don't know if you're really agreeing with me. I'm saying I like your authentic style and I'm not sure I want you to change.

• 1:07:49Lukas Biewald?

  From where I said, I was just curious.

• 1:07:50Speaker 01

  But I, my point is like, there, there's a lot more aggressive style of marketing. And I'm not sure I agree with those. But not, like, people doesn't know where we're building, who we are, is, is not okay with me also.

• 1:08:10Lukas Biewald?

  I mean, I guess you're suddenly running this incredibly important company in the AI space. How has that experience been? Do you feel overwhelmed? Do you feel excited? What's, what's it like?

• 1:08:26Speaker 01

  I feel I'm living my dream.

• 1:08:29Speaker 01

  Because it's a lot of work, obviously, kind of. Everyone working AI, there's kind of, it's very intense, it's fast paced. Things constantly changing, market is very dynamic. But I really enjoying solving all kind of problems across the board. It's not just technical problem, it's such a problem. It's always kind of intersection of product go to market, procurement, finance. It's kind of multi-dimensional. That makes me super excited. And we never lack of those problems to solve. Every day is a new problem pop up and I need to jump in. So our style is kind of, we are very flat.

• 1:09:10Speaker 01

  There's no deep hierarchy and I don't like deep hierarchy because I want to make sure like people all have similar kind of context and be able to make decision. But also don't want to carve out the boundaries. Hey, you know, here's a box, you fit there, here's a box, you fit there. And we stack the box and then we build organization that is not a kind of, it doesn't fit into velocity needed for, by AI. And, and because of that, I got to work deeply with all sorts of teams. That, that makes me very, very excited and energetic every day. But at the same time, it's very humbling.

• 1:09:51Speaker 01

  It's very humbling to see there's so many entrepreneurs fearless, doesn't matter whether they have experience or not. They just go in to, to implement their ideas and figure out the way to get to the market. The creativity is off the chart. And we are, we are doing our best, but I've seen so many other peer entrepreneurs doing their best. And it's a very inspiring environment.

• 1:10:22Speaker 01

  And we'll keep heads down and deliver the best for them because they are, most of them are our customers.

• 1:10:29Lukas Biewald?

  What's been the most surprising thing about actually being in charge of the whole company?

• 1:10:38Speaker 01

  I think.

• 1:10:41Speaker 01

  Eric asked me when I first raised my Series A.

• 1:10:45Speaker 01

  He asked me one question. What would you do differently now you're founder compared with a public company execs?

• 1:10:54Speaker 01

  Guess what's my answer? I go ahead.

• 1:10:58Lukas Biewald?

  What would your answer be? Well, now that I'm part of a public company.

• 1:11:03Speaker 01

  Yes.

• 1:11:03Lukas Biewald?

  One thing I wish we could do is share all the metrics with all the employees. I feel it's a big bummer to not be able to give all employees access to all the relevant metrics. I just feel like I want all the people to know like what the score is and what, you know, True North looks like. But there's a lot more controls around that inside of CoreWeave. But surely that wasn't your answer. What was your answer?

• 1:11:24Speaker 01

  My answer is I don't know.

• 1:11:26Lukas Biewald?

  I love it. That's a great answer, see?

• 1:11:32Lukas Biewald?

  Oh man, that's so good.

• 1:11:34Speaker 01

  I haven't done the company before, so.

• 1:11:36Lukas Biewald?

  Nice.

• 1:11:37Speaker 01

  So I don't know. Obviously, the subtext of his question is there's not much success of, you know, established public company exec to like start a company. Why you are able to build a company? So. So I'm lucky. I started a company with six other co-founders. And they are all top-notch, world-class engineers and technologists. And they are the foundation for us to build this company together.

• 1:12:12Speaker 01

  And I'm more on the kind of the business side and they're on kind of product technical side.

• 1:12:18Lukas Biewald?

  That's so funny. I think your view is so technical. But you're the most business-oriented person of your.

• 1:12:24Speaker 01

  Oh, I'm the dumbest among them. I'm the dumbest.

• 1:12:32Speaker 01

  And a character of this group is the deep intellectual honesty.

• 1:12:40Speaker 01

  The one thing we do a lot, I don't know if this is weird or not, we do a lot of pre-modern.

• 1:12:48Speaker 01

  And as pre-modern is how we're gonna die.

• 1:12:51Lukas Biewald?

  Totally.

• 1:12:52Speaker 01

  And then we got used to it. When other people join the company, you join the company, and we bring them along with us to do pre-modern. They're like, that's scary. And how can you discuss that and go back to that and continue working as if nothing happened?

• 1:13:08Lukas Biewald?

  Totally, yeah.

• 1:13:10Speaker 01

  But we're very candid and transparent. And we want to make the best decision for the company and also see through the strategy. But a lot of time, I think the biggest, he asked me, what's the biggest surprise? Starting this company is there's no data, not sufficient data to make

• 1:13:32Speaker 01

  like data-driven decisions, especially early on. At, you talk of metrics, as Facebook matter. The abundance, there's abundance of data. You can go to like granularity of a point, point one percent of the lift and try to work on it. It's a massive impact also. You have all data to kind of make a judgment which path is the right path. But in the start of a lot of time, there's no data because we are, we travel and pave the path. No one have traveled. If everyone's traveling that path, then you shouldn't be that company. So a lot of that, a lot of decisions got caused.

• 1:14:17Speaker 01

  And intuition, but we have to validate. So the feedback loop of validating is important. So it's okay to, it's okay to say this doesn't work out and we need to shut it down. But it's not okay to not make a decision because of lack of data. Not making decision is a bad decision. So we will never, we never want to be analysis paralysis. And that's why we do a lot of kind of dissimulation. And try to kind of make the best calls and then keep adjusting. Any feedback? Like I'm trying, I'm starting to do the more, like to your point. I feel like

• 1:15:02Speaker 01

  one thing I regret is I didn't do marketing early enough.

• 1:15:07Speaker 01

  Because, hey, we're a bunch of engineers. Engineer has this cynical view of marketing. And based on our past history, we believe product will speak for itself. But this market is so noisy, so noisy, insanely noisy. And people cannot differentiate signal among noise from signal. So I'm trying to do a little bit more. So give me feedback. Like what's your like thinking there?

• 1:15:28Lukas Biewald?

  On marketing? I don't know. I mean, you're like phenomenally successful. Like I do think.

• 1:15:34Speaker 01

  No.

• 1:15:35Lukas Biewald?

  I think that, you know, if you market without the product to back it up, that can be bad too. I feel like.

• 1:15:40Speaker 01

  There's so many company doing that, right?

• 1:15:42Lukas Biewald?

  I know, but I think it's actually bad. Like I feel like Firework has a great reputation. And your customers speak for themselves. Like I think, you know, I really like your authentic style.

• 1:15:53Speaker 01

  We're a similar style, I feel.

• 1:15:54Lukas Biewald?

  Yeah, we're similar style. I mean, I think you, you know, you should find.

• 1:15:59Lukas Biewald?

  I think, you know, you know the space better than any marketer could. So I think if you, you know, if you do do marketing and build out a marketing team, I just think like really trust yourself and keep it as authentic as you are. Like the things I've regretted in marketing has been when I've let kind of someone come in who's like, like in some cases, like, you know, for like a technical thing, you want the expert in technical thing. Like you wouldn't want to hire a CISO that didn't know more about security than you. But I think like marketing is such a core expression of what the company does that don't let a marketer convince you that something you feel uncomfortable about is actually gonna be good. Like a lot of times marketers come in, they say, oh, this is like enterprise marketing. Enterprises want to hear this. And my experience, it's like always bullshit.

• 1:16:44Lukas Biewald?

  Like I think enterprises want to feel like they're dealing with an authentic company. They want to be spoken to in an authentic way. Like at the end of the day, there's a human beings behind that. And you understand your customer. So I think, you know, marketing can help

• 1:17:02Lukas Biewald?

  get the message out, but don't. If I were you, I wouldn't let a marketing person come in and tell you what the message should be. Like it seems like you feel really good about specialized intelligence. That feels authentic to you. Like I would lean on that and just keep it.

• 1:17:19Lukas Biewald?

  Keep it authentic. Because honestly, from the outset, it looks to me like it's working.

• 1:17:22Speaker 01

  Okay.

• 1:17:23Lukas Biewald?

  So.

• 1:17:25Speaker 01

  Anything we can do better?

• 1:17:26Lukas Biewald?

  What's up?

• 1:17:26Speaker 01

  Anything we can do better there?

• 1:17:36Lukas Biewald?

  I don't know. I mean, I liked, I like, I like, I feel like you actually have a pretty clear message. I mean, I guess I didn't, I think of you as an inference company, so you're all kind of fighting.

• 1:17:45Speaker 01

  We're still bottom.

• 1:17:47Lukas Biewald?

  Yeah, you're still kind of like fighting a simple narrative. So I think you're just gonna have to keep saying it over and over. You know, and I think your website doesn't.

• 1:17:54Speaker 01

  Reflect that.

• 1:17:55Lukas Biewald?

  Reflect the messaging that you're saying so that, you know. It's just like, you can't deliver a very nuanced message ever.

• 1:18:02Speaker 01

  Yes.

• 1:18:02Lukas Biewald?

  To people. So like you just need to be like specialized intelligence, that's what we are.

• 1:18:06Speaker 01

  Yeah.

• 1:18:06Lukas Biewald?

  Here's where it is.

• 1:18:07Speaker 01

  Keep bending on that.

• 1:18:08Lukas Biewald?

  Just bang on, you know, the two words and you're just not gonna be able to deliver much nuance broadly, but.

• 1:18:17Lukas Biewald?

  But like, you know, you built like the industry-leading company in your space. It's awesome. Like I don't know.

• 1:18:24Speaker 01

  I'm always.

• 1:18:25Lukas Biewald?

  I should be learning from you.

• 1:18:26Speaker 01

  No, I'm always paranoid. I'm always paranoid.

• 1:18:29Lukas Biewald?

  Totally.

• 1:18:29Speaker 01

  Yeah, because you know the market is so dynamic.

• 1:18:32Lukas Biewald?

  Yeah, I know, I know. I mean, that seems crazy, like the

• 1:18:36Lukas Biewald?

  it's so dynamic. I mean, yeah, it seems like it'd be stressful to be in your seat.

• 1:18:48Speaker 01

  But at the same time, it's good to have a team. So, yeah, yeah. We're building on the team.

• 1:18:53Lukas Biewald?

  Awesome. Well, thanks so much. This is a great interview. I really appreciate your time.

• 1:18:57Speaker 01

  Oh, thanks.

• 1:18:58Lukas Biewald?

  Thanks so much for listening to this episode of Grading Descent. Please stay tuned for future episodes.