How DeepL Built a Translation Powerhouse with AI with CEO Jarek Kutylowski

In this episode of Gradient Dissent, Lukas Biewald talks with Jarek Kutylowski, CEO and founder of DeepL, an AI-powered translation company. Jarek shares DeepL’s journey from launching neural machine translation in 2017 to building custom data centers and how small teams can not only take on big players like Google Translate but win. They dive into what makes translation so difficult for AI, why high-quality translations still require human context, and how DeepL tailors models for enterprise use cases. They also discuss the evolution of speech translation, compute infrastructure, training on curated multilingual datasets, hallucinations in models, and why DeepL avoids fine-tuning for each individual customer. It’s a fascinating behind-the-scenes look at one of the most advanced real-world applications of deep learning. Timestamps: [00:00:00] Introducing Jarek and DeepL’s mission [00:01:46] Competing with Google Translate & LLMs [00:04:14] Pretraining vs. proprietary model strategy [00:06:47] Building GPU data centers in 2017 [00:08:09] The value of curated bilingual and monolingual data [00:09:30] How DeepL measures translation quality [00:12:27] Personalization and enterprise-specific tuning [00:14:04] Why translation demand is growing [00:16:16] ROI of incremental quality gains [00:18:20] The role of human translators in the future [00:22:48] Hallucinations in translation models [00:24:05] DeepL’s work on speech translation [00:28:22] The broader impact of global communication [00:30:32] Handling smaller languages and language pairs [00:32:25] Multi-language model consolidation [00:35:28] Engineering infrastructure for large-scale inference [00:39:23] Adapting to evolving LLM landscape & enterprise needs

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Lukas Biewald?

  You're listening to Gradient Dissent, a show about making machine learning work in the real world, and I'm your host, Lukas Biewald. Today, I'm talking with Yarek Kutylowski. He is the CEO of DeepL, which is a very successful GenAI company. You might not have heard of it because what they do is translation and a primary focus of their business is on enterprise, but they are making really, really significant revenue off of a really specific GenAI use case. I think translation is a really interesting category to talk about when we talk about GenAI because it's one of the first categories that's being completely disrupted by AI systems. Many, many human translation companies have gotten into trouble. They're starting to shrink as GenAI takes off.

• 0:46Lukas Biewald?

  I think it's a real bellwether for where lots of industries are going. So this is an interesting conversation about both the business implications of running a company in this space and also the technical implications of how do you stay ahead of companies like OpenAI when you have a specific use case? Yarek was very forthcoming with answers to my questions, and I found it super interesting. I hope you enjoy it. I was really excited to talk to you as CEO of one of the most interesting GenAI companies that maybe a lot of people haven't heard of, but I have. But I think you better introduce your company to our audience.

• 1:22Speaker 01

  Yeah, hi. Thank you for having me. It's a pleasure. So I'm Yarek, CEO and founder of DeepL. And DeepL is a company that has actually started a little bit before the AI hype. We launched in 2017, and we've been using AI to tackle the language problem in the world. We're specialized in translation specifically for businesses, like in all of those use cases where you have customers in a different country, where your companies may be spread across the whole world. And we're trying to provide solutions which just help you cross that language barrier as good as possible.

• 2:04Speaker 01

  And AI has made amazing strides in making that so much simpler. Yeah, that's basically us.

• 2:11Lukas Biewald?

  And I guess what's kind of amazing about you is like, you know, translation is such a

• 2:18Lukas Biewald?

  It's such a fast changing space. Like, you know, my, my background actually wasn't building translation models back in the aughts and, and um,

• 2:24Speaker 01

  Oh wow.

• 2:24Lukas Biewald?

  I don't think any of it is relevant, you know, at all anymore. And then um, you're also kind of going up against, you know, Google Translate and like actually like, you know, all these language models can kind of do some translation if you ask them. So I feel like you're going head to head against these um, these kind of like juggernauts, but um, you know, beating them on like quality and technology and the technology, the underlying technology is so different now in terms of state of the art than, you know, when you started. So I guess maybe could you talk about like what you, what, how you feel your technology advantage works?

• 2:57Speaker 01

  Yeah, yeah, I, I, I think it was a totally different space when we started. As, as you say, like, it's been a fast changing moment and, and we were really lucky to start in 2017. I think there was this, this, this moment when everything turned to neural machine translation. And I, I think we chose that moment really wisely because everybody had to throw away basically what they have been doing until now. Uh, everyone had to switch over to neural. And at this point in time, I think for a startup, there was like this opportunity to, to go ahead and and build models that are, that are excelling uh, what uh, what was out there maybe

• 3:38Speaker 01

  in academia or uh, what, what the others has, have been doing. And uh, back then it was a lot of own architectures, uh, kind of creating just the best model type uh, that can, that can suit uh, translation. Like Transformer came out very, very quickly. But we found like there's, there's actually better architectures for translation specifically. On the one hand, you need to kind of generate text, of course, but you have to stick to uh, to also what you're seeing.

• 4:09Speaker 01

  It's the source text. You have to both maintain a certain level of accuracy because you the translation needs to be the kind of near to the source text. But at the same time, you want to write in the target language natively. So to say like you, you don't want to let the model do word by word translations. You want to have to give it a little bit more creativity. So to just kind of this mix of both models that are that are good at copying and also at writing itself, monolingual, bilingual. That was there was something that we've been working for for quite a while. And and that has only continued like model sizes are much bigger right now.

• 4:52Speaker 01

  Reinforcement learning and all of those techniques are coming in, also allowing those models to do more than just plain translation from sentence to sentence. So it's been quite a journey. I I think the advantage that we have really comes from the fact that we're focused on this on this one area. And those models that we built, even though they might be really competing with with the large ones on size, they're still really very much focused on on this on this on this use case that that we're building them for.

• 5:23Lukas Biewald?

  But it does seem like, you know, like pre-training, like a model on language would kind of, you know, inform how how translation works. So do you do you train your models completely from scratch? Or, you know, would you use like, you know, if if like, you know, Meadow wants to, you know, publish a llama model and spend, you know, millions and millions of dollars on making that, would you would you use it? Or how do you think about that?

• 5:50Speaker 01

  Yeah, yeah, we are. We're looking at those and we're using them them as pre-trained models. We're we're putting still a lot of compute on top of that.

• 6:00Speaker 01

  There is there is just this advantage of of training on specialized curated data that that we've built up over the years and also making sure that that we have a proper distribution of all of the different languages. I mean, those models need to need to be able to to tackle maybe not only English and German, but also a few more languages, the kind of smaller ones. And for that, you really also have to have the data and give the model the the the training steps to look at at this data.

• 6:32Lukas Biewald?

  And so how much of what you do is like basic research and how much is like engineering the the models?

• 6:39Speaker 01

  That's that's really, that's really a good question. I think all of our research somehow, we tend to think about that it's academic and it's and it's sometimes really, really, really model driven.

• 6:53Speaker 01

  But it always has to be like super applicable and always has to really go into the product. And it also means that a lot of that is engineering. I think it's like, maybe like 50 50 would be a good way of describing, of describing that. I mean, like, performance is super important, compute

• 7:14Speaker 01

  is expensive and and also for training. And we're always like a step ahead, I would say, of the whole market. Like back then, when we came out in 2017, we had to start to build our own data centers because we we essentially couldn't get the GPU compute and and we had to build our own frameworks for like, how do we put the the training workloads onto the data center. So so a lot of that is kind of pioneering and and the same goes and that then just increases your engineering workload because you cannot take like the off the shelf product that's that's already out there on

• 7:55Speaker 01

  on the market.

• 7:56Lukas Biewald?

  Wow. So you were building out GPU data centers in like 2017?

• 8:01Speaker 01

  Yeah, yeah. Like I've liked really the kind of the first machines I've kind of wrecked myself personally. That's been, that's been pretty cool, actually.

• 8:10Lukas Biewald?

  Wow. Did you expect your compute cost to be so big when you started the company? Like, I would think it's, it must, it must be much more competing that you're buying than than you plan.

• 8:20Speaker 01

  I mean, like luckily, there was a good kind of correlation between the growth of the company and our revenue streams and the compute that we that we needed to build what we what we had to build. So we've been able to to finance that that pretty well. But but yeah, I mean, like, it's, it's been, it's been becoming larger and larger and especially with like the advent of and of the DGX generation at Nvidia and now with Blackwell, like, this is, this is substantial cost. But we also consider that being essential for us to to maintain an edge and and be able to to train those large scale models.

• 9:02Lukas Biewald?

  And you talk about proprietary data. So what does that feel like? Again, I think people always thought, OK, Google has this advantage because they're scraping the whole web. They must find a lot of kind of parallel corpora there. What sort of proprietary data do you have?

• 9:18Speaker 01

  Yeah, I think everybody can scrape the web. You can be better at this. You can be worse at this. We've been doing that for quite a while already. And being able to both find, be a parallel corpora bilingual that on the one hand, but also monolingual. I mean, that's important too, especially if you're thinking about languages where you cannot find that much bilingual data, then supplementing with monolingual becomes pretty important. It is an effort and you've got to know what you're doing. I think in 2017, that was even harder. Right now, you have a lot of already pre-cored corpora on the internet. It's a little bit easier.

• 10:02Speaker 01

  You can try to start and kick off with those. I think extracting the data maybe out of the kind of websites, et cetera, et cetera. It's just like a lot of engineering work, sometimes actually pretty fun algorithmic work even to do that efficiently. If you have like a huge website and want a really like a full, extremely large domain and want to match which sentence matches the other, it's computationally not that simple sometimes if you want to do that cheaply, but an exciting problem to solve, really.

• 10:39Lukas Biewald?

  How do you think about the quality of a translation? I think in the past, maybe it was easier, you know, and that like the translations are so bad that sometimes they'd be like incoherent or just wrong. But I guess like, it seems like translations have gotten, you know, in my experience, like pretty, pretty high quality. Like, what do you look at to know? What kind of separates, say, your translation from, you know, a competitor's translation? Or what are the metrics that your models are optimizing at this point?

• 11:09Speaker 01

  I think an important part is taking context into account. So quite often, if you really look nowadays at a sentence like without any context, like just just looking at one sentence, then even like a great translator, like human translator, or you or me, we cannot do a better job without really knowing what this is all about. You have to, you have to take it into account, what kind of document that is, maybe sometimes really like, what is this company about that is translating that in order to to get this one sentence perfect and and then and then with that, you just give it the model so much more power to do that. So I think that's, that's, that's, that's a truly important

• 11:53Speaker 01

  one. Keeping the models fresh, of course, like data changes, language changes, things, things develop and and you want to have the models be able to do that. And then, and then kind of this fine tuning of the model, how much you want to give him an M focus on accuracy versus on fluent writing. And those things are sometimes really contrary to themselves, like especially if you have languages that are really different from each other, there might be this, this clash of whether do you want to sound, make it sound nice, or do you want it to be really correct in a, in a way?

• 12:36Lukas Biewald?

  Interesting. And so do you, does every customer get kind of their own fine tuned model? Or how does that work?

• 12:42Speaker 01

  Yeah, like that, that won't be scalable. So we are, we are not, at least not in large scale training, models per customer. We're trying to find ways on how we can give the models the right context, how we can inject context specific or customer specific information for, for, for the particular use case and for this particular customer without having to retrain everything, basically. And with kind of the, the hundreds of thousands of, of customers that we have, that's, that's just like the only way of, of doing that. I, I think there's many companies, not only in the translation space, but in general in AI, who are, who are trying to train models per customer. I don't think that this is

• 13:26Speaker 01

  a particularly great way, unless you have like really very specialized situations in which kind of there is an ROI on the, on that big investment.

• 13:36Lukas Biewald?

  So how does it work? Like if one customer has an application where, you know, they want like a more, there may be a technical application where you really want like a, you know, like you're saying, an accurate translation and another customer like just wants the language to be fluid. Like, are there sort of like three models to choose from? Or how does somebody like tune like aspects like that that you mentioned?

• 13:57Speaker 01

  I think the models, we have to pre-tune them in order for them to be able to pick up what kind of language that is. And, and then there's kind of for, for the technical application, this customer, they're gonna maybe upload their terminology that they wanna have used in, in their, in their translations to our models so that it all is always consistent across the whole technical documentation base. That's not gonna be so important, for example, in the marketing case, in the, when, when you really want fluency, when, when kind of this, this, the craziness of the model or the creativity of that.

• 14:40Speaker 01

  And sometimes choosing something else out of the probability distribution is going to actually make, make for a, for a great translation. Whereas if you want it really to be consistent, you're gonna maybe just control that on your own.

• 14:57Lukas Biewald?

  Okay, so like, what happens when translation, like, good translation gets, gets really, you know, cheap and easy? Like, like, are you seeing like businesses operate in different ways? Like, once they, they start to have access to your technology?

• 15:14Speaker 01

  I think this, the whole kind of language industry and the whole language problem has changed so much over the last kind of eight years when we've been out in, in the market. That has been both driven by the availability of the technology and the ability to, to just, just throw in something into the translator and get an answer like so quickly. It's, it's sometimes even not about the cost, it's really sometimes about the speed in which you get those translations. And, and, and also the demand from the market, I think, has, has been growing. Customers demand to have customer support in their own language. They wanna see materials being localized when they wanna buy.

• 16:00Speaker 01

  It's, it's not such an easy market anymore if you're just speaking English as a company. So I think that has driven also a lot of our customers to embrace that, really. I think that one of the biggest changes was that, and we're gonna see that in AI in general, I think, is that some of the, some of those customers of translation, like even within, within a company, like be it a legal department or be it a marketing department, they started to really self-serve on those, on those solutions. It's not a centralized function in many companies anymore. They just go out to a provider like, like us. They, they start using our product on our own.

• 16:44Speaker 01

  They integrate it into their tools and do not have to rely on a external agency maybe, or somebody who'd be doing translations with, like in a traditional way. And that has changed kind of this whole access. And therefore also makes up for much more content, much more volume being translated in, in general.

• 17:11Lukas Biewald?

  Do you think that the quality of translation has gotten over a threshold where that's like less of a differentiator for customers? Or do you think most customers are still kind of hungry for even more high-quality translations?

• 17:25Speaker 01

  There's, there's, there's a lot of hunger for quality. I think depending on which quality level you are at, you're always unlocking new use cases for being tackled by a machine translation. Like whatever, whatever is enough for this like single one-to-one email that you're sending to your colleague that's sitting in another office in, let's say, I don't know, Taiwan.

• 17:52Speaker 01

  You don't care so much. I mean, it's, it's honestly, it's gonna be fine. If you're then thinking about translating a contract or translating your terms and conditions and, and putting them onto your website in 20 different languages, that matters a little bit more. And in a mistake, there might have like really legal consequences. So if you're able to do that automatically and you're able to, to simplify this whole workflow a bit more, that, that makes for, for a big difference. And then in, in a lot of workflows, There's still a human in there checking the translation, post-editing it, as we would call it.

• 18:32Speaker 01

  And the easier you can make this job, the less edits are necessary, the less changes are necessary. This really impacts the time needed for that process. And if you think that this person checking is a paralegal, there is like a really hefty hourly salary that is associated with this process. So there's really a big return on investment on any incremental quality improvement that you can do.

• 19:07Lukas Biewald?

  I guess one of the things that kind of came up when I was like, you know, just researching your company, kind of looking around for it, was a lot of, you know, human translators talking about your company and sort of worrying like, is this going to make me obsolete? Like, it does sort of seem like we're on that trajectory, doesn't it? Like, do you think there will be human translators 10 years from now?

• 19:29Speaker 01

  I think that they will definitely be there. I think the amount and all the content of translations that are going to be done by humans only is going to be severely reduced. So I think a lot of the kind of boilerplate and boring work of translation, that's going to be all done by AI. Like, a large part of that is already done right now, and the future even more so. I think humans are going to be still incredibly important in this process to kind of guarantee, especially in those like high-compliance use cases. If you think about life sciences companies and financial institutions,

• 20:11Speaker 01

  like, there's really the need even now to have multiple human translators on a single piece of text. And it's going to definitely, definitely continue. I think we also have to be realistic then on the simplest cases of translation in the most common languages and in cases where quality doesn't matter so much. I think AI is on itself is going to be doing an amazing job by itself.

• 20:39Lukas Biewald?

  Do you do you keep humans in the loop still for some of your translation applications?

• 20:44Speaker 01

  Not for not for like production, not for inference, that that would just be not scalable. I mean, we're working with with thousands of translators and humans to to train their models and to kind of give us the feedback and the quality assurance and and all of that. But you can't employ that, at least not in those volumes that that we are translating in any way in this

• 21:09Speaker 01

  and during inference time.

• 21:11Lukas Biewald?

  Where do humans still outperform the models? Because you're talking about like, like, say, like, legal use case or something. And I would sort of imagine like a human might also, you know, make a typo that that a model might make and like, you know, the, at least from what I see from translation model performance, it seems so spectacular that I I wonder like, would a human do do a better job than a translation model? Of course, I'm not like, you know, doing this all the time for my jobs and maybe maybe I'm way off on that, but

• 21:41Lukas Biewald?

  I don't know, like, my my impression is it's pretty close or maybe even the models might be more reliable in in some cases. Like, what are the cases where the model really still needs like a human to to get it to that that level of of quality, not just like for compliance, but for actually like making the translation somehow

• 21:57Speaker 01

  Yeah.

• 21:57Lukas Biewald?

  work for these guys?

• 21:59Speaker 01

  I I I do think that models are like definitely more reliable and more accurate in a sense like that. They're not gonna be doing those those those mistakes that we as humans from time to time do just because our brain slips. So that is that is an advantage for the models and that's gonna that's going to be an advantage even more so in in the future, I think. I think the models still do not understand the world as we do. And there is there is difference there. I mean, with with with all of those great reasoning models and with also the L L M's that we're using for language, we see they they they kind of get the world just because of all of the text that they've seen. But this knowledge, this understanding is not as deep as as as

• 22:45Speaker 01

  with with us humans. And therefore, sometimes in like those very tricky situations, they just cannot distinguish on what was meant there, like, what was the intention of that that particular text? And which is why I said context helps because it gives you more of this. But even sometimes that's that's not enough. And then You're sometimes running into those edge cases. You're running into like half-broken sentences, which due to, I don't know, some, some kind of text parsing are slightly weirder. You're looking into very short texts because they've been written for an app and the user face of this. And, and the models sometimes like get confused by that, honestly.

• 23:28Speaker 01

  They haven't seen that in the training material so much, or they've just seen that very, very rarely. And they, they cannot cope with this, with this added complexity then.

• 23:40Lukas Biewald?

  That makes sense. What about, um, and I'm just, you know, I think I'm in the, I have the experience of talking to a lot of like enterprises about L L M's in general. And there's always this sort of fear, like stories they say around like hallucination. Is there like a parallel hallucination issue in translation?

• 23:57Speaker 01

  Yeah, it's been coming up. It's not like it's, uh, it's not there. I mean, like the models are encouraged to be a little bit creative and you have to give them the freedom to, to just kind of write on their own. And, and sometimes if they don't know what they should be doing, they start making stuff up. So you have to employ control over that. I think within translation, it's a little bit easier because you can always cross-check and go back to the text, to the original text and, and then kind of even post factum, maybe sometimes do, do, um, do some evaluation of whether this has gone astray or not.

• 24:36Speaker 01

  And the, the kind of creativity space that you're giving those models is then slightly lower than in a general purpose L L M, which is just generating text. But you have to be wary of that. And we in general have seen that specialized models, and that's kind of one of the differences, hallucinate less than general purpose G N A I models when they're being employed for translation.

• 25:01Lukas Biewald?

  Then I guess you keep talking about text translation, but you also offer speech translation, right? Is speech just like a smaller market? Or what, I mean, is your emphasis on text amount?

• 25:13Speaker 01

  It's it's just a newer market. I'm I'm super excited about speech, actually, because it makes such a such a big difference. So that's something that we just put out on the market last year. I think the tech just wasn't there yet for it to be productized in such a good way to be able to like users being just happy with with the output. And and it has just come to to that level where where it's really practically applicable. I think we've gotten accustomed, as you say, to great text translation over the years by now. So it's not making that much of an of an impression to us. I think speech translation is is just like this new amazing thing

• 25:58Speaker 01

  that's that has that has come up. I was on my own in like customer conversations in in Asia, in Japan, where we'd usually have my sales team help translate a little bit, or we would have even an interpreter in the room. And it's it's always a little bit cumbersome. You don't get really fully what's happening in the room, or you get it with like a ton of delay. And with now with speech translation technology, you're you're like fully embedded, fully immersed into the conversation. It's it's not as good as if you really speak that language, of course. But it's but it's pretty damn near, I have to say.

• 26:41Lukas Biewald?

  Have there been kind of new challenges that have come up with the the speech part?

• 26:47Speaker 01

  I mean, yeah, you're sure. You've got the speech recognition part, which is which is super important. Then the the language we're speaking in is just different than what than how we are writing. And it's much

• 27:08Speaker 01

  It's just not as clean. We don't have so much time to think about what we're saying comparing to when we write something. And therefore, it tends to be just like a little bit garbled and you don't know where the sentence starts, you don't know where the sentence ends. So, speech recognition can kind of solve for parts of that because those models are really trained to to package that stream of of words into into some coherent sentences. But still, I think the model has to cope with more. And the quality of the source input is also lower just because speech recognition makes its own mistakes.

• 27:49Speaker 01

  And then the the translation model has to somehow figure out on what should I do then? Like, is this, does that word even really match here? Or should I maybe substitute with something that's just more probable at this point?

• 28:01Lukas Biewald?

  And do you try to like keep, like preserve like the the rhythm of the speech and the tone of the speech? Like, does that somehow carry through?

• 28:10Speaker 01

  Or? Not yet in that that that big, it's like not not a big focus. I think right now, the main focus is really lying on latency and just making the translation as real time as possible. Like, we know this is this is incredibly important for the for the for the user experience. The quicker you get the translation, the better you stay in the flow, the more you can match the mix of the speaker that you're talking to to what you to what you see in terms of translation. So, the just the conversation becomes much better then. And and that's that's like one of the most important parts.

• 28:50Speaker 01

  And then just the pure translation quality, being able to catch up on all of those company specific terminology, making sure that you do not do not miss on the proper name of the CEO of that company. Like, like, all of those things are super important to make to to make a good impression on the users.

• 29:14Lukas Biewald?

  Do you think about like the, you know, the impact that you'll have on the world when like, you know, speech translation is like, you know, very, very easy to turn on like it, it seems like,

• 29:27Lukas Biewald?

  it seems like it'll really change the way businesses work, doesn't it?

• 29:30Speaker 01

  I'm, I'm very much looking forward to that, honestly. I think, I think this way we can really get all of this great cultural diversity that is out there in the world and like the different working styles and the, the strengths of different countries and, and we can really mix and match that through kind of our global supply chains and, and the way that we are, that we are working. And those of us who really speak well English have been incredibly privileged in this, in this international world, I would say. And, and we have the ability to now have, have many, many

• 30:10Speaker 01

  more people join this community so well. And then, then maybe even in the process of participating, really learn that language and, and become fluent by themselves. But at the, in the, in the first moment, like giving them the confidence that they can, they can speak up in that meeting, they can participate when they have an idea, which, which quite often, honestly, does not occur if you, if you're not really proficient in that language.

• 30:36Lukas Biewald?

  Totally. And I mean, even like, you know, like a world without language bears, where you call your, you know, friends in a different, you know, language seems pretty amazing, doesn't it? I mean,

• 30:47Speaker 01

  Yeah, I mean, like for me, there's like a limit to that at some, at some level. I think, I think I would really still want to have friends and speak to people whom like, I, I really understand the language on my own. I think that also brings in like much more, like it, it brings us much nearer from a cultural perspective, because the language is usually kind of even tailored to the cultural history of a country. And there's, there's so much embedded in that. So, so I think, I think, I think there's like, there is realistically a limit to what the AI can do here, especially in all of those private situations. And like, I cannot imagine like living with a partner and like speaking through a phone with them like for my whole life. That

• 31:32Speaker 01

  just, that just doesn't, doesn't work. But like for all of those business situations, I think it's just like that, that's gonna be just purely great.

• 31:41Lukas Biewald?

  And I guess, what's the state of the art of handling like less prevalent language or less less common languages? Like, like, how much data do you need to to collect to make like a usable translation model, either in in speech or text?

• 31:56Speaker 01

  I mean, the question is, yeah, like, usable for which purpose? But yeah, I mean, like, there is a there is there is a gradient in how good translation quality is depending on the different language pairs. And it's it's availability of data on the one side, and then it's also obviously the the amount of work that the companies like us, or even what happens at academia, or with our competitors, can put into these particular language pairs. It's just like a question of, of, once again, business return on investment, and and we're trying to make sure that we cover the languages best that our customers need and and that they're requesting from us.

• 32:38Speaker 01

  So definitely, there is like this tier one of languages that that are the biggest global languages, then there is a second tier of slightly smaller languages, where there's already quite a lot of material, where you get really good results. Like, Poland would be, and Polish would be a good example, which is like where I was born, and and and it's it's a decently large language with a good amount of training material. But then if you go into really, really smaller languages, it like, that's gonna be much harder, and that's gonna take more time to really get those on the board at the same at the same quality level. I think I think also probably we're gonna have to become smarter in how we train models

• 33:22Speaker 01

  and not require so much data for them in order to get those languages to the level that we're that we're expecting to to get to to get really fluent.

• 33:34Lukas Biewald?

  Do you build specialized models for every language pair? Or is it all kind of combined into one gigantic model?

• 33:41Speaker 01

  We've been building a lot of separate models, actually. And in the last time, we've been starting to consolidate that at least into into into chunks of models and and and

• 33:55Speaker 01

  or like models that can handle a group of languages. It's also a little bit different on depending on when you're thinking about text or or voice. If it's if it's speech translation, that then once again, like latency comes into play. Super small model sizes or like smaller model sizes are important. And then they might not be able to cope with all of the different

• 34:20Speaker 01

  all of the different languages at the same time. I just the parameter count is not enough.

• 34:25Lukas Biewald?

  I see. And I guess you could you could do different tokenization strategies for different different languages, probably.

• 34:32Speaker 01

  Oh, yeah, totally. You can do that if if that makes sense, yeah, you can do that.

• 34:35Lukas Biewald?

  But it was I was kind of amazed by. I don't know if you saw that Anthropic had that paper they put out where they were kind of showing how it seemed like it's similar sets of neurons like fire in their in their network with you know, words to mean the same thing in different languages. Like, I kind of always wondered if it worked like that, but it's kind of amazing to to see that. It sort of made me think like maybe these, you know, more combined language models would would start to work better as these networks get more powerful.

• 35:03Speaker 01

  Yeah, yeah, I mean, they work better. I mean, and and on the other side, it's also like it's it's it's much easier on the engineering and deployment size if you don't have like hundreds of models to cope with and like version them and train them independently. So it's just like easier for us. But yeah, groups of languages, especially if they're similar, that that helps a lot. And then if you have like a group of similar languages that don't have enough data, they they fuel each other and and make it easier. And yeah, like over the time, we've been looking at many features of those models and how they map what happens in those models with some of those linguistic nuances.

• 35:44Speaker 01

  And our understanding of language and and sometimes these are really kind of really funny

• 35:51Speaker 01

  things that you can find and how how certain things match to each other and how you find clusters of meanings and how they all really sits sits sits near to each other. But then it's so at the end, it's so many dimensions that if you wanna kind of try to sum it up and and understand really what happens from end to end that at some point in time, it just gets to just far too complicated.

• 36:13Lukas Biewald?

  And Chris, you're like one of the few companies that has really deployed, you know, deep learning giant, Gen AI models, I guess they call them now at at scale. Can you can you talk about some of the like engineering or like operational challenges of of making this work? Like, you know, what surprised you as you as you scaled up the the size and and volume of inference in these models?

• 36:37Speaker 01

  Yeah, I mean, like for us, it was like pretty much everything, just A, we've started so early on that. And as I said, like, we had, we had to build a lot of the of the of the stack for that. Even things like, how do you distribute requests that are coming in from the users to the different GPUs that you have available? Like, you always have to kind of strike the balance on how how big batch sizes do you make? You you you wanna utilize your GPUs well. But also, you wanna maintain like low latency for for for your users. So you have to, you have to make sure that you have the tech which which kind of groups those

• 37:20Speaker 01

  understands what those requests are, sends them off to to GPUs. Like now, like now in 2025, there's like more common technology and with and it's it's much simpler to do and like back in the years, that was that was definitely trickier. If you have a wide range of models, like depending on language pairs, depending on the load that you're getting on the system for for different language pairs, you might wanna like spin up new models, like spin up more models for Japanese, because it's the kind of the Time zone for for Japanese language and then and spin down other models. So we have to build the tech for like scheduling all of that and reacting to load changes. So

• 38:06Speaker 01

  I think GPU compute is really different than CPU compute. And there's been quite a few funny algorithmic challenges to really solve. And that's too from an from an from an engineering perspective.

• 38:18Lukas Biewald?

  Are you one of those companies that's like totally like compute constrained? Like if you had more GPUs, you could generate more revenue immediately or?

• 38:27Speaker 01

  I I think like we we don't have a problem of getting the GPUs. Like, I mean, the the kind of markets. We've gone through a few moments in in time when like just getting the GPUs, even if you had an infinite money was super hard. I think we're not at this point right now. It's it's like the the supply works. And of course, more compute would be great. But I think like we would probably also need more researchers, more more brains, basically, to also utilize that at some at some point. It's it's not only about raw computing power, though, that's that's also important, of course.

• 39:09Lukas Biewald?

  Do you run on all Nvidia GPUs? Or have you experimented with some of the more exotic GPUs?

• 39:16Speaker 01

  It's actually all Nvidia. We we were obviously looking at the other ones and then kind of testing, benchmarking. Migration is hard, though. Like, I mean, that's that's like, everybody knows that. That's that's not a big secret. And and speed matters in this industry. So really, doing big migrations is not is not that easy. And especially as we're running our own individual architectures on the models, it's also not that easy to just go to an off the shelf provider of inference and just stick your model in there. There's gonna be just so much more migration overhead

• 39:59Speaker 01

  for that. So yeah, for for that, for the time being, sticking to Nvidia, but also like really looking at the alternatives all of the time. And and we're seeing the market catching up, I would say. There's there's there's a lot of new fascinating stuff coming up there.

• 40:12Lukas Biewald?

  Do you think it's going to change like what you need to do to stay ahead of the market over time? Like it seems like if, you know, if the LMS get more and more and more, you know, kind of powerful, could general purpose LMS start to like eat into your translation market? Especially the sort of like simpler translations that are as mission critical? Like, how do you think about that?

• 40:33Speaker 01

  Yeah, I think, I think, I think we have to change there. I think we have to understand much better what the translation is being used for. It's, it's less so about kind of translating the sentence from A to B, but understanding like what is the full workflow in an enterprise with this translation? Like, is there going to be somebody reviewing that? Is that actually a second version of a translation which has been done already earlier? And there was, that first one has been done by AI, but then there was like a revision by a human which introduced some changes. And then it makes sense to like feed all of that input into, into the model in order to, to, to enable it to be even,

• 41:13Speaker 01

  even more accurate. So it's becoming more about kind of the, the enterprise workflow and how you can embed the AI into that, how you can do actually like more, more deep, deeper product research, but also fueled by AI in order to, to solve the higher order problem and not just the simplistic translation case. And, and then it's frankly not that trivial because you're, we're coming from a super horizontal product as we are with just translation being embedded in so many different use cases. And, and you have to be smart about picking and choosing the most important ones and where you can also add, add much more value to those.

• 41:55Lukas Biewald?

  Interesting. Have you started to offer services like that?

• 41:59Speaker 01

  I mean, that's just kind of part of our ongoing product discovery, like understanding what our customers are, are using translation for, and then the kind of embedding functionalities into the models that, that, that drive those and then exposing them in, in the proper, proper way. So it's just like part of our normal product development cycle, I'd say.

• 42:18Lukas Biewald?

  Cool. All right. Well, thank you very much. That's all the questions I have. I appreciate your time.

• 42:23Speaker 01

  Lucas, it's been perfect. Thank you very much.

• 42:25Lukas Biewald?

  Thank you very much.

• 42:28Lukas Biewald?

  Thanks so much for listening to this episode of Grading Descent. Please stay tuned for future episodes.