860: Module Federation Microfrontends with ByteDance’s Zack Jackson

Wes and Scott talk with Zack Jackson about Module Federation, and the innovative ways it streamlines collaboration across large teams and complex applications. Zack also dives into simplifying microfrontends, optimizing builds with Rspack, and his work at ByteDance. Show Notes 00:00 Welcome to Syntax! 00:48 Zach background and role at ByteDance 03:10 ByteDance’s web and native apps How ByteDance Became the World’s Most Valuable Startup Lynx ByteDance GitHub Coze Lark MarsCode 06:48 What is Module Federation? Zephyr Cloud 15:49 Evolution from V1 to V2 of Module Federation 24:47 When to consider Module Federation 29:46 How would you publish a utils package in Module Federation? 32:59 How would you do a major upgrade? 36:36 Brought to you by Sentry.io 37:01 Who owns the update process? 39:40 Handling multiple frameworks 42:10 ModernJS and Meta frameworks ModernJS Rspress 47:20 Does ByteDance have its own JavaScript runtime? NAPI 50:02 Why Zack built Rspack Rspack Rsbuild 58:15 The future of Rspack and custom bundlers Unpack 01:04:24 Module Federation’s major features 01:07:29 Sick Picks & Shameless Plugs Shameless Plugs Zack: Zephyr Cloud Midscene.js Hit us up on Socials! Syntax: X Instagram Tiktok LinkedIn Threads Wes: X Instagram Tiktok LinkedIn Threads Scott: X Instagram Tiktok LinkedIn Threads Randy: X Instagram YouTube Threads

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Wes Bos?

  Welcome to Syntax. Today, we have Zach Jackson on the show to talk to us about module federation. I just saw Zach like a week and a half ago at a conference. He's the author of RS-PAC, which is the Rust-based bundler, has the whole same API as WebPAC. It's really neat. And just that one talk, which was 20 minutes long, a super short talk, blew my mind in terms of like, all of the stuff he's working on. He works at ByteDance and was explaining ByteDance's company TikTok and CapCut and all kinds of other neat products. But just explaining the scale of the problems that he's solving. And it just, I just walked away being like, I want to talk to this guy for like nine hours. But welcome Zach. Thanks for coming on.

• 0:46Speaker 01

  Yeah, thank you for having me.

• 0:48Wes Bos?

  Give us an introduction of who you are and what you do. I gave a pretty poor one there, but let's hear it from you.

• 0:55Speaker 01

  Yeah, sure. So I'm an infrastructure architect at ByteDance. So I work on the web infra team there. I've been in open source space for, you know, past couple of years. So I'm probably more well-known from that standpoint. So I'm a maintainer of WebPAC. I'm part of the core team for RS-PAC. And I'm the creator of module federation. So that's the big highlights.

• 1:22Wes Bos?

  Awesome. And that's specifically why we wanted to have you on today to talk about, which is what module federation is, because it's one of those words that I've been hearing, going around. And then anytime I talk to somebody at a conference that is from a very large company, I always find that very interesting to like, how do you build stuff relatively fast when you have seven million teams, you know, and all these different builds? And one little thing that you said during your talk was that people are on-call for the bundler. Imagine being on-call for WebPAC? Like, you have to be the expert on that? And I was just like, man, that's, because it's true. Like, what happens if the build is breaking? You can't merge something. That's, that's pretty wild, right?

• 2:07Speaker 01

  It's kind of similar like to CI, you know, like you always have like your DevOps. And when CI is down, it's okay. Well, there's only like a handful of people in the company who generally deal with it. But when you get like when things scale up enough, the, you know, your actual compilation process becomes equally as crippling, I suppose, because not everybody knows how to fix those. They're also another niche area. And then on top of that, the more exotic the applications are, the more like intricate the build process is. So, you know, if you're like trying to bundle, say, a native app and you're like, oh, hey, why is, you know, this one thing not working under this circumstance? Your general dev doesn't know if it's not like ESM or something. Usually devs aren't like,

• 2:48Speaker 01

  oh, yeah, I know how to, you know, handle like a multi-dimensional output build and, you know, what's going on between the threads or other random things like that.

• 2:56Scott Tolinski?

  Oh, yeah.

• 2:57Speaker 01

  So, yeah, it does end up actually being a large portion of the on-call is just making sure that builds are functioning correctly. And that was one of the reasons why we ended up building RSPAC was to just reduce that pressure.

• 3:10Scott Tolinski?

  Wow. So, how much web does ByteDance do? Because I know so much of my experience with the products is all native apps is how much of it is web?

• 3:20Speaker 01

  All of it, even the native apps.

• 3:22Scott Tolinski?

  Really?

• 3:23Speaker 01

  So, the native apps that we have, so we have a project called Links. We'll be open sourcing it sometime early next year. But Links is essentially our in-house React Native. So, all the apps are Links apps. Everything is a Links app. It's all backed off of the same stack. So, there's a really good article for like some background info on how the company operates. If you just Google, "How ByteDance Became the World's Most Valuable Startup," it'll be the Harvard Business Review article. Should be the first result. But it goes into really good detail on like how the company is structured. And so one of the big things is we call it SSP, which is essentially Shared Services Platform. And so, the whole idea behind it is really you have like very tactical specialized teams

• 4:07Speaker 01

  who focus on only one slice of the overall pie. So, you'll have like an algorithm team or a compiler team or a Kernel team and so on and so forth. And then when product needs something, you can just go get the algorithm team, or you can go get the kernel team and they'll write, you know, modify the runtime that all this stuff runs on. So everything's like really compartmentalized, but it allows for you to have this really kind of like high density skill levels. And yeah, so that's just kind of how like the the company is structured, but it gives us a lot of flexibility in terms of like what we need the products to do. And then on the other end, because of SSP, the whole idea is like, how do you have a unified architecture?

• 4:47Speaker 01

  So what we're maintaining here can be adapted and supported in a hundred different ways. But at the end of the day, we have a very consistent stack that's just very porous. But it's not like one giant, you know, one size fits all. It's more like, how do you create the sum of parts that are designed to work independently, but when combined work very effectively together? And so that's kind of the whole idea behind it. We build all these things as standalone parts that happen to lock into place into a bigger product when needed.

• 5:17Wes Bos?

  Wow. So, because like if people are thinking, oh, what else is there? There's obviously TikTok, the the mobile app, the Android app, but then there's, there's the website. And then there's also the CapCut, which is like a whole blown video editor.

• 5:30Speaker 01

  There's Doyin, which is the Chinese version of TikTok. It's the the bigger one. There is Pico, the VR gaming headset.

• 5:41Wes Bos?

  Yeah.

• 5:41Speaker 01

  P I C O. So we own Pico. There's Mars Code, which is a code of like cursor.

• 5:48Wes Bos?

  What?

• 5:49Speaker 01

  Yep. There is Coze, C O Z E dot com, which is like a AI bot builder, workflow builder tool or something.

• 6:00Wes Bos?

  Man.

• 6:01Speaker 01

  I'm trying to think of the other things. CapCut, Lemonade, Lark, L A R K. It's like Microsoft Teams, but built better and does more.

• 6:11Speaker 01

  So that's what everything, everything that we do, other than the IDE, everything that I do at work is inside of Lark. Calendars, booking flights, looking at payroll, requesting time off, writing documentation, Kanban boards, for the extent that we have those type of things, chat, email, like, everything is in Lark. The only thing that isn't in Lark is, is the browser and the IDE. Other than that, you basically never leave, never leave it. So there's a ton of products and these are just like a handful that I know, but I know that we have over a thousand.

• 6:44Wes Bos?

  Wow, that's, that's unreal that there's that much. So, talking about module federation, obviously you need to be able to share stuff between it. So, what is module federation?

• 6:55Speaker 01

  So, module federation is essentially the ability to dynamically link parts together at runtime in some kind of a stable way. So, there's two versions of federation that came out. There was V1, which was the one that I had PR'd into Webpack back in 2019. So, from that perspective, the whole idea was, you know, it kind of started with an obsession probably in like 2017 of, because I'd started to work in micro frontends already, but it was the LOSA, LOSA architecture. So, lots of small applications, which essentially meant you mount a bunch of apps onto divs in the browser and each app is kind of your old school, traditional micro frontend where, you know, it's a

• 7:40Speaker 01

  a whole bunch of little parts that each have a div, they mount onto the div and you would, you know, use like externals or something. So, everybody needs React, then, you know, you'd have like the vendor choke that sits at the top of the page with like React, Redux, LowDash, a few other things. And then all the parts would just hook into that. And that's kind of where I started to see the pain points, because, so this was when I was working at Fiverr and we had like an old Rails app that we were modernizing. And so how we did it was micro frontends. So, the whole idea was the Rails app would kind of do a post request of its data, would post it to a render layer and the render layer would return the head, the foot tags, and then return the markup and then Rails would render the markup inside of its kind of shell. So, the, the challenge that we started to see with that was- Okay, you can use externals, but you have to manually configure it.

• 8:26Speaker 01

  So if you don't want everybody to duplicate, say like uh Lodaash, you would have to externalize it. But then if you do, everybody has to be on the same version of Lodaash. And what you're developing with locally usually isn't the same versions of the dependencies you actually have in the vendor bundle because that's shipped from like a different build. The second kind of challenge that you would see as well is with externals, it's synchronous. So that means you have to have a render blocking tag at the top of the page before any of the other apps run. So whether or not I use Lodaash doesn't matter. I have to download it every time. And every piece of shared code has to be render blocking and loaded regardless of if the application is actually using it.

• 9:06Speaker 01

  So this is where I kind of started to see the pain point. It was really difficult to manage and orchestrate. And how we were doing the render layer as well was uh, everything was an NPM package. And then you would publish the NPM package and something like renovate bot would pick it up and automatically open a PR to like the render layer. And you'd have to wait for the bot to open the PR and you'd merge it. So now when you send a post request with the package name, it knows which thing to render and can send back the, you know, the correct assets for it. The problem that we ran into with uh, that I had run into with this approach was one, it was really slow. Like everything, what I refer to it as like the double, the double deploy.

• 9:46Speaker 01

  So let's take an example of you've got a, an NPM package. Let's say you have like button. A really small example, but let's just say it's button. And then you have a header. And now you want to update button. And you need to update it inside of header. So first you have to pull button, develop on button, push button, release a version for it. Then you have to go pull header, install a button into header, push header. And then finally, you can actually see it pop up in the render layer where you have to do another installed deploy. So you're looking at like three like kind of pings back and forth before you can actually get the product into production. So you end up losing a ton of time just trying to push things through the pipeline of like a normal package.

• 10:31Speaker 01

  Based or I refer to it as kind of like the install based delivery pipeline. Most of your time is spent like running around the org. Uh, but like at other bigger orgs, same kind of issue. Oh, I need to, I want to, let's say it's just header and you want to roll out header to say 10 or 11 different parts of the org. Maybe you've got like, you know, community content, you have e-commerce and you have checkout. So then just getting that synchronized is that'll take probably six hours. If it's just you doing it to go to each team, nag them, wait on their queues, run it through the pipeline, which takes, you know, 30, 40 minutes to actually get it into environment one. And then after that, you got to run it to, you know, preview environment. And then finally to get it to production and you have to do that three times over.

• 11:14Speaker 01

  Plus, everybody has a merge queue because there's other things in the product that are also trying to move in.

• 11:19Wes Bos?

  Oh, man, that probably just kills your, and like enthusiasm for.

• 11:23Speaker 01

  Yeah, like, you know, you can, some days I could go to work and I would, all I would do is just stare at CI and talk to people, wouldn't actually do any work. It's 10 minutes, fix the typo. And then it's been the rest of the day trying to like scrape this thing through the company and actually get it into production. So it's just a huge productivity killer. So, so back when at Fiverr, the thing that I was, you know, not happy with was the, one, the latency that it takes to actually get things out. And then secondly, the, the bloat. It's like, okay, well, I'm downloading so much because the externals doesn't work very well. Or I have to do this giant render blocking tag with, you know, 300 kilobytes when I'm only using maybe 50 kilobytes of what's in there. And so the idea had come to me, well, why can't we just dynamically import chunks from a different build?

• 12:08Speaker 01

  So if I already have React, why can't I just get the code that I don't already have? Like the actual feature code for header. I have other vendors already because I'm a React app as well. But if I don't have low dash and they need low dash, why can't they just supply me the things that I don't already have? And so that's essentially the concept of federation is there's two parts to it. The first half is the compiler part. And what we do in the compilation step is essentially we say, all right, what are the things that you want to share? The things that if you say, okay, I want to share React, I want to share Redux, low dash, whatever. We will then split your bundle. So that there's a React chunk and a load dash chunk and those are separate from like your main bundle. And then there's exposing and there's sharing. So sharing is cool, semver safe, NPM packages.

• 12:53Speaker 01

  You can say, hey, do I want this to act as a singleton? Uh, what version of it do I specify that I need similar to how NPM install would work with your package JSON? And then we shard the bundle accordingly. And then there's exposing, which is similar to like package JSON exports field. So where you can say, all right, and I want to expose header to you. So now only what's in the exposure, like the exports field can I import on the other end? So the sharing's all like magic under the hood and then I could say, you know, like, you know, checkout slash cart. And then they would have exposed their cart component as whatever they want to call it. And then when we're exposing, the same thing happens. We shard the bundle so that the cart chunk or that cart module and whatever things it depends on is separated from the rest of the bundle.

• 13:36Speaker 01

  So now I'm able to access those chunks independently. Then the runtime aspect is, is essentially we output another entry point that I just kind of call the remote entry. And that's basically just a special entry point that has two, that exports an object with get and a net. Get is how you would say, cool, give me something that I'm exposing. So that's accessing the export field. And a net is where we kind of do like a circular, like kind of waterfall series where we go, okay, when I initialize it, I'm going to pass everything that I have to share. I'm going to pass it down to this remote entry, initialize it. So it's aware, hey, here's everything that's already available in the system to share. And then it's going to finish its initialization, assign any shared objects that it has on to it. And basically it goes through everybody.

• 14:22Speaker 01

  So everybody waits for the whole system to initialize. Everybody's aware of all the sharing. And in that process, that's when we say, okay, they need semver two for this. They need semver one for that. Is it a singleton? Okay, then we need to find who has the best version for the requirements of everybody connected. And it kind of does all the negotiating for semver whatever at runtime as the system's initializing. And then finally, once everybody's agreed on the versions, we seal the object, which essentially means we do something similar to kind of. Like object freeze.

• 14:52Wes Bos?

  Yeah.

• 14:53Speaker 01

  Where we say, okay, now you can't mutate the version of React you're going to use from now on, because it's now loaded in memory and somebody's utilizing it. So if later on, a new, a better version of React comes in, the system can't decide to use that one later because we already have this version of React in flight. So it's kind of like initializing and then after that, we start getting the components that we want. And then, Cal, you receive that on the consuming end is it's essentially like a combo of externals plugin and some special module factory stuff we do where we say, hey, the external is the remote entry file. And then there's a special factory that'll say, hey, call the get method on there. So when it sees the external, it goes, okay, get the remote entry, do the initialization, and then it goes over to like this virtual module

• 15:38Speaker 01

  that we create on the consumer side when you're importing these non-existent modules. And that one says, okay, after the external has initialized, then call the get method on it and get the module I want and bring it back to me.

• 15:49Wes Bos?

  Oh, man. Man. And so that whole thing where you're talking about figuring out what versions are needed, that's not part of a build. Like you don't have to rebuild the entire website.

• 16:04Speaker 01

  It's all at runtime.

• 16:05Wes Bos?

  One tiny little thing, it's all at runtime. That's not happening like on request, right? It's happening, how often does that happen?

• 16:13Speaker 01

  It happens like in the browsers. As soon as you pull the app up, like before the app like starts rendering the app, it's actually busy negotiating with, because we don't know who's connected to the page at that point. Because there's no like server backend, there's no static point. So the remotes can all be dynamic. So imagine if you have like a A B tests or a dashboard that the user configures. So there's no way for me to say, oh, you're going to be using modules A, B, C, and D. So I can't pre-negotiate that, at least in V1, we couldn't pre-negotiate that. So it was when it gets to the browser, once we start pulling all of them in, they then seal themselves, negotiate who's going to use what, and then the app starts running. And the whole process there takes like 45 milliseconds. So it sounds crazy, like a lot of things are happening, but it's extremely optimized.

• 16:58Speaker 01

  Now, then we brought out V2 and so. V2 was essentially like when I had built V1, the whole idea was, you know, I want to just load chunks from another build that I don't have. And it was very much, I want to solve my problem and I didn't really think about. Cause back then micro front ends was kind of iframes or just mounting a bunch of apps. So there was not really a good baseline for what's needed. And the stuff didn't really scale. So it was really just, okay, I don't like the problems that I see and I want them to go away. So I just selfishly built the solution that worked for me, which obviously like, you know, a couple of years on. Didn't exactly fit every enterprise need. And also the tech grew a lot more than I had expected. I really just expected me to be able to use it and maybe one or two other companies.

• 17:43Speaker 01

  And I didn't expect about 45% of the Fortune 50 companies to be running on it. So obviously like there was some design oversights back in the day that we ended up correcting with V2. And so, so initially it was, I just want the chunks from elsewhere. What we had found out though, is that's not really the main, their major issue. It's not getting the scripts here and you just want to dynamic import things from another build. Especially with the introduction of something like, you know, import maps. It, you know, it was kind of, I'd always get the question, well, what's the big difference between say import maps and federation? And I'm like, well, with V1, you know, starting to get the edge, I'm like, oh, it's getting a little tight, you know? They're getting a little similar. Federation obviously had the, the one thing going for it that it works in any environment.

• 18:26Speaker 01

  So this isn't just browser, this works in server, this works on native, this works in embedded systems. You know, it works anywhere that you can evaluate JavaScript. So, you know, it was a little bit more flexible, but still, you know, to, to the, you know, normal user, that's not a big enough wow factor necessarily to say, well, why not just use import maps? So V2 kind of came out to address this because what I realized is it's, it's not really, I want to get chunks from somewhere else. Cause you know, getting the script is actually the easy part. The thing that I, that we actually want is essentially like express on import. So I want middleware, not just like a map that I can statically say, hey, A is A and B is B. But I need something that's like context based. So if button is being imported by who's the parrot?

• 19:11Speaker 01

  So if headers loading button, maybe it has different rules that need to be applied. Or what happens if it fails, then I need some kind of error callback so that isn't in the user's app code where they have to try catch it. Cause what if they're using import from or require or an uncatchable import? I actually need something very similar to like express with a request response flow where I could redirect the request or I could intercept it and I can modify it. Um, and so that's a lot what V2 was around is like, okay, well, what we thought we needed was just the ability to load chunks from somewhere else. And really what we need is something similar to like a express like middleware system on actual dependency and module loading itself.

• 19:52Speaker 01

  Now, speaking of V2, this is also where it's not just in the browser always like you were saying on every request. So with V2, we introduced this manifest protocol, which is essentially like a JSON graph that, that helps describe what each part has and what they can provide. It's, it's like a JSON version of the JavaScript entry point, a representation of that. So what that means is I don't have to evaluate this. I don't have to initialize the system to figure out what it's going to do. Since now federation is just an NPM package, we have all the algorithms that do all the share negotiations and that figure everything out. Uh, figure everything out at runtime. We now have that just as an NPM package. So what we do at ByteDance is we actually have, um,

• 20:35Speaker 01

  a server where when, you know, we, when we deploy everything, we have something called V-MOg. So V-MOg is a basically the in-house version of a project called Zephyr Cloud that I, that I'm part of. And they're both actually come from the same source, which was my original project called Medusa. And so they were, uh, when I first started working at ByteDance, I went and like logged into our CI and it was like, uh, logged into our like, uh, our Git backend. And it was like, oh, hey, welcome back. And I'm like, what do you mean, welcome back? This is my first day here. And I go and I like check my Git history and I had commits going back like three years. And it turned out that like, cause some, like my original Medusa project and a lot of my federation work, they had just pushed the whole history into their repo. So technically I had been contributing there for a long time.

• 21:19Wes Bos?

  Way before you even got hired. That's hilarious. It doesn't welcome back.

• 21:24Speaker 01

  So it was pretty funny and I also like saw things in vMach and I'm like, wait, I recognize the name of that function and I go and plug into it and I'm like, I remember writing that. Like that's my typo in that function name. So it was very funny to go and see it, but, but effectively like if you want to get a better understanding of like what vMach is, Zephyr is like the outside version that's very similar. And again, they both share the same kind of core, but uh, what vMach is essentially like a server backend to help manage and control this. Because yeah, cool. You can have all these remotes and all these parts, but the problem you still run into is the scalability issue. You need some intelligence somewhere along the way. Because what happens when you've got like in our case, we have over

• 22:04Speaker 01

  8,000, 8 or 9,000 micro front ends and some of these things have over 2,000 versions with maybe over 150 of them active in various products at any given point. So how do you manage who has what, where it is, and then also how do you keep it consistent? So with the JSON protocol, what we can do is what vMach does is we have an edge worker. So the edge worker receives the JSON manifests of the remotes and then runs the algorithms on the edge worker to create essentially what we call the module snapshot, which is effectively a lock file. And when that lock file is delivered in the markup somewhere, Federation switches from its non deterministic mode of okay, let's see who arrives and then let's

• 22:47Speaker 01

  do the circular initialization, negotiate all the versions, figure out who comes from where. That gets turned off and instead it goes into kind of like hydration mode instead of render mode where it says obey the snapshot and never change from it. And all the thinking was already done on the server. So just resume what the server was doing and pull the scripts in and, and you know, start the system. Don't actually think and talk to anybody else. We've already done that for you by just running part of the algorithm back on the server side. So we can deliver that from the edge, which also means anytime we want to redeploy a version or change any part of the system, it's just a change in a SQL table. There is no redeployment. There is no real build because technically by building it, it's already deployed.

• 23:28Speaker 01

  And so then it's just a case of, okay, so which part of the kind of the the module graph do you want to swap out with what? So production is a sequel row. And that's really it. It's not, it's not a zip file that you put on a lambda or whatever. All that stuff is already deployed somewhere. And the only thing that makes production, production is the module snapshot vehement return.

• 23:47Wes Bos?

  It's just a map of, so this is the build and this is the manifest of all the pieces you need to run your build?

• 23:56Speaker 01

  Or like, it's like, okay, here's header and header provides react and it has this version available and it exposes A, B, and C. And then here's home app and home app has react as well. And it also has versions, so on and so forth. And it needs it as a singleton. And so we take those two JSONs and we go, okay, well, the home is going to import A, B, C, D, and E. So we go get me homes manifest and get me A, B, C, D, E's manifest. And then we go generate module snapshot from manifest and that will output a kind of graph for us. And then we just send that graph down as like an edge script or, or, you know, like a JSON tag or whatever. And then federation knows to look for the module snapshot and switch into hydration mode. But essentially, we're running the same thing the browser does, just minus script evaluation.

• 24:42Wes Bos?

  Geez. Freaking evil genius over here.

• 24:44Scott Tolinski?

  I know. This is a scale I can't comprehend. I mean, you mentioned how many people are using things like mod- module federation and micro front ends, but at what point does this scale become necessary for people who are, yeah, what is the, the threshold there in which you need to start having these issues?

• 25:02Speaker 01

  So I would say Kent Dodd's had a really good way to put this because somebody had asked them like, hey, Kent, what do you think of, uh, what do you think of module federation? And this was in like one of his tweets where he was saying, uh, that he's super non-confrontational or something. And somebody was like, okay, well, what do you think of federation? Cause that's usually a very controversial subject. And Kent had the best, uh, kind of reply to it. It solves a problem I hope I never have. So that's essentially, that's essentially what it is. If you don't know why you need it, then you probably don't, is the first thing I say. But the, I would say generally there's two, there's two areas where federation comes in really helpful. So the first one As a scale issue. So, okay, the build is just too slow, or you know, you have these giant merge trains and you're getting bottlenecked here.

• 25:45Speaker 01

  Or maybe your release cycles are really quirky, or you work in an industry like banking, or the military, or you know, other industries like that where getting large bundled releases is really tedious. So decomposing them into smaller, more, you know, liquid parts is very helpful. Also, just size of repo, stuff like that. Like, we have some repos that are over 100 gigabytes and it takes about three minutes to change branches because my SSD, you know, we call that repo the SSD killer. So there is just the thing where the monorepo literally stops functioning at a certain level. Then there's also the other side of it, which is organizational problems. So what happens when teams just don't work well together? Like, they just refuse to communicate. Usually you'll see this in like very product-led companies,

• 26:30Speaker 01

  where like engineering is essentially the crown and the product just colors whatever they want. So then you have a lot of dysfunction on the engineering where, hey, you technically could do this with just better engineering processes, but you're not afforded the luxury of it. So the easiest way to make everybody behave together is to split them apart and then just glue it together in the last mile. So those are the kind of the two major categories. Then there's a couple of others, like, let's say you white-label a product. At any size, those are really useful. Or you do a lot of A/B testing in multiple regions. Having to not, you know, code that into, or even if you have like an A/B test team who's separate from your engineering team, usually you would put that like, you know, Optimizely or Google Tag Manager or something.

• 27:14Speaker 01

  And that conflicts with like your React code because you're essentially just, you know, patching the DOM on top of it. And that also means that your tag manager is competing for compute with the application. So the revenue funnel is competing with the thing that's measuring the revenue funnel. So in cases like that, regardless of size, it's a lot better when you say, well, what if the tag manager just dispatched something that told Federation what to do? And everything is still managed under a centrally aware system that's prioritizing, show the user something before you try to measure the monetization of it. There's nothing to monetize on a white screen. So coming to size and scale, though, I would say generally I see

• 27:53Speaker 01

  I see it start to become useful with, let's say, I don't know, I've seen, I've seen them as small as like five or six parts. But usually the parts we're talking, it's like checkout, community, commerce. And these are like super high-risk funnels that you want to just keep separate. But you don't want, like if you're in e-commerce, this is a big one. You don't want that page reload when you jump zones. Because that has a big revenue hit. Every second that adds on to the, on the transition of something equates to X number of millions of dollars. It's that ability to have a single-page application-like feel without increasing your blast radius. That's also really good. And I've seen those where, okay, well, each team might have like 30 or 40 people under it.

• 28:35Speaker 01

  But there's maybe five teams or five, you know, director parts, essentially, that's split at a park. Usually I say like the golden numbers, probably when you have like 10 to 15 parts is when you start to maybe want to consider it. Or, you know, look at your monorepo and compose it really well. On the monorepo end though, what I usually tell everybody is start with the monorepo until that starts giving you problems. And then, just when you build the monorepo, pretend it's federated. So what that essentially means is don't do weird cross-importing in the repo. Like, don't have this package depend on that one and you have like four circular imports. Where now you couldn't take this half of the monorepo, split it and put it in a different codebase because it's too tangled. If you maintain really good boundaries between that monorepo and just keep a really well-organized one.

• 29:20Speaker 01

  I've found thinking like it's going to be federated and that the code isn't in the same folder. Actually makes it so that you, your monorepo will scale longer. And then if you do want to split it up, now you've already primed the system where, okay, well, it's still just, it's the same package import. That just becomes a remote entry, not a sim-linked file in the monorepo. And then I can easily switch it over to more at runtime composition from a install link-based composition.

• 29:46Scott Tolinski?

  Wow.

• 29:47Wes Bos?

  So I have a, in my monorepo, I have like a util's package. And I throw all my like shared util's, the format money is a very simple function that I have in there. And I use that in all of my, I have three or four different properties. Is that a bad idea for that type of thing? To have a util's package that shares util's? Or how would you publish a util's package in module federation?

• 30:11Speaker 01

  So for things like that, it usually, for me, it also depends on like the, like who owns it? And then like, how granular do you want to get? Because granularity can also backfire on you. I'll see a lot of users who are like, oh, well, let's make everything federated. And we'll have like 160 parts to make up, you know, something that doesn't need 160 outrun time parts. So I always like, think about the balancing act of it. So one is like, well, how often are you deploying those util's? Is it a major friction point? Two, like the rate of change on there, does the API contract potentially break more often? Is it something that maybe is just worth duplicating? Because it's all right if it drifts a little bit. Now, if it's something, say, like authentication, that's something you couldn't have drift on.

• 30:53Speaker 01

  So if it was, say, auth util's, I would probably put that as a remote or as a shared module. That way I could say, hey, I can always go and get X version of auth. Or if it's like a, you know, kind of the end facing part, you could expose the authentication handler. So everybody just pulls it in and then you use something like Vmac or Zephyr to say, okay, you know, home app is going to use this version of auth. That one's going to use a different one and we can roll back or roll forward. So Zephyr is a little faster than Vmac, so I'm going to go with the Zephyr number. But it takes about 300 milliseconds to change any part of the system. So roughly 50 more milliseconds than it takes to complete a mouse up, mouse down event. And that's your full deployment, like update to all your users everywhere.

• 31:36Wes Bos?

  Wow.

• 31:37Speaker 01

  So, you know, it kind of depends on, well, where do you need that speed, that agility in there? At the same time, I usually try to say, well, like maybe component library? Okay, depending on how you do it, that might be a good one to put as a shared package or maybe have like a vendor remote who doesn't expose anything. They just have kind of like the old school vendor bundles. But what if you had a vendor remote where you say, hey, I'm going to have all the packages that generally everybody wants. And then with federation, I have the whole middleware concept. So I can write a runtime plugin with the resolve share hook and say, hey, if the vendor remote has a version that's viable for you, prefer the shared module from the vendor remote over your own. If not, go into negotiation and find the best option available.

• 32:21Speaker 01

  So in those kinds of cases, I might bubble them up a little differently. So I can just set preferences on where the module comes from, but still have like, you know, a bit more non-centralized kind of control. So in the case of util's, I would say probably it's really small and. You know, that would probably be something I would just consider like bundling in because is it worth another network request? And to like go out and get the extra chunk parts or is it worth maybe an extra three or four kilobytes of bloat per bundle that's using it, plus the tree shaking, which thins it down. So you know, you kind of want to balance it out between like, what's the necessity of it versus like, what's the excessiveness of it?

• 32:59Wes Bos?

  And then how would you do like a major upgrade of of like, for example, React, you're moving to React 19? People want to start using React 19 features. How do you do that? You can't ship multiple versions of React, can you?

• 33:14Speaker 01

  So you can, but here's the kind of this is the this is the hole that I dug myself into. So when I created Federation, I really wanted the whole, well, I just want to import stuff. Like, I don't want mount functions or the kind of LOSA app thing. I didn't want that anymore. I wanted the whole, oh, you know, you should you shouldn't need to know how to do anything besides say, I want to share React and I want to expose header and I could just go import, you know, nav team slash header and it's just like an npm package. That works great, except for in this scenario where it backfires of, oh, but React's a singleton. So how do you upgrade the parts? Because you're using the import from, so it's technically one render tree. Now, if you were to use Federation with like the LOSA kind of architecture or adapter solution where you say, okay, I'm gonna

• 33:58Speaker 01

  I'm gonna render a div that mounts another React app onto the div and then we just pass the props and you know, like component did mount or component will mount and just, you know, run render or hydrate again when the props change. Then, you know, you can have any any amount of versions of React you want and they will all recycle the minimum number of React major versions available. But if you're doing it in a single render tree where you're import from everything, you do kind of lose that ability because it is utilizing the whole singleton paradigm and you're technically in a single render tree. So for doing multiple React versions, we did come out with like a framework paradigm or framework capability for Federation, which is essentially we provide adapters to allow you to say like, take

• 34:43Speaker 01

  old React to a newer React or React to view or A bunch of other kind of directions for different frameworks. And what we do in there is we actually rebind the routers between them. So your React router will be aware of the view router, or the view will be aware of the React router. So you can click through it and it roots and behaves like a normal app. But essentially what you're exporting is like create component adapter, consume component adapter. And then that would say, okay, now you can load multiple versions of React, so on and so forth, but you would have to have some paradigm to say, okay, here's a separate tree. So that's generally how we do it when we need to do like upgrades on things. The other nice benefit as well is inside of the Federation runtime plugin, we have a hook called load remote.

• 35:28Speaker 01

  So what that does is that's like, okay, I'm going to go and get the module I want to load. So if I want header, now I can get header back, but inside of that kind of middleware pipeline, I know who's loading header. So I can, I've done this before where I can go and look ahead and see, okay, the thing I'm pulling in needs React 19, but the consumer is currently using React 17. So when I pull load remote, I can say, all right, but don't just return the, the, the, the factory function. Return a new JSX function that says, okay, get their React DOM and return essentially like a proxy React component that mounts that says, okay, parent app, render a div, thing you're importing, mount onto that just rendered div and essentially do it all automatically for you.

• 36:11Speaker 01

  So you don't need the whole mount adapters that you define in user code, but you could do the same thing in the runtime plugin and say, hey, if the thing coming in is, needs a different major than the thing it's going to, have the export default be render a div and then have, you know, the thing you've just loaded be, okay, put that as JSX on the rendered div you just created. And then you can like magically stitch it together.

• 36:37Scott Tolinski?

  And if you want to see all of the errors in your application, you'll want to check out Sentry at sentry.io/syntax. You don't want a production application out there that, well, you have no visibility into in case something is blowing up and you might not even know it. So head on over to sentry.io/syntax. Again, we've been using this tool for a long time and it totally rules. All right. So who owns the like update process for even like minor depths? Not like minor bumps or things like that? Like, is it like the team who owns the chunker or however you refer to it? Like, who owns those choices?

• 37:15Speaker 01

  So this is the other thing where a lot of it, it depends. So at like, so when I was at other companies, I would say the platform team would own certain upgrade phases. So we would own things like the application shell. So we had a federated app shell as well. So when you develop locally, you pull the app shell off of stage or production and we encapsulate local host in productions app shell. So you're technically developing as if you were using productions code and your local app is essentially just like a zombie application with nothing. And all the providers, all the auth, everything like that is coming from stage or prod. And then when you deploy it, we reverse the process and we say, okay, now the page is federated into the shell, not the shell is federated around the page. And so we can kind of reverse that.

• 37:57Speaker 01

  So then the platform team would own the wrapping layers to ensure that telemetry was there, that when we update the telemetry, we would, we would be the ones who would, you know, push the versions to all the product teams and so on. At ByteDance, a lot of this is controlled by the product team. So the product team will say, all right, maybe if I want to update, say button or whatever, just so that we kind of have that safety. Cause again, you don't want things changing without your notice, unless you do. So generally we'll say, okay, the product team owns the lock file or the Vmac, you know, the Vmac dashboard. And then I can go in there and say, request an update to it. But I also have like Chrome Dev Tools, which allows QW to assume a certain module snapshot and test it without needing to go in and change Vmac or change production or do anything.

• 38:43Speaker 01

  So any environment can be any environment you want. You can use production as local host and develop locally, telling the Chrome Dev Tools, use my local host remote and replace production. And we can turn hot module reloading on in production. So every time you hit save, it HMRs the app just like in development. But generally we say the lock files owned by the product team that owns it. And when we need to contribute to it, you know, we can either request the update happen or whatever. Now, again, this can also come down to just the dynamics between the team. Because maybe it's something like header and it's super generic. There's nothing that's really going to break in there. And maybe the product team doesn't want the overhead of having to approve things or whatever. Then the teams can agree that, hey, we just want the latest of header. Or what they do in Vmonk is they would say header star, which just means get the latest.

• 39:27Speaker 01

  So then every time header pushes out a new one, it's instantly updated. But they can control the rules on, you know, where do they want the governance on their own product? And what's going into it versus where do they want, you know, less restrictions on what's flowing in and out?

• 39:40Wes Bos?

  Man, and do you ever see people use multiple frameworks with this as well? Cause often I talk to specifically at conferences, always people that work at banks or insurance. And they say, we have 5,000 devs in 14 Angular apps and 20 React apps and three Vue apps. And it's just like, sometimes you need to share stuff. How do you approach that?

• 40:04Speaker 01

  So, so this also works with multiple frameworks. This is one of the reasons we brought out that framework bridge system. So essentially, oh, if you have a Vue app, you can mount it and the consuming person treats it like React. They're using JSX. They have no idea they're talking to a Vue app. So the consumer receives it in the format that they, that their application is regardless of what the other framework is. The thing I don't like about the multi-frame work approach, and this is one of the reasons that micro front ends was like, where it got popular is, oh, you can use any framework you want anywhere. And that's like a terrible idea because yeah, your devs might love it, but your devs are the, the revenue funnel. So for me, it's like, okay, well, now I'm loading multiple frameworks which have weight on them.

• 40:49Speaker 01

  So for, yeah, you can use it for that and a lot do. It's also really good for like transitionary periods. So, oh, if I'm going from like Angular to React, boom, Federation lets me link all the parts together and it behaves like one. But generally I try to say, you want to try to unify on your framework just so that, well, one, pull dilution. If you have to maintain two or three component libraries that all look the same, but are in different like formats. Yeah, you just have to hire 3X and develop 3X on it. So generally, I like to say, well, try to keep it all in one framework because you have more usable parts. It's lighter to load and I think that's what gave microfront end a bad name was the wild west of, hey, we can just use anything, anywhere and it all will work, which yeah, it will, but at what cost?

• 41:34Speaker 01

  So generally, I don't like that approach. Like at ByteDance, everything is essentially in React and that's how we use it because that gives us the most like flexibility and interchange between any part of the company. But we are also very big on unified architecture. You know, the sum of parts makes a product and everything there is kind of geared for how fast can we get stuff out? And you know, how efficiently can we? If we can bring down the cost of R and D, we can develop a whole lot more, see what happens without it becoming like a, without a bad idea being a negative thing. Oh, build 20 apps and if it takes you three months, who cares if one of them is successful?

• 42:11Wes Bos?

  At ByteDance, when you're writing React, I think I saw somewhere you have a meta framework that you use. Is that true?

• 42:17Speaker 01

  So we have two meta frameworks. So the one that we had, so the big one is Modern JS. And so this was, this was a problem that we had seen with Modern JS. It didn't get a very welcoming launch. And so I had to translate the posts from Chinese when I read about like why it wasn't well received from somebody who read it. But essentially the, the reason was it's like, this looks super great if you need to do anything in your ByteDance, which was kind of true. Like it was essentially this massive, you can go, if you go to Modern JS doc, I think it's getting better now as we've reduced the scope of it. The fact when I first got there, I could read the docs for 10 minutes and be like, yeah, this is what I need. But what does it do? Because it was Modern JS doc, which was like static doc gen.

• 42:58Speaker 01

  It was Modern Module, which was a like meta framework for authoring NPM packages and it was the Modern framework. So you go there to read the docs and you aren't a hundred percent sure what it does within the first like five minutes. You just know it does a lot of things. So what we'd actually ended up doing though, is now Modern Doc is actually RS press, which is our RS pack based, you know, static site generation, like, you know, all of our RS pack federation, all of All of our documentation sites are all just RS press. So we split that part out of Modern JS and said, hey, same idea. But having a one that fits all is just too intimidating and too like polluting. So while it works great for unified architecture, there's that lack of composition, which just adds to confusion about what the point of the product is.

• 43:43Speaker 01

  So we made RS press, which is essentially the same idea, just repackaged and reduced the scope out of Modern JS. And then RS lib, which is also the same thing as Modern Module. It's a meta framework designed for authoring NPM packages. And now that's got its own side, it's own everything, and it's completely separate from Modern JS. But it's effectively almost the same architecture, just refreshed onto like a Rust ecosystem. So Modern JS is the web framework, but highly composable. So a big chose that we had for us was, again, we need to support like all sorts of weird environments, like WeChat things and native apps and all sorts of stuff. And so Modern JS is actually just Remix based application. But Remix isn't a meta framework.

• 44:25Speaker 01

  Remix is really, and this is what they've all said, Remix is just a library. It's React Router with a render handler. Then they added a VEep plugin because users want some meta framework preset. So the VEep plugin is what takes it from library to meta framework. So essentially what we wanted was, okay, let's get like a next JS kind of equivalent of what the Remix library can provide. And then that became Modern JS. One of the big challenges that we had had though is like Remix at the time was very SSR heavy. And we don't have enough compute to handle the traffic to SSR everything. So, because we've got, I don't know, like a couple billion users.

• 45:05Speaker 01

  And if you have to SSR every one, you will, especially with highly personalized, ever changing content. You'll effectively exhaust an AWS region very easily. So we'll run out of Lambdas before we can fill the spike in demand and nicken in point. So we needed mechanisms that we could control it to say, well, if I don't need to SSR this part or this route, I shouldn't have to. Or if I need to SSR it, if the Lambda doesn't reply within X milliseconds, it should be able To skip the server render and then the edge should be able to send back the CSR app and it'll be able to restore itself in the client. So modern JS kind of handles a lot of those other mechanics for us.

• 45:46Speaker 01

  But it's effectively, you know, very similar to remix, similar API, stuff like that, but with, you know, micro front-end support kind of baked into it for us, you know, all the parts like that. So virtually every app that I see or work on is modern JS based. Now in China, we have another app called Pace, which is essentially like if you took RSC, Astros, Server Islands, PPR, like if you took all the fancy acronyms that Astro and Vercel have kind of created. It's effectively all of those things combined, but based on remix and it's super lightweight. Like it's CLI is literally run this long bash shell command that like makes directories and like writes a whole. It's like a one liner bash that's like that big and it just stencils out the app.

• 46:32Speaker 01

  Um, but Pace is really, really cool. It's like probably our more modernized, like super light, adaptable kind of response to Server Islands, RSC and so on. And but that's primarily only used on like DoYun, on the DoYun web, which is the homepage for the Chinese TikTok. And I think a lot of that was really just proving ground for, okay, like, how would RSC work? Like, you know, what do we want to do with it? And then now we've like fiddled with that for about a year. We're looking at, okay, now we'll pull those features back into modern JS. So modern JS is essentially like where all the big stuff happens, that we have maybe two or three other web frameworks that aren't open sourced.

• 47:11Speaker 01

  Where like faster iteration or experimentation happens just because it's, it's cheaper to write a new framework than absorb the risk of changing the one that we know works.

• 47:20Wes Bos?

  Another thing I want to ask you about, because I, I tried for a year to find this out and I did, I did finally get the answer, but I'm curious if you know about it is ByteDance has its own JavaScript runtime. And I, for the longest time I could not figure out, I was trying to, I was asking everyone, like, could somebody please tell me what they're using this for? And then finally I found some posts in Chinese and I translated it and I found the author of it. And, and he tweeted a couple of lines back at me, but are you, are you aware of, of that at all?

• 47:48Speaker 01

  There's a few places that it's used. So the one is, I think we have something called I can't remember the name. I think, I think it's Byte Cloud or something like that. But essentially, we have our own like edge runtime. So like, I think it's more popular in Asia, but essentially kind of like how Vercel has whatever they've got. And Cloudflare has, what's theirs? WorkerD.

• 48:11Wes Bos?

  Workers.

• 48:12Speaker 01

  We've also, yeah, we've got our own edge runtime. And then we also have something basically similar to Hermes, which is in React Native. We've got like, similar to Hermes kind of makeup as well for, you know, doing whatever like Hermes would do as well. The, the main differences, though, I would say are we've pushed a lot for web standards. So making sure it's actually compliant with like all the web specifications. Hermes, I know, doesn't have full standards in their APIs. And then the other thing that we also have is ours is supports the NAPI APIs. So you can use this tiny runtime, but it's NAPI compatible, uses the NAPI API or can use the NAPI APIs.

• 48:54Speaker 01

  And, and I think we can compile it down to WebAssembly and, you know, all the other kind of things that are useful for it. But effectively, like it's, um, those are the, the couple of areas that I know it's utilized in.

• 49:06Wes Bos?

  Awesome. And for, for listeners, that NAPI is the node API for basically kicking out to native APIs that may sit on your user's machine.

• 49:16Speaker 01

  Yeah, this is like how RSPAC is really quick. So RSPAC is fast because it uses NAPI and NAPI is where we combined to, like, this is, I think, a challenge with, say, WebAssembly based stuff is you lose a lot of the speed building it in that way. But if you don't target like WebAssembly, what you're usually targeting is the NAPI APIs that you can use. I know NAPI RS now has WebAssembly targets, so it's a lot more interchangeable. Uh, but yeah, essentially that lets you fast access to the, to the C layer. And you can skip the, the JS communications for

• 49:50Wes Bos?

  Okay.

• 49:51Speaker 01

  Various access.

• 49:52Wes Bos?

  That's awesome. So one thing we didn't even talk about at all, and we'll have to have you come back at some point is like, like RSPAC entirely, right? Like, so part of the problem at ByteDance was that building was very slow, right? Maybe you can just give us a little teaser. Maybe people can watch the conference talk, but build times are very slow and, and you need to speed that up.

• 50:15Speaker 01

  It's really hard now in like the modern times to get the full benchmarks because like a lot of areas are using Zmac or use Federation. Yeah. So I had the business measurements on Federation and this was still what it was WebPAC based. So we haven't like re-consolidated it with now Rust plus that. But I know with, for any product team that used Federation, you would see a 300% increase in product output. So it's a little different from engineering output, which means, oh, I can spin the wheels in the mud quicker. But product output is more like, you know, meaningful things that enter the product, you know, that have like a positive impact. So like feature, actual feature release. So with Federation, we had seen about a 300% increase. And so part of this was smaller shards, faster builds. But the other part was again, just that ability to interchange it

• 51:00Speaker 01

  and kind of avoid the double deploy scenario. But now still, there's lots of aspects where there were still challenges. So, you know, one of the other big areas was really just builds in general. So what we had seen is for any build that, for any CI process that takes 35 minutes or over, the mean time to merge is about 1,600 minutes. So it was kind of also regardless of size. Obviously slower the build, the bigger the code base. That does skew a little bit. But when we introduced like Rust tooling, you know, that obviously helped kind of normalize our hypothesis here. But essentially what we saw is if a build is under five minutes,

• 51:43Speaker 01

  the merge latency is about 20 minutes regardless of the size of the application or anything. If the build is within five to 10 minutes, or if the CI is within five to 10 minutes, the merge latency is roughly about three hours. Now, if it's say 30 seconds, it's still 20 minutes. And if it's four minutes and 30 seconds, it's still 20 minutes. So there was virtually no improvement beyond getting into the golden zone. And generally, you're not going to do a whole lot of merges within 20 minutes just because human processes. So generally what we try to target is the five to 10 minute range for CI to complete. And that gives us a realistic, cool, it's done, people can preview it. And generally, you know, within, within three hours, that merge will happen. But a major constraint of this really became,

• 52:29Speaker 01

  Okay, like how much latency are we losing in the product? Because the build takes 30 minutes and how long are you staring at it before you change another branch and you go and do something else or it's the end of the day or you forget about it and you get sidetracked? And now that product velocity starts to take. So getting that, getting that curve down to the habitable zone of, you know, five to 10 minutes, you know, I think it had something like just on a single repo, I did the math on it, but on a single repo that would churn from taking it from like a 30 minute build down to that zone. It's something like a million hours in latency a year recovered for that one, for just one product. And that's just one of the repos among thousands of them.

• 53:12Speaker 01

  So a huge amount of like actual product value time pushed back in, more iterations, more meaningful iterations.

• 53:18Scott Tolinski?

  Yeah.

• 53:19Speaker 01

  But the, the actual reason that we'd built RS pack, yes, builds were slow and we saw this latency curve in here, but the, one of the primary reasons for RS pack was actually cloud costs. That was the biggest cost center was the cloud.

• 53:33Scott Tolinski?

  Really?

• 53:34Speaker 01

  So, because I think also labor's not quite as premium as it is here in the US. So manpower is not the major burn, it's compute. So the primary reason for RS pack was to reduce the cost of compute. So what we had seen when we switched everything over to RS pack was across the board, 80% reduction in cloud cost, which was huge. On top of that as well, we'd also seen like bandwidth egress costs go down because we've done a lot of work to improve tree shaking and code splitting and how modules are optimized and where the chokes go. Outside customer, outside user had told us that they'd seen about a, I think it was like a $10 million a month reduction in egress costs?

• 54:18Scott Tolinski?

  My gosh, yeah.

• 54:19Speaker 01

  Just by taking what they had currently, changing the package from web pack to RS pack and then switching out a couple of the loaders like we suggest. And other than that, it was, you know, just a lift and shift, took them about two hours and that was, and you know, you're looking at $10 million in egress cost.

• 54:35Scott Tolinski?

  Being responsible for something like that is, is pretty amazing. Like even being on a team to, to cause that much of a shift in a reduction of costs is out of the scope of my brain power here. It's pretty amazing, yeah.

• 54:51Wes Bos?

  We had the folks on from Amazon L L R T and they were explaining that as well. They're like, simply if we can cut. 30% off of how some, some specific things take to run. That is, it might just be 30 milliseconds, but compound that by how often it's run. It's a massive savings.

• 55:12Speaker 01

  Yeah, yeah, I think like our own, our own savings, just like from our own like measurements and stuff like that internally. RSPAC's ROI has been in the hundreds of millions of dollars for us, ourselves in utilizing it. And I mean, given, given how much it costs to develop, this was another crazy thing is like, the problem was so expensive that the cheapest solution was to write a new bundler from scratch. And that was truly, it was cheaper than migrating to a different build solution. Cause again, we've got like 10,000, 20,000 repos. So there was, we had tried to like, I think early on, there was, okay, maybe we look at Veet and we see, hey, maybe Veet could be, it's getting popular and all of that.

• 55:53Speaker 01

  And so there one was the, the, the cost of just moving everything, cost of change management was massive. It would have taken, you know, a huge amount just to switch everything over. On top of that, build is already the biggest on-call area. Now you're going to swap it out with a different build. And on top of that, we also had like things that were non-negotiable, like, I know they're fixing it now with RollDown, but at the time it was the whole unbundled problem. So unbundled, for us anyway, like if you have Chrome extensions on the browser, or you have anything in like the dev workflow where maybe they have their cache still churned on or churned off. Like, you know, usually you'll see, like habitually, that could lead to like minutes to hot reload the page because the network requests have to go down again.

• 56:37Speaker 01

  So technically some of that is user error, but the problem is, is again, on-call pressure and the fact that it doesn't just magically do it. And then like, I think this is more like issues with, I think ES build and a couple of others, but like the chunk fidelity. Not being able to chunk things really well, sounds really silly, but when you start looking at the, the revenue pipeline on the larger end. Artifact stability and integrity and like the ability to chunk things correctly, not make it too granular, not make it too, you know, too large, became a really major. Concern and ultimately the cheapest option was to just rewrite Webpack and Rust. Now granted, there were four other Bumbler's we wrote in between and architectures we wrote. It wasn't like, oh, let's go and make Webpack and Rust.

• 57:22Speaker 01

  That was the last attempt after several other design attempts before we kind of realized, look, there's just nothing out here that has what we need it to do. And ultimately, everything else we tried was a case of everything that a Bumbler lacks that we would require, we're ending up, we just found out, we're ending up just writing Webpack again, minus the test, minus the stability, minus the ecosystem. To the point where it's like, okay, well, if we need so much of these Webpack type capabilities that we know have worked for the past decade, it's probably just cheaper to just port Webpack to Rust. And then on the back end of that, we can look at speeding it up, getting rid of things that we don't like, but starting with a guaranteed, we know it works, one-to-one

• 58:03Speaker 01

  match, anything that we change from there, we know we have 60,000 unit tests backing it that it always works. And it's a lot cheaper to just on the tail end, decide to make changes afterwards, once we know we have an exact business drop-in for it.

• 58:16Wes Bos?

  Yeah, man. And like, the one thing I thought, I know we're getting to the end of the hour and I'll wrap it up soon, but one cool thing I wanted to say is that like, you obviously built RS pack and then also made the API work with Webpack. But one kind of interesting thing you said is like, that's not the only way that we could surface the technology behind this, because like, if you like the way Veet works, maybe we could build something like that. Or like, the engine of this thing could be surfaced in other ways.

• 58:45Speaker 01

  So this is, this is a lot of hate that Webpack gets, unfortunately, is a lot of it I find is like, people just don't like how you configure it.

• 58:53Wes Bos?

  Yep.

• 58:53Speaker 01

  Like, that's it. And Veet, you know, changed the game in terms of simplicity and streamlining the user land overhead massively. And so like, we have something called RS Build, which is essentially the Veet style API, you know, super slim, no loader plugin config, totally different config layer. And it's just a build tool, similar to how Veet is a wrapper around Rollup and And ES Build, RS Build is a wrapper around RS Pack. So it gives you this really streamlined, oh, if you want React, there's a React plugin that sets up all six pieces that you want. So much smaller surface area to chain things that you want together. It has an environment API as well, similar to Vets.

• 59:34Speaker 01

  All of that. So we have like those layers to separate you from the underlying horsepower of the tool, which framework authors probably still want. And users want that simplicity. But a lot of people go, oh, well, this is just a Webpack port, you know, it's whatever preconceived notions you have. But the main reason that we had chosen RS Pack, there's two, there's two sides to it. So one is, yeah, there's a plugin API and having all of that for free and work right out the box, obviously saves time, saves money, gives us an ecosystem from day zero. But the second aspect of Webpack is its actual architecture internally. And so this is where we ran into the most problems developing bundlers is, building a bundler is relatively easy and cheap to do. Designing it and architecting it is not.

• 1:00:18Speaker 01

  And because RS Pack is in Rust, the nice thing here is the Webpack part of it, that we think of, oh, Webpack, I don't like that. Okay, what you're talking about is the bindings, because this is a binary, so technically there's no deep seated or connected things. The bindings happen to look like Webpack's front-facing API and config. But the backend of RS Pack is using Webpack's architecture, how they designed their actual compiler to work. And the compiler architecture is totally separate from the user plugins, the APIs, the config layers, all of that just happens to be a style. So we did this, we're working on releasing in the next year the RS Pack SDK, which is essentially saying, okay, if you want to build a custom bundler,

• 1:01:01Speaker 01

  do you want to have to build your own module graph and chunk graph and compiler class and all these parts again? Probably not. So we're going to release an SDK where you could say, all right, well, here's all the parts and you can make any bundler that you want. And those bundlers won't have any kind of, you know, connotation to Webpack's design or anything. Purely, it's just here are the parts and you can build it. And if you create certain set of bindings, it happens to look like Webpack. So that architecture, we've taken, we have a project that we've been experimenting with called Unpack. It'll be under HardFist, I think his. We get a handle as hardfist, so it'll be like hardfist slash unpack. Yeah. And essentially, it's using unplug-in as the bindings, as the Rust bindings.

• 1:01:46Speaker 01

  So instead of using like webpack's bindings or whatever, we said, well, what if we took RSPACK's architecture, or the webpack-style compiler architecture, but we used unplug-ins bindings, it's API. So instead of using webpack's one, it's just whatever unplug-in does, that's the bundler's API. And in there, we kind of noted, okay, it's about 100 times faster than RSPACK, but it has 10% of the power of it. So obviously, again, you know, what makes it powerful, what makes it slow, all of that is really just, okay, well, when you use webpack, for example, it's applying like 100 plugins for you. When you put your little webpack config in there, when you say like target node, that's a node target plugin, and a common JS chunk format plugin, and like six other plugins that make up,

• 1:02:30Speaker 01

  oh, make this thing go into node JS output mode. So when you slim all of that down, you go, okay, well, here's a compiler class, and if you want to chain things on to it, then you can. Obviously, you can make it super fast, or simplify the API greatly, but in doing so, you might lose some of the functionality that it can have, just because user API is more limited, or we're just not adding as much on there to give it that one-for-one experience to webpack. But at the same time, it does kind of prove out that, hey, the only thing that makes it webpack is the bindings file. So in V2, what we're looking at is, okay, so what happens if we potentially look at, let's maybe replace a couple hooks that we know are slow, or we know are burdensome. Maybe we get rid of those plugin hooks, or deprecate them, and we bring in something new that can do

• 1:03:15Speaker 01

  this in a better format. Or, you know, what if we start changing the user API a little bit, and maybe offer something that's a bit more streamlined? Or what if we use it in tandem with something like RS Build, where we can already say, hey, we can simplify all of this, and we can make the backend of the compiler more oriented to dealing with the complex tasks. And we say, hey, the end-user side, your main point of use is through RS Build, which is this lightweight streamlined tool. But yeah, essentially, there's no real connection between the webpack architecture and the webpack config and plugin API. And for RSPACK, the only real connection there is the binding.

• 1:03:50Scott Tolinski?

  Man.

• 1:03:50Wes Bos?

  Man. I just had my, my mind absolutely blown for an hour straight. I could, I could probably go on for forever.

• 1:03:58Scott Tolinski?

  Oh, yeah.

• 1:03:58Wes Bos?

  We will start to wrap it up there. Scott, do you have anything to add here?

• 1:04:04Scott Tolinski?

  I have, I have questions, but I'm sure they would invoke another 30 minutes of discussion here. But no, I have nothing else to add. Zach, this has been amazing. It's been mind blowing talking to you. You've made sense of so many things that have big, big old question marks in my head for so long. So I really appreciate the time you've taken to break all this down for us. It's, it's been really amazing.

• 1:04:25Speaker 01

  Yeah, thank you for having me. There, there is one other thing I wanted to add that we never covered on the Federation side, which was just, so NV2, while we came out with the whole middleware side of things, there are some other major features that we had brought out, which I would say these are probably some of the killer aspects that you don't find with the other solutions. One is that we have actual hot module reloading available, which, you know, you can turn in, it works through a Chrome extension. So even if you're in like production, you can hit the switch on the Chrome extension. And because production is powered by Federation, we can say, oh, well, React is now the development version of React and we can source that from a different location. And we have a plugin built into the system that tells it, hey, if the Chrome extension tells you to use the dev React, you're going to source the dev React from the Chrome extension instead.

• 1:05:09Speaker 01

  So it allows you to switch it on and off without actually any build complexity. One of the other major things though, that I didn't cover is that we support remote types. So this has been probably a major issue with any distributed system is you lose the ability of type safety. And so this is something that Federation does support is hot reloadable types and remote types. So if you deploy something to stage or production or whatever, when you link to that manifest, in that manifest, we have a piece of metadata that says, hey, here's where the types bundle is. And when the compiler starts up, it will search through the manifest and it will resynchronize the types and extract them on your disk for that version you're using. And then obviously like if say you push new version to prod or stage and the contracts change,

• 1:05:53Speaker 01

  the next time you run a build, your types will also be updated and you'll get a type validation error, you know, whatever type error you get to say. Hey, the contracts don't work anymore. So that's like probably one of the coolest features that I've seen uh come out there. Uh, one other thing that's really neat with the types, because we got this a lot, which was, well, how do you deal with it when it's not statically analyzable? Like what happens when you're using our runtime, since Federation is not bound to a bundler anymore? There's a plugin that helps prepare your chunks. And then there's the runtime that does all the orchestration of it. And the two aren't necessarily mutually exclusive. Like you can use the runtime without a build plugin, no problem. So, you know, you can consume it anywhere. But a lot of people also use the runtime when, say, it's super dynamic, like the CD or, you know, a database controls

• 1:06:37Speaker 01

  which remotes are going to load and which slots on the app. So you don't know what goes where, or you're using state to kind of set the import that you want. Inside of there, we also have when, when the dev server is running, our type engine runs in the background and the browser will actually send the at runtime discovered, you know, because we can't analyze it statically, but when you go, hey, load remote var slash var will go, oh, they're asking for home slash checkout and will actually tell the, the type engine, hey, here's another type that we located that could only be found in the browser. And we post that back to the type engine and it will synchronize dynamic types as well.

• 1:07:15Speaker 01

  So even when you can't statically analyze something, the browser is informing it of any, any unknowns that couldn't be found during compile.

• 1:07:23Scott Tolinski?

  Yeah.

• 1:07:24Wes Bos?

  That's sick. Man, so many hard problems to solve. That's really fun. Cool. Wow. Well, thank you so much for all your time. Uh, the last thing we'll ask you is, is what you want to plug to the audience? Do you have anything else other than the stuff you just talked about all day today?

• 1:07:41Speaker 01

  Okay. So one, definitely if you're into micro front ends or anything like that, check out zephyrcloud.io. That's kind of the, the, the equivalent of V-MO for outside. So if you're looking for stuff to manage or, and it's not just, oh, manage the assets, but it kind of, it's like Kubernetes for the front end. It's your infra, your cloud, no markup, and it just helps you manage your cloud and where things go. And it's got stuff like, I think the coolest thing we saw was uh that they pushed a bad deploy deliberately to a native app, to a backend service and to the front end. And within one second, the system automatically recovered from.

• 1:08:17Speaker 01

  Three critical failures throughout all three applications. They were all running on Federation and it was down for one second and brought back up autonomously. So really cool thing to definitely check out. Another one that we've been cooking up at ByteDance has been MidScene.js. Definitely check it out. It's essentially ChatGPT meets end-to-end testing and browser automation. We have a really cool Chrome Accenture, which I've used like to help manage my stock portfolio and stuff, where I can just tell it, hey, go do this and click around here and I hit play and it like takes over the browser and does some stuff. But there's actually a library for it too, to do all of your end-to-end testing. That's definitely another cool one. That's like under our web infra hood at the moment. That's definitely worth checking out.

• 1:09:00Wes Bos?

  So that's like a vision, that's a vision model, right? I was just telling Scott about this. I was like, there's models now that can like look at something and figure out what buttons and actions are.

• 1:09:09Speaker 01

  Yeah, and it's super reliable. So it uses, it's got like a whole inspector that comes up, but essentially it uses machine vision, then it tags the page and figures out like what each aspect is. And then it interprets your prompt and goes, okay, cool. Then I should be using this tag for this part of the page. But I've used it to like open and close positions, like, hey, every morning, run a Chrome job that goes clicks through each stock in the list. If the stock has spiked, you know, like sometimes pre-trading, they skyrocket. So I say, hey, if it's pre-trading window and the thing has risen by X amount, then I want you to step one, click on sell position. Steps two, sell 50% of the position, then click review order and confirm and run that Chrome job every morning before I get up to check for any spikes and sell in and out. So I obviously use it for something else,

• 1:09:54Speaker 01

  but it's designed for end-to-end test assertion with like PlayRite or something like that.

• 1:09:59Scott Tolinski?

  This is amazing. I've never seen this before and I'm gonna be spending some time with it today. So thank you for that.

• 1:10:05Speaker 01

  Yeah, it's got lots of cool uses.

• 1:10:07Wes Bos?

  My nefarious scraping brain just started pinging, being like, oh.

• 1:10:12Scott Tolinski?

  Yes, right. Yeah, your scraping brain.

• 1:10:14Wes Bos?

  Cool. Awesome. Well, thank you so much. Appreciate all this. We'll have you back on at some point. You're a fountain of information and yeah, we'll catch you next time.

• 1:10:25Speaker 01

  All right. Thanks for having me. Good to meet you all.