Chunking

My name is Tobias Koppers, I work at Vercel as a tool pack team, and today another Bundler topic, chunking. So what is chunking at all? So my definition of chunking is it's a process in a Bundler that places module into output files. And I usually call these output files chunks because whatever, it came up with this name. So it's basically like we take the module graph of your application, decide in some weird way how to put it into like chunks, and that's chunking the process.

If you visualize it, there could be like a really simple example here. So we have a module graph, a really small one for this one. So we have a bunch of modules, these like white dots. These modules are usually connected via dependencies, like import, or acquire, or an async import. And these are the edges of the graph. And I visualize a really special edge here because it's really special for chunking, there's a red dashed one, that's an async import, so like a dynamic import or something where you load something on demand.

And also like, usually your module graph has a lot of empty points, maybe your pages or that stuff. And here are the most important pages of an application is, of course, the legal page and the pricing page. And you can basically visualize chunking in a way that you just draw cycles around these modules. And yeah, I did this with the screen cycles, and basically means like all of these modules that are cycled together are put into one output file. And yeah. And an example like this async import here on the left side, with the red dotted thing, is put into its own chunk so we can load it on demand.

And we have like all the entry points being its own chunks because then we can load them independent of the other pages. And you could also, like in this example, I also put like the middle area of modules that are shared by these two entry points into like its own chunks, so we can share that. But yeah, it's really complicated. You can come up with a lot of different ways to cycle these modules. And yeah, that's the process of chunking, not of cycling. Yeah.

And how to do chunking in a good way? So yeah. One thing is you can look at metrics. So most often, we care about the performance metrics. So these metrics basically influence how we chunk up stuff. And before we're looking into the metrics, we want to look into another term that is chunk groups. And I say chunk groups are basically a set of chunks that are loaded together that could be on an initial page load, you load these 10 chunks in script tags.

And that's an initial load chunk group. But it's also like other types of chunk groups, like on a dynamic import, there's an async loading chunk group where you load these chunks together as a group. Or a navigation between two pages is also like a chunk group. And we really want to look at a few metrics. One metric is the download size. And with download size, I don't mean bundle size because people often see bundle size as this is the size of the folder that the bundler, the output folder the bundler produces. But the total size of that folder, that really matters.

It might matter for compile time of your application, but it doesn't matter for the user perceived performance. What really matters is the download size per chunk group. These initial page load has whatever bytes downloaded or these on-demand loading of whatever bytes downloaded or this navigation has that much bytes downloaded. So, we really, really want to look at these metrics on a sense of per chunk group. And similar for the second metric, which is request count. Request count is obviously not good if you do a lot of requests because every request has an overhead.

Even with HTTP 2.0, there's still an overhead. It's fewer than before, but it's still not good to load millions of files in your application. And now we have two metrics already. If you look at an example, this is a really small one. I have two pages, A and B, and they both share a shared module. And now you can come up with different strategies of doing chunking here. So, one strategy might be you put page-specific modules into the page chunks, like the blue and the green one and then make a common chunk where you put the shared modules in it. That looks good. It basically means every page has to do two requests, but it only downloads the modules it really needs. And if you do a navigation from page A to page B, you don't have to download the common chunk again, and you only download a single request and not page B. Sounds good. But there's another strategy, too. You could duplicate these shared modules and only make a single chunk per page.

So, basically, you have only one request now. You still download the same amount. You might even download a few bytes less because compression usually works better if you have larger chunks. If you split it up, it's worse compression-wise. But now you have the problem if you navigate from page A to page B, you download the shared module again. So, you basically waste a little bit of download size and that navigation. It's basically a tradeoff. And that's basically the main scheme here. But there are more metrics. Another metric is long-term caching, and I especially call it out here because often developers don't look into that much. It's a really interesting metric. It basically means if you do a code edit, maybe you change a line of code.

That basically means a module is changing, a module is different. That means the chunk that contains the module is different. And for the competed visitor of your application, so one user that was there before the code changed and now after, comes back. It means they have to download these changed chunks again, and it means they have to redownload all the modules that are in these chunks. And how can I affect that? If I can basically do smaller chunks, like putting less modules per chunk, obviously increases my request count.

But then I have better caching because fewer modules have to be downloaded. You could also be a little bit smarter by grouping up these modules by their change rate. So, you try to put modules that usually change together in a chunk. In the simplest case, it could be just put all your application code into App.js and put all your vendor code into Vendor.js. Now the frequently changing App code invalidates App.js, and only that is changing, and the Vendor.js file is only invalidated or has to be redownloaded by the user if you're doing a dependency upgrade, which you hopefully do a lot. So, you see, it's a lot of tradeoffs.

You could do more smaller chunks for better caching, but more requests. Alternatively, fewer, bigger chunks mean fewer requests and worse caching. Duplicating modules results in fewer requests but potentially longer download times. It's crucial to consider these tradeoffs as there's no perfect chunking strategy. Computationally, it can be expensive to analyze all these factors, especially in a large application with various request patterns and chunk groupings.

Chunking correctness is vital for delivering a correct application view. The delivery method of chunks heavily influences correctness. JavaScript chunks are typically delivered via script tags, where modules are wrapped in closures as module factories for deferred execution. This separation allows for benefits like module order independence and the prevention of duplicate executions. Overshipping modules is possible for efficiency without impacting execution. Handling CSS delivery, via link tags, differs as CSS files contain global rules affecting order and side effects, adding complexity to the chunking process.

That means what we do or a lot of bundlers do is they put a lot of modules into a chunk but they don't put it as modules, they basically wrap it in closures to make it like module factories and that basically means we register all the module factories on chunk load to a run time code on the page and then the run time can decide when to execute it. That's really important here, the loading aspect is separated from the execution aspect. So we can load it at one point and then later execute it or not execute it at all. And that gives us a lot of benefits.

An example, the order of the modules in the chunk doesn't matter, you can just put it in any order. We could also duplicate module factories so we can put a module in one chunk and another chunk. It will register twice at the one time but it will not execute it twice because the runtime is in chase of executing it when it really needs it and only executing it once. We can also overship modules, basically overshipping means you put a module there in a chunk which is actually not needed but it would just be the transfer to make chunking more efficient. And that also possibly you register the module factory at the one time but it will never execute it. That's totally fine.

It's getting more complicated when you look at other delivery methods like CSS. CSS is delivered via link tags and link tags are not a list of, it's not a container, CSS files are not a container of module factories, they are basically a list of CSS rules that are loaded in order and in order when you load your CSS files. So now everything is really global and has side effects and the order matters which makes a lot of stuff more complicated. The order matters of the stuff so we need to figure out in which order we place the modules in these chunks. You can no longer duplicate modules because if you duplicate style rules they basically appear in two different orders, that's breaking stuff.

You can sometimes overship modules, it depends on the type of styles you have. If you have CSS modules and they are isolated, have no side effects then you can overship them, but if you have global CSS it usually has side effects or could have side effects on your code and so it's not okay to just load it when the user didn't request it. So in a summary, CSS is super easy to chunk, you have all the freedom to do, but CSS makes everything super hard. And especially because of ordering. So when order matters, it's a hard problem and, yeah.

How to figure out the order of placing CSS files in your chunks in the first place? You could just think about, we could just do what the imports say, basically import order is the order of CSS loading, that works well as long as you only have a single chunk group. So if you only have a single page, no async imports and that stuff, that works well because then there's the defined import order.

But when you have multiple chunk groups and multiple chunk groups are loaded in any order, we don't know which page is visited first by the user and that stuff, they could all be loaded at the same time, page and async import is loaded at the same time. And so then in this case, there's a multiple defined import orders of all these chunk groups and in practice there's always conflicts between them, it's not possible to have always the same order of imports.

So that approach didn't work. The next approach could be like we say, we ask the user to be explicit about dependencies between CSS files, could be by using @import in a CSS file. But in practice, nobody does that and likes that and it's super impractical and existing CSS doesn't work with that, so it doesn't work.

You could try to analyze it from CSS, the JavaScript usage, how the CSS is used, but it's way too complicated and also not really possible. So we came up with an approach called implicit dependencies, so we analyze the whole application, look at all the chunk groups at once and see if we see something that module A or CSS A is always used before module B in all chunk groups, then we say oh, this is probably some dependency encodes that we think is there and we extract that and then we see like B depends on A. That's pretty cool because you get a clean dependency graph, there's no cycles, so it's easy to come up with an ordering from that.

But in practice, there are also issues with that. An example is what always is a problem here. So if you say always need to be then dependencies extracted. So if you have 1,000 pages which use A, B as order and then your new intern comes here and creates a new page which has an opposite order, you expect only this page to be broken but actually because we see it's not always used in that order, we don't extract this dependency and then your whole application can break due to that. So that is also not an issue. So the next approach would be to basically extract the most common order of CSS being used.

Like if many pages use A, B as order and only some pages use B, A, then we say A, B wins and that sounds good, but it's also really complicated due to you have to build a large graph of all orderings of all CSS in your application and then do some weird cycle analytics to break all the cycles and figure out which dependencies are most often used.

But let's assume that all works and we have some kind of ordering extracted of your application. Now we have to chunk it up with considering this ordering thing. If you have our two examples here again, legal page and pricing page. The legal page has a global CSS, a legal page specific CSS in the footer and the pricing has the same global CSS and some pricing page specific CSS files.

You see, if you want to keep the order always correct, you have to chunk it up into a lot of chunks. You basically have to split it too often which causes like in practice, it causes a lot of requests and for real applications with like thousands of chunk groups, it basically caused like nearly every CSS file be a separate chunk. So that doesn't really work. It causes too many requests, which really doesn't work for your application. And the problem with that is that really usually in the JavaScript world, you could just duplicate modules to solve to reduce the request. But because CSS is order dependent, we cannot do that. So our main tool for reducing request count is gone and we have to come up with a new tool for that.

And the new tool that helps here was overshipping so we can like we can overship modules. So include extra modules into chunks that don't need them to reduce the request count because then they can be reused more often. And that works really well. For example, we could technically reduce the whole application to a single chunk by overshipping a lot of CSS. So it would reduce it to only one request of that, but we can also be more granular. You can also move it like a little bit. And there's like basically a lot of approaches you can like find a right threshold of chunking. Again, it's a heuristic between download size and request count. You can balance between like less request counts, more download size or less download size and more request count. And it comes back to like finding a heuristic to figure it out.

In practice, it works well because often when you have a lot of CSS files, it's often a lot of CSS modules. And CSS modules are isolated so we can overship them. And they are also often small, which means like it's not that bad if you overship them. It doesn't cause like so much trouble. Yeah. And this whole balancing act is again driven by heuristics. So the conclusion of that, I really hate CSS ordering. It caused so much trouble in the past and like for years now. And I don't consider it like solved from that perspective. Like the whole ordering stuff is always problematic. But in general, I also see a lot of potential, a future potential of more optimizations we can do with chunking.

An example, we could like, currently we set every chunk group as like independent and equally in this sense of chunking, but this is not the case. Maybe your homepage is more important than your admin dashboard and you want to like optimize that. Maybe we can give bundlers hints that helps with like optimizing, special routes or like giving us more white in the optimization process. We could also be more generic by like just using analytics data from a production system to figure out how likely users visit our chunk groups or pages and optimize them based on that. So like if the homepage is the most often visited page, then maybe we should give that highest priority in the optimization.

Yeah. So thank you. Yeah, I have another, I have two minutes left, so I can come in with some extra stuff. It's also really tricky, like maybe, but another cool thing with chunking is compile time also. Because like you have this problem of you do that, you do, like, you have maybe this example here, I have a module graph, which like two pages are using a shared async import. So if you go back, go from the left side here from the legal page and chunk it up, it might create these blue chunks. But if you go from the right side and chunk it up, it might create like different chunks for this async import. So here you see the red module is only used in the entry point of the left page. So it doesn't need to be chunked for that version, but it needs to be chunked for the other page. So that creates a lot of problem with compile time. Because now we could either do what Wepek did, like worst case, bunch chunking. So we just chunk up the worst case of this shared async import. That works well, but it basically, like, overships it, or for the left page, it overships this red module. So it's not perfect from download size. But you could also do this all-path tracing, where you basically create two chunks for these async import and choose whatever you want, whatever fits best, depending on from which page you're coming. But this creates a problem of, like ... This is really a small example. It only creates two routes to this application. But in practice, if you implement this this way, you basically have to trace all your possible paths through your application, and it could create really weird ... If you have 10 nested async imports and a cycle with another 10 dynamic imports, it basically multiplies all that and creates a lot of work, creates a lot of output files, creates a lot of work for the bundler. So there's also a metric about compile time considerations about how you chunk up stuff. So a lot of things to consider when you do chunking. Yep. Thanks.

With the rise of React server components and server-driven frameworks like Next.js, how has the strategy for chunking evolved? Are we shifting complexity of code splitting from the client-side back to the server? Yeah, that's something I didn't consider for this talk. Examples were only like simple Motograph, but in practice, like, the Motograph is containing client components and server components. So everything is mixed in a single Motograph to do chunking across server and client boundaries, which creates a lot of problems and a lot of challenges. And yeah, I think a lot of stuff shifted, but we are also not optimal regarding client-side chunking. Currently, we always chunk all client components into the page, even if it's not needed. But technically, there's a lot of dynamic stuff from there.

So the server decides which client components to ship. It could be like any combination of chunk components you have to find in the application. So it's like a complicated process. I think there's a lot of future potential to optimize that by making it more smart, how to chunk these client components up to make it possible to not only transfer client components you actually need. So it's like, there's a lot of complexity we don't do yet, but there's a lot of future potential to do further optimizations here. Nice. Thank you.

The next one here, let's say, regarding resolving ordering of CSS, if there are five classes where A is dependent on B, and five classes where B is dependent on A, what are the results of the chunking? It's basically not to demonstrate, as currently we implemented this approach three, where it's expecting these implicit dependencies. And in this case, if it's not always the case, it's basically in any order. So it's not defined. We need to get some freedom from the module graph to decide to not have to chunk every module into a single chunk. So we spread it as we can decide which order it is and then it's in any order, basically, for the user. That is most often causing some problems if this is a problem. The problem is it's any order, but it's somehow deterministic. So it might be in this order for a year, and then you change some code that's technically independent of that, and that affects that, and then maybe the order changes. And that could break your application in weird ways. So that's tricky. A lot of tricky problems here.

Do you think bundlers will eventually become invisible infrastructure, or will developers always need to understand dependency graphs and build pipelines? I would say it's already mostly invisible infrastructure. But I think it's still valuable for the developer to know the concepts. That's why I'm trying to explain many of them. And hopefully it's mostly transparent to you, and you don't have to understand it and it just works, but it often is cool to understand it and helps in general. How are you going to get people to understand it if they won't even read their code these days? Yeah. I'm not sure what the world of AI says to that. Yeah, yeah. All right. Yes, maybe an attention to that. We try to offer a lot of tools to inspect the output, like we have a bundlerizer that gives you a lot of insight into your bundling output. It's mostly not the chunking that is relevant, but the module graph you're looking at and which modules to ship. And I think there's value in looking into that and understanding why your application is so large or why are you shipping whatever big libraries five times in different versions over there.

So we try to offer more tooling for that and also give more AI access to this tooling to be able to figure out these cases and analyze their stuff. Nice. I think this would be, what would be the order of unimportant... Okay. I'm sorry. I'm having a hard time reading this one. Would the order be unimportant for CSS when you use utility classes like Tailwind? Tailwind is a good workaround to this problem of CSS chunking or ordering, because basically CSS takes your application and generates a single CSS file from Bundler's perspective. So a single CSS file, and so no order is important because it's a single file from our perspective. And Tailwind basically handles the ordering themselves by having just a fixed order of these classes that occur in that order. So that is pretty good for our perspective, because then we don't have to care about ordering again and it also helps with that stuff. So Tailwind is the workaround, or a solution, workaround, workaround slash solution for this ordering problem. And yeah, all the atomic CSS solutions are basically that in that sense. It has trade-offs, again. Tailwind, because it does a single file, it basically breaks all your tree-shaking stuff. So you get only the classes you use in your application, but it's not page-specific, it's not chunk group-specific. So on your homepage, you're now loading all the CSS classes you need on your admin dashboard, and that's also not optimized. So it's a trade-off again. As always, everything is a trade-off. Yeah, right. Yeah, exactly. Have you explored topological sorting for CSS ordering versus request count? Yeah, so we do topological sorting after we figured out a non-cyclic graph of that. So the first thing is to get a non-cyclic graph to do topological ordering. But if you have multiple chunk groups, you don't have a non-cyclic graph in the first hand. The slides I was showing basically show you the process to figure out a non-cyclic graph, and then on this graph, we do topological ordering, but it's pretty simple then. What about layers in CSS? So it says now that layers are widely available in CSS, could they not help greatly optimize CSS? I thought the same, and we tried that, and it didn't work, because layers, if you put every CSS module into its own layer, and then just do ordering on the layers, it breaks all these CSS selector preference stuff, because then it basically doesn't work anymore, because you cannot style something with having different selector precedence and that stuff, so it breaks that kind of thing. So layers, sadly, didn't work. I thought the same, and we implemented it, actually, but then figured out that it doesn't work. Yeah, it's wild that you have to approach all of these problems and go down. This is so much complexity, right? Let's see. Regarding the order of CSS, how common is the issue if you write your styles well, or do you have common examples that would affect most people? Yeah, so if you're using the approach 3 currently by default, and that depends on you writing your styles well, and it works if you write your styles well, but, yeah, that's a problem.

I thought one mistake can break it all. If you, at some point, in some page, like in your 1,000 pages app, write it not well, then it could break and could affect other pages, and that's kind of not what we want. We currently do that, but it has issues, and I wouldn't say it's served, but some people run into that and broke their application, and we also did that. That's why I look into more solutions to that.

Yeah, we can do one more here. Let me see. If CSS ordering is fundamentally a dependency graph problem, do you think future CSS features should provide native dependency declarations instead of relying on tooling to infer them? I mean, we already provide that. Add, import, exist in CSS. We just don't use it. I think what would really help is something where you don't rely on ordering at all. There was a proposal for that for Web Components, I think, but it was not accepted or something. It would be cool if you get a styling solution that doesn't depend on ordering. That would be cool. I think many people would be happy with that.

Well, thank you so much, Tobias. Yeah, thank you for having me. Awesome talk.