Chunking

My name is Tobias Koppers, I work at Vercel as the TurboPack team, and today another Bundler topic, chunking. So what is chunking at all? So my definition of chunking is it's a process in a Bundler that places module into output files, and I usually call these output files chunks because, whatever, I came up with this name. So it's basically like we take the module graph of your application, decide in some weird way how to put it into chunks, and that's chunking the process.

If you visualize it, there could be a really simple example here. So we have a module graph, a really small one for this one. So we have a bunch of modules, these white dots, and these modules are usually connected via dependencies, like an import or a require or an async import, and these are the edges of the graph. And I realized a really special edge here, because it's really special for chunking, is this red dashed one. That's an async import, so like a dynamic import or something, where you load something on demand.

Also, usually, your module graph has a lot of empty points, maybe your pages or that stuff. And here, the most important pages of an application is, of course, the legal page and the pricing page. And you can basically visualize chunking in a way that you just draw cycles around these modules. And yeah, I did this with the screen cycles, and basically means all these modules that are cycled together are put into one output file. And an example, like this async import here on the left side, with the red dotted thing, the dashed thing, is put into its own chunks so we can load it on demand.

And we have all the entry points being its own chunks, because then we can load them independent of the other pages. And you can also, like in this example, put the middle area of modules that are shared by these two entry points into its own chunks so we can share that. But yeah, it's really complicated. You can come up with a lot of different ways to cycle these modules. And yeah, that's the process of chunking, not of cycling. And how to do chunking in a good way.

Also, yeah, one thing is you can look at metrics. So most often we care about the performance metrics. So like these metrics influence how we chunk up stuff. And before we're looking into the metrics, we want to look into another term, and that is chunk groups. And I say chunk groups are basically a set of chunks that are loaded together. That could be on an initial page load, you load like these 10 chunks in script tags, and that's an initial load chunk group. But there's also like other types of chunk groups. Like on a dynamic import, there's like an async loading chunk group where you load these chunks together as a group. Or a navigation between two pages is also like a chunk group.

And we really want to look at a few metrics, and one metric is the download size. And with download size, I don't mean bundle size, because people often see bundle size as this is the size of the output folder that the bundler produces. But the total size of that folder rarely matters. It might matter for like compile time of your application, but it doesn't matter for like the user perceived performance. What really matters is like the download size per chunk group. So like these initial page load has whatever bytes downloaded, or these on-demand loading of whatever has bytes downloaded, or this navigation has that much bytes downloaded. So we really, really want to look at these metrics on a sense of per chunk group.

And similar for the second metric which is request count. Request count is obviously not good if you do a lot of requests, because every request has an overhead. Even with HTTP2 there's still an overhead, just fewer than before, but it's still not good to load like millions of chunks, millions of files in your application. And now we have two metrics already.

And if you look at an example, here's a really small one, I have two pages, A and B. And they both share like a shared module. And now you can come up with like different strategies of doing chunking here. So one strategy might be you put like page-specific modules into the page chunks, like the blue and the green one, and then make a common chunk where you put like the shared modules in it.

But there are more metrics, and another metric is long-term caching. And I especially call it out here because often developers don't look into that much, and it's a really interesting metric. It basically means if you do a code edit, maybe you change a line of code. That basically means a module is changing, a module is different. That means the chunk that contains the module is different. And for the repeated visitor of your application, so one user that was there before the code change and now after, comes back.

You still download the same amount. You might even download a few bytes less because like compression usually works better if you have larger chunks. If you split it up, it's like basically worse compression-wise. But now you have the problem, if you navigate from page A to page B, you download the shared module again. So you basically waste a little bit of download size and that navigation. So it's basically a trade-off, and that's basically the main scheme here.

But there are more metrics, and another metric is long-term caching. I especially call it out here because often developers don't look into that much, and it's a really interesting metric. It basically means if you do a code edit, maybe you change a line of code. That basically means a module is changing, a module is different. That means the chunk that contains the module is different. And for the repeated visitor of your application, so one user that was there before the code change and now after, comes back. It means they have to download these changed chunks again, and it means they have to re-download all the modules that ended in these chunks.

How can I affect that? I can basically do smaller chunks, like putting fewer modules per chunk. Obviously, it increases my request count, but then I have better caching because fewer modules have to be re-downloaded. You could also be a little bit smarter by grouping up these modules by their change rate. So you try to put modules that usually change together, together in a chunk. In the simplest case, it could be just like put all your application code into app.js and put all your vendor code into vendor.js. Now the frequently changing app code invalidates app.js, and only that is changing, and the vendor.js file is only invalidated or has to be re-downloaded by the user if you're doing a dependency upgrade, which you hopefully do a lot.

So you see, there are a lot of trade-offs. You could do more, smaller chunks. That means better caching, but more requests. You could do less, bigger chunks, that's fewer requests and worse caching. You can duplicate modules, that means you have fewer requests, but have larger download in some cases. So that's basically not a perfect chunking you can do. It's basically a lot of trade-offs you have to consider. And it's sometimes also really expensive to compute all these heuristics on a really large application because there are so many weird ways you can request and go for chunk groups and that stuff.

But let's look into another part of the whole chunking thing. Chunking correctness. So we must produce something that is correct from the application point of view. And that heavily depends on the delivery method of our chunks. So usually the chunks you know is like JavaScript chunks, we deliver via script tags. And that means what we do, or a lot of bundlers do, is they put a lot of modules into a chunk, but they don't put it as modules, they basically wrap it in closures to make it module factories. And that basically means we register all the module factories on chunk load to a runtime code on the page, and then the runtime can decide when to execute it. And that's really important here, like the loading aspect is separated from the execution aspect.

So we can load it at one point and then later execute it, or not execute at all. And that gives us a lot of benefits. An example, the order of the modules in the chunk doesn't matter, you can just put it in any order. We could also duplicate module factories, so we can put a module in one chunk and another chunk. It will register twice at the runtime, but it will not execute it twice, because the runtime is in the chase of executing it when it really needs it and only executing it once. We can also overship modules, so basically overshipping means you put a module there in a chunk which is actually not needed, but it would just be the transfer to make chunking more efficient. And that's also possible, you register the module factory at the runtime but it will never execute it, that's totally fine. It's getting more complicated when you look at other delivery methods like CSS.

CSS is delivered via link tags, and link tags are not a list of... CSS files are not a container of module factories, they are basically a list of CSS rules that are loaded in order, and in order in order which you load your CSS files. So now everything is really global and has side-effects and the order matters, which makes a lot of stuff more complicated. So the order matters of the stuff, so we need to figure out in which order we place the modules in these chunks. You can no longer duplicate modules, because if you duplicate style rules, they basically appear in two different orders, that's breaking stuff.

You can sometimes overship modules, it depends on the type of styles you have. If you have CSS modules, they are isolated, have no side-effects, then you can overship them, but if you have global CSS it usually has side-effects or could have side-effects on your code, so it's not okay to just load it when the user didn't request it. In a summary, CSS is super easy to chunk, you have all the freedom to do, but CSS makes everything super hard. And especially because of ordering. And so when order matters, it's a hard problem. So how to figure out the order of placing CSS files in your chunks in the first place?

You could just think about, you could just do what the imports say, basically import order is the order of CSS loading. That works well, as long as you only have a single chunk group, if you only have a single page, no async imports and that stuff, that works well, because then there's a defined import order. But when you have multiple chunk groups and multiple chunk groups are loaded in any order, we don't know which page is visited first by the user and that stuff, they could all be loaded at the same time, like page and async import is loaded at the same time. And so in this case, there's a multiple defined import order of all these chunk groups, and in practice, there's always conflicts between them. It's not possible to have always the same order of imports.

So that approach didn't work. The next approach could be like we say, we asked the user to be explicit about dependencies between CSS files, could be by using at import in a CSS file. But in practice, nobody does that and likes that and it's super impractical and existing CSS doesn't work with that, so it doesn't work. You could try to analyze that from CSS, the JavaScript usage, how the CSS is used, but it's way too complicated and also not really possible.

So we came up with imports called implicit dependencies. We analyzed the whole application, look at all the chunk groups at once, and see if we see something that module A or CSS A is always used before module B, and in all chunk groups, then we say, oh, this is probably some dependency in code that we think is there and we extract that and then we see like, B depends on A. And that's pretty cool because you get a clean dependency graph. There's no cycles, so it's easy to come up with an ordering from that.

But in practice, there are also issues with that. An example is what always is a problem here. If you say always need to be then dependencies extracted, so if you have 1,000 pages which uses A, B as order and then your new intern comes here and creates a new page which has an opposite order, you expect only this page to be broken. But actually because we see it's not always used in that order, we don't extract this dependency and then your whole application can break due to that. So that is also not an, it has issues. And so the next approach would be to basically extract the most common order of CSS being If many pages use A, B as order and only some pages use B, A, then we say A, B wins.

Yeah, but let's assume that all works. And we have some kind of ordering extracted of your application. Now we have to chunk it up with considering this ordering thing. So if you have our two examples here again, legal page and pricing page, and like the legal page has a global CSS, a legal space specific CSS and a footer and the pricing is like the same global CSS and some specific pricing page specific CSS files. And you see like, if you want to keep the order always correct, you have to chunk it up into like a lot of chunks. You basically have to split too often, which causes like in practice, it causes a lot of requests. And for real applications, it's like thousands of chunk groups. It basically costs like nearly every CSS file be a separate chunk. So that doesn't really work. It causes too many requests, which really doesn't work for your application.

And the problem is that is that we really, usually in JavaScript, we could just duplicate modules to solve that, to reduce them request. But because CSS is auto dependent, we cannot do that. So our main tool for reducing request count is gone. And we have to come up with a new tool for that. And the new tool that helps here was overshipping. So we can like, we can overship modules. So include extra modules into chunks that don't need them to reduce the request count, because then they can be reused more often. And that works really well. For example, we could technically reduce the whole application to a single chunk by overshipping a lot of CSS. And so it would reduce it to only one request of that.

But we can also be more granular. You can also move it like a little bit. And there's like basically a lot of approaches you can like fight a wide threshold of chunking. Again, it's a heuristic between download size and request count. You can balance between like less request counts, more download size or less download size and more request count. And it comes back to like finding a heuristic to figure it out. In practice, it works well because often when you have a lot of CSS files, it's often a lot of CSS modules. And CSS modules are isolated so we can overship them. And they are also often small, which means like it's not that bad if you overship them. It doesn't cause like so much trouble.

And this whole balancing act is again driven by heuristics. So the conclusion of that, I really hate CSS ordering. It caused so much trouble in the past and like for years now. And I don't consider it like solved from that perspective. Like the whole ordering stuff is always problematic.

But in general, I also see a lot of potential, a future potential of more optimizations we can do with chunking. An example, we could like currently we set every chunk group as like independent and equally in this sense of chunking, but this is not the case. Maybe your homepage is more important than your admin dashboard and you want to like optimize that.

Maybe we can give bundlers hints that help with like optimizing special routes or giving this more weight in the optimization process. We could also be more generic by just using analytics data from a production system to figure out how likely users visit our chunk groups or our pages and optimize them based on that. So like if the homepage is the most often visited page, then maybe we should give that the highest priority and optimization. Yeah. So thank you. Yeah. I have two minutes left so I can comment with some extra stuff.

It's also really tricky, like maybe another cool thing with chunking is compile time also. Because like you have this problem of you do that, you do like you have maybe this example here, I have a Motograph, which like two pages are using a shared async import. So if you go back, go from the left side here from the legal page and chunk it up, it might create these blue chunks. But if you go from the right side and chunk it up, it might create different chunks for this async import. So here you see the red module is only used in the entry point of the left page. So it doesn't need to be chunked for that version, but it needs to be chunked for the other page. So that creates a lot of problems with compile time. Because now we could either do what Repec did, like worst case bunch chunking. So we just chunk up the worst case of this shared async import. That works well, but it basically like overships, for the left page, it overships this red module. So it's not perfect from download size. But you could also like do like this all path tracing, where you're like basically create two chunks for these async import and choose whatever fits best, depending on from which page you're coming. But this creates a problem of like, this is a really small example, it only creates like two words to this application. But in practice, like if you implement this this way, you basically have to trace all your possible paths through your application. And it could like create really weird, if you have like 10 nested async import, then you have a cycle with like another 10 dynamic imports that basically multiplies all that and creates a lot of work, creates a lot of output files, create a lot of work for the bundler. So there's also like a metric about compile time considerations about how you chunk up stuff.

So a lot of things to consider when you do chunking. Yep. Thanks. With the rise of a React server components and server-driven frameworks like Next.js, how is the strategy for chunking evolved? Are we shifting complexity of code splitting from the client side back to the server? Yeah, that's something I didn't consider for this talk. Examples were only like simple Motograph. But in practice, like the Motograph is containing client components and server components. So everything is mixed in a single Motograph. So you do chunking cross server and client boundaries, which creates a lot of problems and like a lot of challenges. And yeah, I think a lot of stuff shifted, but we also like not optimal regarding client-side chunking. Currently, we chunk all the client components into the page, even if it's not needed. But technically, there's a lot of dynamic stuff from there. The server decides which client components to ship. It could be like any combinations of chunked components you actually have to find in the application. So it's like a complicated process.

I think there's a lot of future potential to optimize that by making it more smart, which how to chunk these client components up to like make it possible to not only transfer client components you actually need. So it's like, there's a lot of complexity we don't do yet, but there's a lot of future potential to do further optimizations here. Nice. Thank you.

All right. We have another one here. Let's say regarding resolving ordering of CSS, if there are five classes where A is dependent on B and five classes where B is dependent on A, what are the results of the chunking? It's basically not to diminish. Currently we implemented this part three, where like it's extracting this implicit dependencies. And in this case, if it's not always the case, it's basically in any order. So it's not defined. We need to get some freedom from the module graph to decide to not have to chunk every module into a single chunk. So we set it as like we can decide which order it is. And then it's in any order basically for the user that is most often causing some problems if like this is a problem.

It's any order, but like it's somehow deterministic. So it might be in this order for like a year and then you change some code that's technically independent of that and that affects that. And then maybe the order changes and that could break your application in weird ways. So that's tricky. A lot of tricky problems here. Do you think bundlers will eventually become invisible infrastructure or will developers always need to understand dependency graphs and build pipelines? I've seen it already mostly in visible infrastructure. But I think it's still valuable for the developer to know the concepts. That's why I'm trying to explain many of them and hopefully it's mostly transparent to you and you don't have to understand it and it just works. But it often is cool to understand it and helps in general.

How are you going to get people to understand it if they won't even read their code these days? I'm not sure what the world of AI says to that. Yeah. All right. Maybe an attention to that. So like we try to offer a lot of tools to like inspect the output like we have a bundle analyzer that gives you a lot of insight into your bundling output. It's mostly not the chunking that is relevant, but like the module graph you're looking at and which modules to ship. And I think there's value in looking into that and seeing, understanding why your application is so large or why are you shipping whatever big libraries five times in different versions over there. And so we try to offer more tooling for that and also give more AI access to this tooling to be able to like figure out like these cases and analyze their stuff.

Would the order... I think this would be what would be the order of unimportant... Okay. I'm sorry. I'm having a hard time reading this one. Would the order be unimportant for CSS when you use utility classes like Tailwind? Tailwind is a good workaround to this problem of CSS chunking or ordering. Because basically CSS takes your application and generates a single CSS file from Bundler's perspective. So like a single CSS file and so no order is important because it's a single file from our perspective. And Tailwind basically handles the ordering themselves by having just a fixed order of these classes that occur in that order. So that is pretty good for our perspective because then we don't have to care about ordering again and it also helps with that stuff. So Tailwind is the workaround or a solution workaround, workaround/solution for this ordering problem. And yeah. All the atomic CSS solutions are basically that in that sense. But it has tradeoffs again. Like Tailwind, basically because it does a single file, it basically breaks all your tree-shaking stuff. So you get only the classes you use in your application, but it's not page-specific. It's not chunk group specific. So like on your homepage, you're now loading all the CSS classes you need on your admin dashboard and that's also not optimized. So it's like a tradeoff again. It's like as always, everything is a tradeoff. Yeah. Right.

Yeah. Exactly. So you explored topological sorting for CSS ordering versus request count. Yeah. So we do topological sorting after we figured out a non-cyclic graph of that. So the first need is basically a non-cyclic graph to do topological ordering. But if you have multiple chunk groups, you don't have a non-cyclic graph in the first hand. So the slides I was showing basically show you the process to figure out a non-cyclic graph, and then on this graph we do topological ordering, but it's pretty simple then.

What about layers in CSS? So it says now that layers are widely available in CSS, could they not help greatly optimize CSS? I thought the same. And we tried that and it didn't work because layers, if you put every CSS module into an own layer and then just do ordering on those layers, it breaks all this CSS selector preference stuff because then it basically doesn't work anymore. Because you cannot style something with different selector precedents and that stuff, so it breaks that kind of thing. So layers, sadly, didn't work. I thought the same and we implemented it actually, but then figured out that it doesn't work. Yeah, that's why you have to approach all of these problems and go down. It's so much complexity, right?

Regarding the order of CSS, how common is the issue if you write your styles well? Or do you have common examples that would affect most people? Yeah, so we're using Approach 3 currently by default, and that depends on you writing your styles well. And it works if you write your styles well. But yeah, that's the problem. I thought one mistake can break it all. If you, at some point, in some page, like in your 1000 pages app, write it not well, then it could break and could affect other pages and that's not what we want. We currently do that, but it has issues. And I wouldn't say it's like surf, but some people run into that and broke the application and we also did that. And that's why I'm looking into more solutions to that. Yeah, we can do one more here. Let me see.