Bundlers: A Deep Dive into Modern JavaScript Build Tools

Hello, everyone. I'm Edoardo. I'm a DevRel at Storyblok. And in this talk, I will guide you into a deep dive, an exploration of the world of JavaScript bundlers. We will see why we needed to make bundlers in the first place. Then we will try to learn by coding, learn by making a simple bundle. I think the best way to learn something is to actually do it. So we will see how we can do it.

I will share the final result of this in a GitHub repo. So don't worry about studying all the snippets that we share. Don't worry. It's just to introduce you to some basic concepts. When we know the basics, we can then understand some more advanced concepts. And in the final part, we will see why Vite is different from Vite.

First of all, in Storyblok in the last few months, we were in a process of migrating all of our JavaScript SDKs and libraries to Vite. And this was really interesting to me. So I learned a lot. I learned about bundlers and about Vite. I suggest you to check out the GitHub page of Storyblok when you can see all our JavaScript libraries. And how we use and we configure Vite. And in this process, I discovered about Vite. And this inspired me in making this talk, basically.

Storyblok also actively supports the Vite project and the Vite community. So I can only thank Storyblok for that.

Then let's start with an example. This is like an old JavaScript API. This is like an old JavaScript application. When we load every part of our app together into an index.html. So we have a web application. And you can see that we load every JavaScript file here. Because this was the thing at the time.

What is the problem? It's not important what this application is doing. It's just like logging something. But what's important is to see that what we are using from other modules, other files. We cannot predict. Because like this comment, for example. Which result are we getting? With this format date function. Which format data are we using? There's no import at the top. Everything is global. So here we have functions. We have format functions. We have variable. But we don't know what we are using. Right? So with this first example, we have problems. We have problems with the variable name conflict. With the function overriding. That the global namespace is polluted. And we cannot really control the dependency order. But we know that all these problems are solved using ES modules.

So let me share this second app where the index.js, we load that single file. But with the type module attribute. With the type module attribute, the browser knows that index.js is using ES modules. In fact, the index.js business logic is the same. But now we are using import statements. So we are sure that, for example, here the app name comes from utils. Format data as well. And we have a format date which is overridden. But we alias it with a different name. So we can control the things that we are importing. And also the dependency order. And everything just works.

Okay? But we need to check the browser compatibility for that. We can be sure that it's supported in all the major browsers. So for compatibility, it's fine. For performance, though, there are some issues. Because for every single file that we are loading in our app, the browser makes an HTTP call. So we have multiple HTTP calls. And this is more important. When we just need one function or one thing from a module, the browser needs to load the whole library, the whole file, which includes that single function that we need. And this is not okay for performance.

Okay? We need to get here. This is these are the first lines of the real story block JS client library that you can see on GitHub. And this is the bundle. So in this file, you see that everything is bundled together and is optimized. And we just have a few files that the browser needs to load. So we want to get here. So let me introduce you to the mini-bundler. The mini-bundler is what we can call a bundle. Because this time in our index.html, we load the bundle JS file from the dist folder.

So we can expect that everything is packed together in this bundle. So if I show you the bundle from the dist folder, you see here that we have a single folder with everything inside.

How we did that? The basic concept is that we need to understand the dependencies in our application. So we need to start from an entry point. In most cases, it would be the main.js or the index.js. We start from the entry point. And then we start crawling to build a dependency graph.

When we have a structure that represents our dependency graph, then we can generate a bundle from it. And then we can just take the bundle and write it to a file. Not much more. Like, we need just functions or methods to read module contents, to build a graph, to transform the module content.

We need to transform because we have ES modules, so we have imports and exports. But in the final bundle, we need to take the full function and put it into the module content. Without relying on external dependencies. Because we handle everything. So we need to transform the module content. And it's actually very simple.

The final result is this. I wanted to show you that in the bundle file, the structure that we use to represent the dependency graph is just an object. And entries in this object is the path to the modules. And for each entry, we have an ID, we have the file path, the content, the full content of the module or the file. And dependencies, imports and exports.

So we define and describe every single module of our application here in this object, which is our dependency graph. And from that, we build our bundle. The rest of the simple bundle is really simple, actually. You see that we are using RegExp to take imports and exports. And this is, of course, a simplified approach. And then we transform.

One interesting part is that when we write the content in the bundle file, we wrap everything into an auto executing function. Because then the browser knows that they can just execute what's in the file. And inside this, we start by defining a required function. And the required function is like a simplified common JS require. With this, we take functions or variables from other files and we put them into the file that we are reading or executing. And with this simple approach, we can put everything together and we can tell the browser what to do. In fact, if you look at the code, we are just like defining the require here and then starting from the index.

So from the entry point, we are taking, for example, the format day, which is a function using the require from a path and take the function. This is just like the common JS. And then we have the content of the modules. We pack everything into the module content and then we have the business logic and we execute it. This for every single module in our app. Everything is in a single file, right? I hope this is simple to understand.

This approach is really similar to the original webpack. In fact, if you look at the bundle produced by webpack, it's really similar to what we have. You see, you have the webpack require on top. You have an auto executing function and you have the dependency graph. Modern bundles, in comparison, have a more sophisticated module system and more sophisticated stuff with webpack 5. Roll up is more optimized for ES modules and ES built is written in Go. So it has better performances and it can leverage on parallel processing. It also have a native support for TypeScript. These are the three like most common bundlers.

These are the three like most common bundlers. OK, but wait, how we are parsing code again. Remember, we are using RegExp for imports and exports, but this is not good. So we need to introduce a slightly more advanced concept, which is the abstract syntax tree or AST. The AST is a data structure that we can use to represent a program or a code snippet is a tree representation of the syntactic structure of a text. It can be a source code. OK. And this formal representation is written in a formal language. So each node is each node is a construct in our text.

For example, here you have a very simple JavaScript code, but we translate this to the abstract syntax tree. And and this structure is very easy for a machine to crawl. So the machine takes node by node and can easily pass this. If you want a more human representation. Here we are. This is the abstract syntax tree of the code, the snippet of code that we saw. So we have parsers, we have widely used parsers in the JavaScript world. We have Acorn, which is very lightweight and we have the Babel parser, which is more advanced. Acorn for using Acorn, you just require an Acorn and you call the parse method from Acorn, you get the AST.

So I want to introduce you to the simple bundler, which is the final version of our project. And of this version, you will find the code on GitHub. This is a bit more complex, so I cannot guide you through the code. But the the core modification is that we now pass the content using Acorn. So in the read module, instead of relying on imports and exports, like EXP, now we have we transform the module content with AST. And we use the AST to traverse it to extract imports and exports and also to transform the code. This is the the the core modification that we have. So since we want to get here, remember where we was with our simple bundler, with our mini bundler, we were here. We have this bundle. We want to be to have this one. So we don't only need to change the way we transform, but also we need to minify this into this thing. To do that, I added another dependency, which is Tercer, which is another plugin which uses Acorn to minify things.

And the minification process also optimizes the bundle. So in our final simple bundle, you can check the code again on GitHub. All of this is implemented. Also, I implemented a way to use plugins, which is again similar to what Webpack is doing with using hooks. So we have hooks at specific steps in our code. And in these steps, we can call other functions. For example, I implemented a plugin that uses Tercer, implements the bundle hook, and then we call the minify from Tercer to actually minify the final bundle. So this is the bundle step is in the end of the process. For everything else is what we what we saw in the in the slides. So more or less is like the mini bundler, but a bit more complex and using the Acorn AST to do stuff and to analyze code. But again, feel free to go to GitHub and play with this and look at this.

OK, so this is the final result. Every file in my application is now packed and bundled into one single file, which is this one. Now some more advanced concept, the tree shaking. Tree shaking is dead code elimination. So basically, if we have code that we cannot reach or we don't use in the final app, we just take it away from the bundle. So, for example, in this case, in Nutils, we have subtract and divide that we don't use in the app. So the final bundle will not contain that function. We strip them away. So this is shaking the tree and dead branches just come out. Then we have code splitting. You see now a single bundle with everything. We can have separate bundles of code splitting is the practice of splitting the code in multiple chunks. And we can do that with this simple pseudocode. When we add the new module, which is generate chunks and for each dynamic import. When we have a dynamic import, we just call a new function. We build a new dependency graph and we generate a new chunk. So we define a dist folder will be made of the bundle and then chunks, which are different files.

Then another concept, the hot module replacement. Modern bundles all have hot module replacement. Is when you are working on your app locally and you change the source code. When you change the source code, you don't want to reload the whole application, but just the part that you changed. With hot module replacement, you just tell the browser to reload just that single part. And we can do that using a watcher and using sockets. So with sockets, we just send to the browser a JSON. For example, when we tell the browser to replace only a specific chunk or a specific module.

Now for that final part, I can introduce you to VIT. Knowing everything that now we know, we can understand that VIT is not exactly a bundler. In fact, they call themselves a no bundler. Why? Because VIT is like an orchestrator of bundlers. VIT is using and optimizing other bundlers. So other bundlers are dependencies for VIT. In VIT, we have the dev mode and the production mode. The dev mode is using ES build. So when we build things in dev mode, we are using ES build. And we have hot module replacement. We have pre-bundling of dependencies. We have optimizations. In production mode, instead, we have rollup. VIT is using rollup.

So the final bundle for production is heavily optimized by rollup and by opinionated configurations made by the VIT project. This is a simple comparison between VIT and a simple bundle in the final production app. I mentioned that the dependency pre-bundle, if you look at VIT when you are in development mode, you will see this in the console, pre-bundling dependencies. This means that the dependencies of your app are getting optimized. In fact, when you look at the final production and the bundled code, you will see that, for example, use state is not coming from React, but from modules React, because it's pre-bundled and optimized by VIT. Or here, for example, in production, we have here we have configurations opinionated with rollup. Instead, in development mode, we have a web server. Sorry.

Now with VIT, we get a faster dev startup. We have better dev performance and we have native ESM support. Of course, we also have more aggregates of maps, so the development experience with VIT is really better and they opinionated, they use the opinionated approach to pre-configure and pre-optimize your bundles for you. We also have a better HMR. So for development and production mode, everything is optimized and better. We need to understand that also we need modern browsers to fully leverage and to fully use the features of VIT.

That there are differences in the dev bundle and the production bundle, because we are using different bundles. And in development mode, we still have many HTTP requests. So please keep in mind these limitations. There are not limitations, but keep in mind that.

Again, this is me. You can find me on Mastodon. You can find me on my YouTube channel. You can find me on my LinkedIn if you want to contact me. I'm available to answer all your questions. You can join the Storyblok community. We have our Discord and we have our ambassadors. Check this page. And as I promised, the Simple Bundler code is on GitHub.