The Superpower of ASTs: How We Saved 16% on Our Bundle Size

♪♪ ♪♪ ♪♪ Take a moment to think what this means to you. So you're probably thinking that this is a bundle, but you're most likely thinking that this is painful, because we, as front-end developers, spend a lot of time and effort on reducing it as much as possible. Our story is about a super tool we used to reduce the bundle size, and it took us from here to there without changing one line of code. What if you, too, could have a super tool that would automatically remove 4,250 input statements while improving your performance by 16% more than Terza's tree shaking, which is considered the state of the art regarding to bundle optimization? Also, it will reduce 14 megabytes from your bundle size, which is really a lot. Hi, I'm Omri Lavi. I'm a tech lead at Monday's DevTools group. This means I'm building tools for developers so they can work with higher velocity and with higher confidence. I'm also part of the Foundations group, so this gives me unique opportunities to work on some really interesting stuff like the one I'll talk about today. To give you some background, at Monday.com, we have a large and complex client code. We have more than 300 developers changing the client code on a daily basis, and we have millions of users, so each kilobyte we save matters a lot. This is why we look for innovative ways to reduce the bundle size as much as possible.

And a while ago, we noticed the following pattern. We saw that we have many files that look like this. They exported nested objects that in the end of the day only contained strings. This is why we call them constants files.

And the files that referenced the constants files only used one or two values out of these constants. So you're probably thinking to yourself, okay, I have Tree Shaking exactly for this case, and if you're unfamiliar with Tree Shaking, then it's a very common algorithm that most bundlers use today to remove the unused values from your bundle, so you'll end up exactly with what you need in your bundle. But in our case, Tree Shaking didn't work properly because we used objects, and Tree Shaking doesn't take care well with objects. So we ended up with something like this in our bundle, and you see that all the red parts are actually constants files.

And the problem with that is that we have many constants files inside our bundle. This means we have more JavaScript than what we actually needed. This increased our bundle's size and made it more network-heavy, so our application became heavier to load. Additionally, all of these files were kept in memory and were never removed, so it increased our memory footprint more than what we wanted. And it felt a lot like this. We had a small piece of code that carried a huge luggage with it.

So we had a dream. We had a dream that while we're building the bundle, we'll find the reference values out of the constants files, remove the references, and replace them with the actual values, and then we can remove the import for these constants files and end up exactly with the values that we need inside our bundle. So we knew what we wanted to do, but we weren't sure how to do it. We knew that we wanted to avoid major refactors because we had a lot of code that needed to be updated, and we knew we wanted to avoid manual update of the code because, again, it was a lot of code, and we needed to do it programmatically to avoid errors that humans can do. So after some thinking, we understood we can use ASTs, but wait, ASTs aren't too common, right? Think for a moment if you've ever used ASTs before.

So the truth is that we all use ASTs, and we just may not know it, and we'll see it in a second. So ASTs are abstract syntax trees. It's a data structure used to represent a piece of code, and the best way to understand it is by a tool called AST Explorer. AST Explorer is a free online tool, and you can put a piece of code in it, like the one here, and it will show you its AST. It may seem scary on the first glance. It really is, but the power of AST Explorer is that you can select a piece of code, and it will show you its matching node in the AST. So for example, you can see here we have a string literal node with the value of hey, Berlin. We can also select this part here to see it's a variable declaration node with the name of hi, the value of hey, Berlin, and we can also see its kind is a const. It's important to understand that ASTs really everyone. Many tools use AST. If you think of ESLint, PostCSS, Prettier, Webpack, and many other tools, they all use AST to read and transform code programmatically.

Let's take, for example, how Prettier is working. Let's say I have this piece of code that uses single quotes, and when I save the file, then Prettier runs behind the scenes and updates the file and updates the code to be like this. The result is that it has double quotes. And the way it's working behind the scenes is that it has this piece of code that says if you're seeing a string node with a quote type of single, then set its quote type to be double. And this part here is actually AST code. This is the magic and the engine behind Prettier that allows it to be so powerful.

So we knew we needed ASTs, but which tool, which framework should we use? We knew we needed something that will allow us to manipulate code using ASTs. We wanted the tool to be popular, so we'll be able to enjoy the community it comes with. We wanted it to be flexible, so it will allow us to do exactly what we want. And lastly, we wanted it to be pluggable, so we'll be able to use it in our existing ecosystem without making any major changes. So after some research, we decided to use Bubble. And Bubble has all of these characteristics, and you may recognize it from your web configuration, where it's very popular.

Bubble knows how to read code, parse it into AST, then it can run plugins to generate new AST out of it. This is where you can step in and write your own plugin that will go over nodes, mutate them, insert new ones, or remove existing ones, and then Bubble will take the new AST and create new code out of it. So now that we understand how Bubble works, let's get back to our original problem. After some work, we had a working Bubble plugin implementation, and it worked something like this. First, it found all of the reference identifiers. All of these identifiers are references for identifiers, for example, variables or functions and stuff like that. Out of these references, we found those that are coming from constants files. Then we calculated which exact values are being referenced, inline them instead of the references, and then we could remove the input statements along with the files, the constants files, to end up exactly with the code that we need.

This sounds very simple, right? What can go wrong with such implementation? The best way to understand it is by implementing something ourselves. Let's build a mini plugin that does something very similar. Let's say I want to handle this case where I'm declaring some variable and then directly export it. But instead of declaring the variable, I want to directly export the value. So I want to write a plugin that will automatically update the code from the top to be like the one from the bottom, and all of this during the build time. So let's build it together. This will be our input code, and we'll write the Bubble plugin itself. And it will look something like this. And it seems very scary, but we'll go over it line by line.

And it seems very scary, but we'll go over it line by line. The first thing I want you to understand is that Bubble allows you to use paths, and paths can be thought of as nodes inside the AST. But the power of Bubble is that you don't get just any path. You get specific types of paths as you require them. So for example, we can ask Bubble to give us all of the referenced identifiers, which are the types of nodes that we needed as I showed you before. So our path, for example, would be the greetings reference here.

Now that we have it, we can access its name, that is the actual string greetings. And the next thing we can do is that we can access the scope that is used inside the reference, meaning the actual lexical scope that is marked here on the right. Then we can query the scope. What is the binding for greetings, meaning what greetings actually references to. So we'll have this variable here.

We need to handle the case where the binding doesn't exist. For example, it may happen if we're having a global reference. For example, if we're using a console log, then console isn't declared anywhere in the lexical scope. So in this case, we'll exit early. Next, we can access the variable declaration and ask what it's being initialized to. So in this case, it will be this string literal here. So once we have the actual initial value in our hands, we can access our original reference again and replace it with the value that we just found. And it will look like this. Now there's no longer any need for our original variable declaration since it's entirely unused. So we can access the binding and tell it to remove itself. And we'll end up with this, which is exactly what we wanted.

This feels like magic, right? We wrote a few lines of codes and we changed drastically how the code behaves. So you're probably thinking, why is it not more common? Why am I not seeing more AST codes anywhere? And the best way to understand it is by playing a short game. So let's play a round of games of guess the output. Let's say I change my code. And instead of declaring a variable that is initialized to a string, I'm now initializing it to be an object. And then I deconstruct the object and export one of these properties. So if you think what we wanted the output for this case to be, we wanted it to be the same output as before, but we won't get this. Instead, we'll have this output.

And this is a major bug because it changed drastically how the code behaves. Instead of exporting a string, we're exporting an object. For sure, it will cause us bugs. So let's understand what happened here.

When our plugin runs, it finds this reference identifier, which according to our code is being initialized to this value here, that is the object itself. So we inline this value instead of the reference, and we'll end up with this code. And so far, so good. Nothing really changed. It will work the same way. But on the next phase of our plugin, it will find this reference identifier, and according to our code, it's being initialized to this value.

So we'll inline the reference with the object, and we'll end up with this code that is entirely not what we intended. And cases like this taught us that super tools come with super bugs. And the reason for it is that regular code has some user input that you run some logic on top of it to end up with your output. But in the case of AST tools, you don't have user input. Instead, you have code input. And code input is very hard to work with. It's less structured. It has many edge cases, and you need to learn how to use it properly.

In many ways, it reminds Regexes. We all used Regexes before, and we saw how simple cases become very complicated very quickly. And you need to learn how to work with Regexes properly to avoid the pitfalls. So with time, we learned how to work with AST tools, and we defined our own best practices, and we wanted to share them with you today. I have dozens of tips, but this talk is very short. So we'll focus on two things. First is to guard your assumptions. And the second is to write exact tests. So when you develop AST tools, you can take nothing for granted, even if you're using AST Explorer.

You're guaranteed to meet new JavaScript features, which is very exciting by itself, but it's less exciting if it's due to a bug in production. So what we'd like to do is to guard our assumptions. So, for instance, in our mini plugin that we just wrote, we knew that it worked perfectly for string bindings.

We checked it through and through, and it worked perfectly. And we thought it would be okay. What can go wrong? But I had an assumption here that we're always bound to strings. When I make such assumptions, I should always question them.

So, for example, if we have this code, meaning we have a variable initialization that is not bound to string, but to an object, then in this case, we'd like to have a nice error during the build time that's saying, look, you're using an unsupported initialization. Here's the file. Here is the line. Please deal with it.

And the way we can get these errors is different between the various frameworks. But generally speaking, it looks something like this. If you're seeing an initialization node that isn't a string literal, then throw an error. But you don't throw just any error. In the case of Babel, for example, you throw a code frame error, which gives you all the context that you need, the location of the problematic node, along with the file, the line, the column, and everything, along with any detail that you'd like to add.

And it's maybe annoying to get such errors during the build time, but it's much better than a bug in production. It also follows the fail fast principle that says that you should fail as soon as possible instead of later on. And lastly, it allows an incremental approach where you're adding more and more layers as you find more cases that you need to take care of. So remember, when you're uncertain, just throw.

So throwing errors allows you incremental growth. But how do you know that the code even works in the first place? We all know that we need to write the right types of tests. We need to write unit tests, we all agree on that, but it's not enough. We need to write at least integration tests to make sure that the entire solution is working as a whole.

So we needed integration tests, but for AST toolings that have some very specific requirements. So after some research, we learned about exec tests. And exec tests look like this. It may seem scary, but when you look again, it's just like a regular test. It has the it part on the top, it has some inputs. Then you do something in the middle, you run something to get some output. And then you can make assertions on the output to make sure that the code is actually working as it should. But the input is actually code. And what you do with it is that you transform it using the plugin that you test to end up with the updated code.

And then this is the magical part. You run the updated code to get its output. And then you can make assertions on the output of the updated code. So you'll know that your plugin updated the code exactly in the way you meant it to update it. So you know it didn't introduce any weird behaviors that you didn't mean to. And exec tests are very powerful. And for us, it found many bugs and prevented many bugs.

And in the end of the day, it saved a lot of time. By following these principles, we developed our own healthy AST development lifecycle. And this is the important part. First, you start by implementing the solution for all the known cases that you found using AST Explore. Then you guard your assumptions by throwing meaningful errors for the cases that aren't handled properly. Then you run your plugin to discover the unknown cases and by meeting these meaningful errors. Then you write exec tests for the new cases that you found, like the one I showed you before. Then you implement the solutions and ensure that the tests pass so you'll know that the cycle is complete and all of the cases are handled properly.

So by following these methods, we're able to deliver super impact. But most importantly, we have a new super tool in our belt and hopefully, so do you. Our key takeaways here is that ASTs are a superpower. If you're meeting an impossible challenge, think about ASTs as one of the tools you can use to solve them. And lastly, when you do, use the AST dev lifecycle to maximize your impact. Thank you.