De-bloating the Web: The "Ecosystem Performance" Initiative (e18e)

Hello, everyone. I'm talking about the Ecosystem Performance Initiative today, which in short is just E-A-T-N-E. The Ecosystem Performance Initiative is just a community that's passionate about modernizing the JavaScript ecosystem. It's probably a community with one of the largest diversity that I have seen of all big open source projects. We have maintainers of Vue, of Svelte, Solid.js, Angular, React, all the big frameworks. Even the maintainer of Flourish is on board, just to name a few here. And it's really a big group of people that works together to make the JavaScript ecosystem a better place.

I'm Ullima. I'm an open source maintainer and work at Istly.js. I'm also a consultant freelance developer and a proud member of the E-A-T-N-E community. So the problem that we try to solve is that JavaScript has been growing historically and has accumulated a lot of technical depth over the years. You can run JavaScript everywhere now, there are packages for everything. So when you start a new project, install all your stuff, you often end up with a NodeModules folder that somehow looks like this. You have hundreds or even thousands of packages that are pulled in, and all your dependencies have dependencies themselves. And you end up with a lot of data just for your project, and sometimes reminds of a black hole, and that's not really cool.

There are three main problems with that. A lot of packages you install are redundant, because a lot of packages do the same thing, and are just pulled in from different dependencies, and you end up with lots of packages that you actually don't need, because you only need them once and not two packages doing the same thing. They're also often bloated, because a lot of packages come with functionality you actually don't need, because you're not using it. You're importing one function, but get the whole thing. Luage is a good example of that, even though you now can import them individually if you want to. But earlier, that was not the case. And also, there's a problem with unmaintained packages, because packages that have been unmaintained for a long time could have security vulnerabilities and might just not use the newest syntax, or good performance, or platform features that are available.

So the impact that it has is that performance suffers. You have longer install times, bigger bundles, slower builds, and obviously also security risks from outdated dependencies. And if you ever had to do a security audit, you will know that going through thousands of Node modules is not so nice, because you have to audit all of them. So the less dependencies a project has, the better. So this is exactly the problem E18E was created to solve, and we go about that in three categories. We call them the three pillars of the E18E. The first one is about cleanup. So we go through open source projects and try to reduce the dependencies that they use. We remove unneeded polyfills, for example, for older Node versions, replace packages with alternatives that are faster or smaller.

We also have speed up, where we just try to make packages faster. So we often try to target packages that are at the base of the ecosystem, for example, FDL, which is used everywhere for file system access and all that stuff. So speeding that up speeds also up a lot of other packages that could consume FDL, for example, and just makes everything a little bit faster. The next thing is level up, because sometimes it's just not enough to make existing packages better because they are already bloated or the maintainers just don't want us to intervene. So we try to come up with better versions that are faster, leaner, and smaller, which you can then use instead. So now I want to go into these categories a bit more.

The first one is the cleanup one. So what we actually do is, as I said, we remove unrelated polyfills and replace unmet in packages, eliminate one-liner packages that are often like isNumber or isArray or isOdd, things like that, that's actually a plug-in by Dan and Ro that you can install now when you use Vite that would do that for your bundle already, so they don't end up in your production build. But if you don't even want to install those, that's what we do. We go into these projects and eliminate those in the dependencies. We also upgrade dependencies just as simple as that, because often this is just enough to get all the good things like better performance and stability because we already did the work somewhere else to make something better. And now we just have to make sure that the new version is present in all projects. Here's one of these GitHub issues that we in our repository create to track the progress that we've made. In this example, there's a pump package that can just be replaced with a native functionality from node stream.pipeline because since node 10, you can use that and you don't need the pump package anymore. So if you just do that, you have less dependencies, less to worry about. As you can see, then you have a whole list of packages that use pump. In this example, we have tarfs and pumpify and the traffic they generate when they're always downloaded every month. And the way we create these tables is with a CLI that I created when I first joined E18E. And we had to basically copy the whole NPM registry for that to add indexes so we can query dependencies. So we can know which package depends on what so we can basically create those tables. A community member, Roman, actually decided to build a better frontend for that instead of just the CLI.

So you can just use that and you can just, for example, put side channel in here. And it will just search for this package and you will see the same table that you've already seen before. For example, QS using side channel with a lot of traffic generated here. And you can just go ahead, copy SS markdown and report it to GitHub, to our repository if we want to.

So here are some clean up examples that we have already done, already happened, also from our collaborators. For example, Netify is working closely with us to reduce their size of their stuff. They were able to reduce the edge functions package by a huge amount from 34 megabytes to just eight kilobytes. I'm not sure which black magic they used to do that, but it's pretty amazing. It's pretty amazing and we're happy that they were able to.

Also, the functions package went from nine megabytes to just 40 kilobytes. And that's really cool to see that across the ecosystem. And it's nice to see the impact that we have with our vision to other companies and projects. There's also Storybook. Storybook is one of these examples we already show everywhere, because they're so amazing. Because Storybook was able to flatten the whole dependency tree by about 50% from version eight to version nine, and then cut again a huge chunk out of it when they went version 10 ESM only. You can actually see it.

They have this little nice blog post where they explain what they did. And this nice picture where you can see the old dependency tree in version eight and the new dependency tree in version nine. And it's already clear from the size that they made a pretty amazing chunk here.

The speedup category is one way we try to make tools faster or pictures faster, where a lot of things depend on. A good example here is the new prettier CLI that was developed by E18E community members. As you can see, the speed went from 29 seconds to 12 seconds, for example, in the TSES Lint repository. And that's just speed you get for free if you just use a new version.

So every time you hit save in your editor, it will just happen a bit fast. And your CI runs will also just happen a bit faster. And you just benefit from it because some people did the work to actually make this faster. There's also, for example, Astro, where James, the head behind E18E, went ahead and improved the chunking and emitting of code in Astro. So it just became a lot faster to just build like 11 times in some cases, just by some optimizations.

Obviously, we have to find those first. It's not as easy as to just search for all packages and then do all the replacements. You actually have to know when something is slow first in order to make it fast. There's also Sveltech, which was improved a lot. And in this run, in the Chatsia and Svelte repository, it went from 14 seconds to 10 seconds. That's also pretty nice. Also in collaboration with E18E.

The last category that we have is level up, and the mention I want to do here is TinyGlobby because it has seen a lot of adoption. TinyGlobby was created to get rid of all the glob packages out there that already exist, like glob, fast glob, and all the others. And they all had some problems. Or were pretty slow. Had some edge cases. And TinyGlobby tries to standardize this and be a lean alternative for all of those. And it's now used by VEED, TS-Down, Unbuild Rack Router, and a lot of others. That's really nice to see. These stats are actually outdated right now. They're way, way more now. This is an older number that I pulled a few weeks ago. So that's pretty amazing to see.

Although these new packages can be adopted pretty quickly because people actually care about performance. Then there's also the TinyLibs repository, which is just a collection of zero-dependency alternatives that you can use for a lot of stuff where you actually don't need the bloated version that covers all the edge cases that you actually never encounter in your code. It's a good alternative that we always recommend to use. There's also the Anja's ecosystem, which focuses on packages that can run everywhere, like Node and the browser, Edge runtimes and such. They focus on being runtime agnostic and also performance. So they're also always a good thing to use.

Then there's a new category that I encountered while researching this talk because it turns out that we started building more and more tools to make our own life easier and also to show the impact we have on the ecosystem. I call this tools and data because we now also have a manifest where we explain what package can be replaced with which package, which alternatives exist, how can you replace them? And there has been some adoption of that and tools that already exist that I now want to show here.

The first one is the E18e CLI. The CLI can help you to scan your code base, find dependencies that could be replaced or appear multiple times in different versions, and also shows notes about publishing. It runs Publin for you to show when something's wrong there. It also shows you when you use old syntax that could be replaced by better, newer syntax. It comes with an analyze command that will do this for you and show all the issues. And it also has a migrate command that will run code mods that we also maintain to do all these package replacements for you. So you can basically run this over your code base now and it will help you already do the work, and you don't have to do it manually.

So here you can see a run in the Netify CLI repository, and you will see it has a lot of issues. There's still a lot of work to do in the Netify CLI, and hopefully, we will get to that very soon. You can, for example, see that Netify CLI still uses chalk and this can now be done by Node itself with style text. So there are a lot of things that can be done here, and this is basically what you already can use. In fact, there's actually Anthony Xu from Wurzel who created a skill, an agent skill to do these dependency work for you in your project and use that skill. And the skill directly mentions the E18ECLI already besides all the work that the agent can do itself. And it's pretty cool to see this adoption. So you can use this today to make your project a little bit leaner if you want to.

We also have a GitHub action that shows you all these things. It shows you when you install more dependencies, when install size grows, shows you dependencies that could be replaced. And what I think most important also shows your trust levels. So when one of your trust levels of your dependencies changes, so maybe it was not published by a GitHub workflow anymore, but by an individual, the trust level would change. And this is often a sign that this package got infected by some malware or something. So it can catch a lot of security vulnerabilities, even though it wouldn't have caught the recent attack on TAN stack because the provenance actually did not change there. Still, it's a good thing to have, and there's no harm in using it right now. And I think you all should use it in for some of your renovated PRs where all your dependencies are updating to have a bit more security there.

Here, this is our website where we also have docs for all the replacements we have. This is based on the manifest I mentioned earlier. So here in this example, you can see how FastLob should replace with TinyGlobby and how to do it. We also have replacements.fyi, which is just in the front end to basically browse the manifest because the manifest is just a big JSON file, which is not nice to traverse. So you can just search for packages and see if they have better replacements. Maybe you heard about npmx as well, which has been grown recently. It's basically an alternative front end to npmjs.com, where you can also just see the whole interface npmjs and has lots of stats and also shows you in this little warning box there that there's replacements you can use for Chalk. And this is pretty neat and also based on the manifest to basically make the data more visible to other users to use, so they are all more aware of what's happening in these endeavors.

Then we have the framework tracker, which is not based on the manifest. It's its own thing. Basically, what we try to do, we try to track how different frameworks compare across their versions and how they develop through the versions in case when it comes to dependencies, startup times, install size and such. We did that in order to see what impact we can have and also what the frameworks do themselves to make them faster or reduce the dependencies and such. And this is still in the working. We're trying to figure out how to measure this all properly. So you take this with a big grain of salt right now. It's also not public, but I want to show it here as a little sneak peek. We had some major collaborations throughout the last years and we'd work with the companies and projects on a lot of things. So just to name a few here, it's just a nice cost like with Netlify, Century, Astro. We also work with Void Zero and a lot of others. We even collaborated a little bit with BlueSky to make the bundle a little bit smaller. All these things, there are way more, but there are just a few of them just to show that we actually do some work. So now that you know about what EITE is about, let me talk a bit about how it actually started.

Because at the start, there was only James. He's now the head behind EITE and he was always doing a bit of clean up work behind the scenes, chipping away with this work and also tracking cleanup efforts in its own repository. But at some point, there was a Twitter thread by Anthony Fu where he quoted Bjorn, who was actually doing performance work at the Astrodocs build command. And he was able to speed up the build command by 44%. Anthony Fu was like, okay, that's pretty cool. Why is there not a team doing this full time? And James commented on that, yeah, this team should definitely exist. So they started looking into how to collaborate on these issues and they created Discord.

And Marvin, who was creating blog posts about ecosystem performance at the time, was also on board, as well as Paytag, which was another Veedcore member. And so they created this Discord and started kind of growing from there. And there was also this Node 0.4 drama going on in case you remember that. So that also helped to grow the community a lot more. We basically now have more than 1,500 members on Discord and they're all doing work all the time, replacing dependencies left and right, a lot of issues being solved, which is pretty amazing.

Where do we go from here? What's next for the e18 initiative? We want to get better at all our tools and all the data we have. The CLI is a big part of it, obviously. It already works quite well. We want to refine it, see if we can add more features or take features away. So it's like the best CLI you can have. And also in all these other demos I've shown before, we try to make them better and show our data better and also goes into the next thing, make data visible. Because we have now collected all of these data about dependencies and try to make everyone more aware of performance issues.