De-bloating the Web: The "Ecosystem Performance" Initiative (e18e)

Hello, everyone. I'm talking about the Ecosystem Performance Initiative today, which in short is just E18E. The Ecosystem Performance Initiative is just a community that's passionate about modernizing the JavaScript ecosystem. It's probably a community with one of the largest diversity that I have seen of all big open source projects. We have maintainers of Svelte, SolidJS, Angular, React, all the big frameworks, even the maintainer of Lodash is on board, just to name a few here. And it's really a big group of people that works together to make the JavaScript ecosystem a better place.

I am Ulima. I'm an open source maintainer and work at Istly GJS. I'm also a consultant freelance developer and a proud member of the E18E community. So the problem that we try to solve is that JavaScript has been growing historically and has accumulated a lot of technical depth over the years. You can run JavaScript everywhere now. There are packages for everything. So when you start a new project and install all your stuff, you often end up with a NodeModules folder that somehow looks like this. You have hundreds or even thousands of packages that are pulled in and all your dependencies have dependencies themselves. And you end up with a lot of data just for your project and sometimes reminds of a black hole. And that's not really cool.

There are three main problems with that. A lot of packages you install are redundant because a lot of packages do the same thing and are just pulled in from different dependencies. And you end up with lots of packages that you actually don't need because you only need them once and not two packages doing the same thing. They're also often bloated because a lot of packages come with functionality you actually don't need because you're not using it. Luage is a good example of that even though you now can import them individually if you want to. But earlier, that was not the case. And also there's a problem with unmaintained packages because packages have been unmaintained for a long time, could have security vulnerabilities and might just not use a new syntax or good performance or platform features that are available.

And also there's a problem with unmaintained packages because packages have been unmaintained for a long time, could have security vulnerabilities and might just not use a new syntax or good performance or platform features that are available. So the impact that it has is that performance suffers. You have longer install times, bigger bundles, slower builds, and obviously also security risks from outdated dependencies. And if you ever had to do a security audit, you will know that going through thousands of node modules is not so nice because you have to audit all of them.

So the less dependencies a project has, the better. So this is exactly the problem e18e was created to solve. And we go about that in three categories. We call them the three pillars of e18e. The first one is about cleanup. So we go through open source projects and try to reduce the dependencies that they use. We remove unneeded polyfills, for example, for older node versions, replace packages with alternatives that are faster or smaller. We also have speed up, where we just try to make packages faster. So we often try to target packages that are at the base of the ecosystem. For example, AFDIR, which is used everywhere for filesystem access and all that stuff. So speeding that up speeds up a lot of other packages that consume AFDIR, for example, and just makes everything a little bit faster.

The next thing is level up, because sometimes it's just not enough to make existing packages better because they are already bloated or the maintainers just don't want us to intervene. So we try to come up with better versions that are faster, leaner and smaller, which you can then use instead. Now I want to go into these categories a bit more. The first one is the cleanup one. So what we actually do is, as I said, we remove unneeded polyfills and replace unmaintained packages, eliminate one-liner packages that are often like is number or is array or is odd, things like that. That's actually a plug-in by Deniro that you can install now when you use Vite that would do that for your bundle already so they don't end up in your production build. We also upgrade dependencies, just as simple as that, because often this is just enough to get all the good things like better performance and stability because we already did the work somewhere else to make something better.

We go into this project and eliminate those in the dependencies. We also upgrade dependencies, just as simple as that, because often this is just enough to get all the good things like better performance and stability because we already did the work somewhere else to make something better. And now we just have to make sure that the new version is present in all projects.

Here's one of these GitHub issues that we in our repository create to track the progress that we've made. In this example, there's a pump package that can just be replaced with a native functionality from Node Stream.pipeline because since node 10, you can use that and you don't need the pump package anymore. So if you just do that, you have less dependencies, less to worry about. As you can see, then you have a whole list of packages that use pump. In this example, we have tarfs and pumpify and the traffic generate when they're always downloaded every month.

And the way we create these tables is with a CLI that I created when I first joined E18E. And we had to basically copy the whole NPM registry for that to add indexes so we can query dependencies, so we can know which package depends on what. So we can basically create those tables. A community member, Roman, actually decided to build a better front end for that instead of just the CLI. So you can just use that and you can just, for example, put side channel in here. And it will just search for this package and you will see the same table that you've already seen before.

For example, QS using side channel with a lot of traffic generated here. And you can just go ahead, copy this markdown and report it to GitHub, to our repository if you want to. So here's some cleanup examples that we've already done, already happened, also from our collaborators. For example, Netify is working closely with us to reduce their size of their stuff. They were able to reduce the edge functions package by a huge amount from 34 megabytes to just 8 kilobytes. I'm not sure which black magic they used to do that, but it's pretty amazing and we're happy that they were able to.

And also the functions package went from 93 megabytes to just 40 kilobytes. And that's really cool to see that across the ecosystem and it's nice to see that the impact that we have with our vision to other companies and projects. And there's also Storybook. Storybook is one of these examples we already show everywhere because they are so amazing because Storybook was able to flatten the whole dependency tree by about 50% from version 8 to version 9 and then cut again a huge chunk out of it when they went version 10 ESM only. You can actually see it. They have like this little nice blog post where they explain what they did and this nice picture where you can see the old dependency tree in version 8 and the new dependency tree in version 9. And it's already clear from the size that they made a pretty amazing job here.

So the speed up category is one way we try to make tools faster or pictures faster where a lot of things depend on. A good example here is the new Prettier CLI that was developed by E18E community members. As you can see, the speed went from 29 seconds to 12 seconds, for example, in the TSES Vint repository. And that's just speed you get for free if you just use a new version. So every time you hit save in your editor, it will just happen a bit faster and your CI runs will also just happen a bit faster and you just benefit from it because some people did the work to actually make this faster. There's also, for example, Astro, where James, the head behind E18E, actually went ahead and improved the chunking and the emitting of code. And Astro, so it just became a lot faster to just build like 11 times in some cases just by some optimizations. Obviously, we have to find those first. It's not as easy as to just search for all packages and then do all the replacements. You actually have to know when something is slow first in order to make it fast. There's also SwellCheck, which was improved a lot. And in this run in the Shazian repository, it went from 14 seconds to 10 seconds. That's also pretty nice also in collaboration with E18E. The last category that we have is level up and mention I want to do here is TinyGlobby because it has seen a lot of adoption. TinyGlobby was created to get rid of all the glob packages out there that already exist like Glob, FastGlob and all the others. And they all had some problems or were pretty slow. It had some edge cases.

And TinyGlobby tries to standardize this and be a lean alternative for all of those. And it's now used by Vite, TS-Down, Unbuild Rack Router and a lot of others. And that's really nice to see. I think these stats actually out there right now. There are way, way more now. This is an older number that I pulled a few weeks ago. So that's pretty amazing to see, although that these new packages can be adopted pretty, pretty quickly because people actually care about performance.

Then there's also the TinyLibs repository, which is just a collection of zero dependency alternatives that you can use for a lot of stuff where you actually don't need the bloated version that covers all the edge cases that you actually never encounter in your code. It's a good alternative that we always recommend to use. And there's also the Anjes ecosystem which focuses on packages that can run everywhere, like Node in the browser, Edge runtime and such. And they focus on being runtime agnostic and also being safe performance. So that also always a good thing to use.

Then there's a new category that I encountered while researching this talk, because it turns out that we started building more and more tools to make our own life easier and also to show the impact we have on the ecosystem. I call this tools and data because we now also have a manifest where we explain what package can replace which package, which alternatives exist, how can you replace them? And there has been some adoption of that and tools that already exist that I now want to show you. The first one is the E18E CLI. The CLI can help you to scan your code base, find dependencies that could replace or appear multiple times in different versions and also shows notes about publishing. Like it shows it runs Publin for you to show when something's wrong there. It also shows you when you use old syntax, but could be replaced by better, newer syntax. It comes with an analyze command that will do this for you and show all the issues. And also has the migrate command that will run code mods that we also maintain to do all these package replacements for you. So you can basically run this over your code base now and help you already do the work and you don't have to do it manually. So here you can see a run in the NetDefi CLI repository and you will see it has a lot of issues. There's still a lot of work to do in the NetDefi CLI. And hopefully we get to that very soon. You can, for example, see that NetDefi CLI still uses Chalk and this can now be done by Node itself with StyleText. So a lot of things can be done here and this is basically what you already can use. In fact, there's actually Anthony Hsu from Wurzel who created an agent skill to do this dependency work for you and your project. And he used that skill and the skill directly mentions the E18 ECLI already, besides all the work that the agent can do itself. And it's pretty cool to see this adoption. So you can use this today to make your project a little bit leaner if you want to.

We also have an action that shows you all these things. It shows you when you install more dependencies, when install size grows, and it highlights dependencies that could be replaced. One crucial feature is it shows your trust levels, indicating changes in dependency sources. This can reveal security vulnerabilities, although it may not catch all incidents like the recent TanStack attack. Using this tool for renovate PRs can enhance security. Our website contains detailed docs on replacements based on the earlier mentioned manifest.

FastLob as a replacement for TinyGlobby and its implementation are illustrated on our website. Replacements.fyi serves as a frontend to explore the manifest efficiently. NPMx, an alternative frontend to npmjs.com, offers enhanced visibility and stats on replacements like for chalk. The framework tracker, a separate initiative, compares frameworks across versions regarding dependencies, startup times, and more. This ongoing project aims to assess impacts and enhancements within frameworks, although still in the refinement phase and not yet public.

Over the years, we have collaborated with various companies and projects such as Netlify, Century, Prettier, and others. Collaborations with Void Zero and Blue Sky have aimed to reduce bundle sizes and optimize processes. These partnerships reflect our commitment to streamlining tools and data management, showcasing our efforts to improve efficiency and performance across different domains.

It's basically an alternative front end to npmjs.com where you can also just see the whole interface in NPMJS and has lots of stats and also shows you in this like a little warning box there that there's replacements you can use for chalk. And this is pretty neat and also based on the manifest to basically make the data more visible to other users to use. So they are all more aware of what's happening in this endeavors.

Then we have the framework tracker, which is not based on the manifest. It's its own thing. Basically, what we try to do, we try to track how different frameworks compare across their versions and how they develop through the versions in case when it comes to dependencies, startup times, install size and such. We did that in order to see what impact we can have and also what the frameworks do themselves to make them faster or reduce the dependencies and such. And this is still in the working. We're trying to figure out how to measure this all properly. So you take this with a bit grain of salt right now. It's also not public, but I want to show it as a little sneak peek.

We had some major collaborations throughout the last years and we'd work with companies and projects on a lot of things. So just to name a few here, it's just a nice, nice cause like with Netlify, Century, Prettier, Astro. We also work with Void Zero and a lot of others. We even collaborated a little bit with Blue Sky to make the bundle a little bit smaller. All the things, there were more, but there are just a few of them just to show that we actually do some work. So now that you know about what it is about, let me talk a bit about how it actually started.

Because at the start, there was only James. He's now the head behind EITNe and he was always already doing a bit of clean up work behind the scenes, shipping away with his work, and also tracking key efforts in its own repository. But at some point there was a Twitter thread, Anthony Fu, where he quoted Bjorn, he was actually doing performance work at the Astro Docs build command and he was able to speed up the build command by 44 percent. Anthony Fu was like, OK, that's pretty cool. Why is there not a team doing this full time? And James commented on that. Yeah, this team should definitely exist. So they started looking into how to collaborate on these issues and they created Discord.

And Marvin, who was creating blog posts about ecosystem performance at that time, was also on board, as well as Paytek, which is another Veedcore member. And so they created this Discord and started growing from there. And there was also this Node 0.4 drama going on, in case you remember that. So that also helped to grow the community a lot more. We basically now have more than 1,500 members on Discord and they're all doing work all the time, replacing dependencies left and right, a lot of issues being solved, which is pretty amazing.

So where do we go from here? What's next for the e18 initiative? We want to get better at all our tools and all the data we have. The CLI is a big part of it, obviously. It already works quite well. We want to refine it, see if we can add more features or take features away. So it's like the best CLI you can have. And also, in all these other demos I've shown before, we try to make them better and show our data better and also goes into the next thing, make data visible. Because we have now collected all these data about dependencies and try to make everyone more aware of performance issues and we want to make them visible.