Configurational Dependencies in pnpm

So, yeah, I actually renamed this feature to config dependencies after I submitted the talk, because it was really hard to pronounce. So yeah. Okay. So I want to talk about a new feature that came out in PNPM version 10. This is a new type of dependency called config dependency. And I will show a few examples how you can use them. And I will also show some hooks you can use with PNPM to change its behavior during runtime. And in combination with config dependencies, you can kind of create your own custom plugins for PNPM.

So I was already introduced. So just a few words. I'm the lead maintainer of PNPM since 2016. I currently work at Bit, which is like a GitHub alternative, you could say. But instead of repositories, we have components. And at Bit, I work on dependency management. It uses PNPM under the hood for installation. And it allows me to work on PNPM features as well. And I live in Hungary.

Have you heard about PNPM? I hope yes. Okay. And do you use PNPM? Okay. Good to... Good that you did not switch yet to something else. So PNPM is an alternative package manager written in JavaScript. And it's, I guess, most fancy feature is that it uses a central content addressable store. And when you install this PNPM, files inside Node modules are hard links to files in this central store. And so your Node modules use less disk space on the computer. But PNPM is also a very good choice for monorepos. If you check our website, we have a list of our users and some of the popular open source projects that utilize monorepo currently use PNPM. But sometimes a monorepo is not enough. Companies usually have multiple monorepos or a mixture of monorepos and repositories.

Like single package repositories. And when you have multiple repositories, you kind of lose the advantages of centralizing all your configuration, your .files, stuff like that. And there's a lot of things you might want to centralize in terms of dependency management. Like you might have your overrides that you use across the company, or you might have a list of trusted dependencies, or some specific settings that you prefer to use with PNPM. Or you could have a set of patch files, because PNPM and also Yarn support patching your dependencies with patch files.

And I don't know if there's a well-known solution for this. I know there are some tools that automatically synchronize files across repositories. So it's basically copy-pasting. And we had a lot of requests to solve this issue. And when I started to work at BIT, I was really inspired by how BIT solved this issue. In BIT, we have components instead of repositories, and every component is completely portable and independent. So every component actually ships its own set of settings.

And the way it's solved is by a special type of dependency, which is called environment component. As you can see, we have our components in the cloud, and when you develop some feature, you can dynamically import those components that you want to change. And every component can even have its own stack. Some components might run tests with Jest, some others might use VTest. And this is possible because of these environment components which are installed very early on. That's all about BIT. So in PMPM, we implemented something similar, and we called it config dependencies. So this is a completely new type of dependency, which is installed before all the other types of dependencies are installed.

So before prod dependencies, dev dependencies, and optional dependencies. And because it's installed so early on, it allows us to kind of plug into how PMPM works with this config dependencies. These dependencies are somewhat limited, because we don't want people to create complex config dependencies and make PMPM slower. So currently, config dependencies cannot have dependencies of their own. If you need dependencies, you need to bundle them with your config dependency, like with ESBuild. And also, config dependencies cannot run any lifecycle scripts like post installs.

To install a config dependency, you can use a new flag that we added to the add command. So you can run pmpm-add-config and then the name of the package. This is like a regular package hosted in the MPM registry. And when you run this command, a new entry is saved to PMPM's configuration file. It's the pmpm-workspace.yaml. And it will contain the name of the package, the version, and the integrity checksum. So no entry will be added to the log file, but because an exact version will always be saved in this config file, we don't need to change to the log file.

What are these config dependencies good for? I will show us a few examples. And if I have time, maybe even a small demo. Or I will be just switching between slides and demo. So for instance, you can use config dependencies to ship your patch files. So let's say you have some patch for React and you want to use it in several repositories. So instead of copying the patch files, you could create a config dependency and publish the patch file with this config dependency. And then I have an example here. Okay. But this is not React, but a different package. But I have patched this package with a patch file from a config dependency. You can see my pmpm-workspace.yaml. It contains this config dependency. You can check actually this repository on GitHub if you're interested. I have a few examples. And I'm loading the patch file for isod from this config dependency. And if I run this isod package now, you can see that it prints out the message, which was added with the patch file. So if I go to the content of the package, you can see that it has console.log here, which is not present in the original package.

You could also load, allow lists of dependencies that are trusted to run lifecycle scripts. In pmpm version 10, we now prohibit dependencies from running scripts during installation. This was actually first introduced by BAN, and we followed their example. But now it's... This gets hard to manage this list of trusted dependencies. So you can create a config dependency with the list that your organization trusts, and then reference the list via this setting, only build dependencies file. We maintain such a list in our organization called trusted-debs. This is currently just a copy-paste of the list used by BAN.

Oh, shit. Boom. Serial declutters there. So in this package, I'm using the trusted-debs config dependency, and you can see that esbuild ran its postinstall script. But if I run core.js, it won't run its postinstall script because it's not in the list. Okay. You can see here how it looks like in pmpm workspace.

As I mentioned in the title, there are also hooks that you can use to change pmpm's behavior. And we have... And by moving the hooks into config dependencies, you can actually create plugins. Plugins are something that also YARN supports. YARN supports it for a long time, so it has a few advantages over pmpm's hooks and plugins. Actually YARN plugins are... And actually we have YARN's lead maintainer here, and I asked him a few questions about YARN plugins before the talk, so he verified that this is not a lie. YARN plugins are more powerful. You can even create new commands with YARN plugins. That's not possible with pmpm hooks. There are a couple of open source plugins for YARN plugins, and there's a tool you can use to build your YARN plugins, so you don't need to set up esbuild yourself.

But the disadvantage of YARN plugins is that there's currently no commands to install them to your project, so you need to fetch the file for the YARN plugin yourself, and then you need to keep it up to date yourself. And there are no simple hooks you can use. You always need to declare a YARN plugin, which is more work than the pmpm hooks. So basically, pmpm hooks are loaded from a pmpm file.js. And there are two different kinds of hooks. Advanced hooks and simple hooks. With advanced hooks, you can create custom fetchers, importers, resolvers, so you can basically change what gets installed from where. An example of something that uses advanced hooks is StackBlitz. StackBlitz runs pmpm in a web container, and they created these custom fetchers and importers to make pmpm really fast inside the web container. And simple hooks can change the package manifest during installation and change settings.

So I'll be now quick, because time is... There's not much time left. So one of the hooks is the update.config hook, the simple hook. And this hook allows to change the configuration object of pmpm during runtime. So for instance, in this example, you can see that we set an override for Socks, because Socks has a vulnerability in all the versions than 2.8. This way, you can fix vulnerabilities and share these fixes across your organization. Just install this config dependency. You can change the default settings of pmpm to be more strict, for instance, because the default settings allow you a lot. They are good for people that just start using pmpm. But if you want to be more strict, you can have your own policies, create a config dependency and share it across. We maintain a config dependency called better defaults, which we can use, and it will make pmpm more strict, and it's better for advanced users. Also, any breaking changes to pmpm might come first to this config dependency. You can also share version catalogs with this hook. I hope you heard about version catalogs, but if not, check it out. Another simple hook is the read package hook, which allows you to modify the package's manifest during installation. So if some package is missing a dependency, for instance, you can extend the dependency subject here, and it will be installed. In this case, pmpm-to-agent is missing a debug from its dependencies, and we can basically check if it's there, and if it's not, we edit. Another use case is creating allow lists or block lists of packages. Maybe your organization has a process of allowing new packages. You can use this hook to check the package which gets installed and throw an error if you don't want it.

This is another example of a config dependency that is used in the read package hook. We maintain it. It's a type fixer, and it basically tries to see if a package is missing a type dependency, and if it's missing it, it will add it to dependencies or peer dependencies and fix any type script issues you might have with pmpm.

Okay. I'm in time. If you want to check out the examples, you can check them on GitHub. Thank you.

Okay. We have a couple of questions that came in on Slido, so let's go through those. One of them is, can you provide examples of must-have quality-of-life improving pmpm hook implementations? I think you mentioned a couple, but maybe ... Yeah, I mentioned a couple. What do you reckon for other people is the most sort of life- Yeah, I think sharing your settings with this update config hook, so any preferences you have, I would put them into this hook and publish it as a config dependency, and then you can use it as a dependency in any of your projects. It's very convenient. I already use it in my projects. Perfect. Thank you.

Let's see. All right. Let's see which one, because there's a couple of questions about features and so on. There we go. Can config packages be malicious? Maybe a typos or types called package overriding the allow post-install list? Anything can be malicious. Anything can be malicious is like, yeah. Maybe it's a bit easier to verify if it's good or not because it's a single package. It cannot have dependencies of its own, but for sure you need to be cautious about them. Right. Exactly. What actually does the P in pmpm stand for? It's performant npm. Wow, love it.

As a question for me myself, you as a maintainer, how do you sustain yourself working on a project like this? Since I joined the BIT, it's a lot easier because BIT benefits from pmpm. They use pmpm under the hood for their CLI. Before that, I worked at another company, a product company, which didn't care about it, and I used only my free time. So it was really hard. But for me, it worked out because I found a good well-paid job. It was a big investment I had to pay.

So it was really hard. But for me, it worked out because I found a good well-paid job. So it was a big investment I had to pay. But now it's easier.

Do you have any more contributors that joined you along the way that help you build this thing? Yeah, we have. Or are you looking for anyone to help you out building the project? Maybe there is some people in the audience that might be interested. Yeah, I think there are multiple companies that would be willing to hire a person who has good knowledge of pmpm codebase. I know about a few big names. But it's a risky thing because it's a very narrow subject area.

The benefit is that large language models will replace you maybe a bit later on if you're so specialized. But I guess not so many people are needed in this area. But we have one person from Vietnam. His name is Kai, who also works full time on pmpm. He's sponsored by BIT as well. Very cool. Thank you for that.

Are there any tools for migrating from npm workspaces to pmpm for monorepos or anything? From npm to pmpm? I think there's this company that creates mods, Codemods. And I think maybe they created some mods for that. And maybe we even have it in our docs. But we don't have anything built into pmpm for this. All right. Thank you for that.

Let's see, what else? People still want to ask about, you know, underused features or anything else that you would really want to highlight. Oh, this is maybe, like, a little bit of a controversial question. Why are some of the examples that you mentioned not hosted on BIT? Yes, BIT is not so popular right now. Like people are more used to GitHub. So it's like... I feel like it's only a benefit, you know, like, they're not coupled together. You can use either or both of them, but you don't need... And also, like, it's very expensive to give free... How to say it? Because GitHub is used by open source. So they spend like millions for these open source projects because they are free. And BIT is a small company, it can't give all these free tiers to open source projects. Yeah. So it's more enterprise level product, I would say. Okay, I think that's fair enough.

Let's see. Let's do one more question and then we're switching speakers. Let's see. How do you impose restrictions on packages and their specific version? So you can create this package hook, and it's called on every package that gets resolved during installation. And you can put a new statement and check the name of the package and the version. And if you don't allow something, you can throw an error. Yeah. That's how. Thank you so much. And thank you so much for your talk again. We'll do a little switcheroo here for the next speaker. Yeah. Thanks. Thank you very much.