Package-based Monorepos - Speed Up in Under 7 Minutes

So, they told me I have seven minutes, so no code, just slides, just code, no slides, and what I want to talk a bit through in this short lightning talk is how we can speed up MonrayBus, all right?

So, what I have here is a pmpm workspace, and so it uses pmpm as the package manager system. It's the same thing almost like for npm or YARN workspaces, so there's no big difference there. So, what you can see here in this file, we specify where our packages live, which is here in this packages folder, where each of the packages basically has its own node modules, they have their own kind of package JSON definitions, entry files, also their own scripts, and most importantly down here, they might even have dependencies between them, right?

So, in a MonrayBus scenario, and this is a special case where it is very often used in open source libraries where you want to publish multiple packages and they might have some Like, if you look at React, GitHub repo, VEET, Vue, Angular, a lot of them use these types of approaches. So, how does this work? Now, in PMPM specifically, there's this prefix, and so that means you should resolve the package locally within the Monray but not go out for some MMPM registry, and in your MMPM workspace you don't need that, but that star there will be the same, and so that simply means I always want to depend on the latest version because it is locally in my workspace, right? Don't do this for production apps. This is just within the Monray repo.

So where does this package live? That's where this comes into place again because it can figure out, can it even live in this examples folder or in a packages folder? In this case, this UI library here. And again, this is a package with some build scripts, test scripts, and potentially authors as well, and then we have at the top here some example applications. Now, if you develop this as your application Monray repo, this might be the actual production apps. In open source package-based Monray repos, this is most often example pages where you test out your product or your packages, or it might even be deployed alongside your documentation for some live demos. So how do you run things in such a Monray repo? Well, you could obviously cd into the packages and just trigger these scripts, right? That would totally work. It's not really handy, though.

And so, for instance, PMPM has a concept such as filters. And so, for instance, I can filter me all the packages in this packages folder, and run me that script here, which is that build script. So it recursively traverses these packages. Now, in this case, it is a simple setup. So we just have two of them and it would build those. Now, I can also go and just build the single one. For instance, I have here that remix up there, and I target the dev script. So that would then launch my remix application, and it can serve it then at local 3,000. And I can kind of browse my remix application here, right? So I have, it just renders the components that are in here. Yeah. It's super right. All right. So, you might be wondering, like, what is wrong with this setup? And there's nothing really wrong here, right? Because it works, and especially in a smaller setup, it totally works fine and scales nicely. However, there are already some kind of things that we could optimize. For instance, these packages depend potentially on each other. So, for instance, the next, our remix app imported product list, which internally we have seen depends on the UI library. So what happens, for instance, if I delete this folder, because they consumed this folder, because in the package json we referenced these files, which is compiled output of our typescript files. Then if I serve, for instance, again the remix app, it would kind of break, because it cannot resolve those entry points anymore, and so it breaks, right? And so I would need to kind of run that filter again, rebuild it, and so then it would work. The things I need to keep in mind to order how I build them, I need to first build the UI, then the product list, and then the application.

Plus, if I rerun this filter again, it would always keep running, right, even though we didn't touch anything, we didn't change anything. And this is where, for instance, a package like nx comes in, which can help you speed up some of these things.

So, how can you install nx? First of all, you can just run nx add latest init or just add the package on your own. The advantage of running this script here is that it looks at your workspace and the structure of it, and it kind of walks you through a couple of steps of kind of trying to configure it for you.

For instance, one of the first questions here is what scripts need to run in order? And we just learned that the built one might need to run in order because we need to respect that ordering of those things. And so I would choose the built one here. Next up, it asks me what type of scripts of those here in your package monorepo are cacheable. And so here, for instance, probably the built would be cacheable. Linting, type check and test, potentially. Usually, the dev server and the start scripts are not cacheable because that's the development environment you want to kick off. And then it asks me for outputs. We can just skip them because it already captures most of the common outputs, such as dist and build, and it would already monitor those for the output there.

Before we dive in, what we can already do now is run the nx-graph command. And this gives us an overview here of how that monorepo now looks like. You can actually, by the way, run this even on a monorepo setup that doesn't have an x. So you just run npx nx-graph and it would analyze the dependencies and give you such a graph, which is really useful for reasoning about how the monorepo structure looks like. You can then filter and do a couple of nice operations on top of here, but the more interesting part is this graph is just now for visualization, but behind the scenes, NX obviously uses this also for figuring out ordering and how to build things, so to optimize stuff.