Monorepos & Spaceships – Navigating Successfully Through Code and Cosmos

Hello everyone. I'd like to start off by telling you a story about a guy named Dan. So this is Dan right here. He's a mechanic and his job is fixing spaceships. So you can see him here sitting in this spaceport bar, you know, he's drinking a beer that is technically derived from fungus. But he's just sitting, you know, he's enjoying, he's having a good time.

And Dan is really a metaphor for all of us developers. And like any developer during the day, Dan will go from being cautiously optimistic to falling into a pit of utter despair and depression. But he will come out on the other side on top feeling like he's understood everything and he's on top of the world.

My name is Max and I've personally gone through that cycle of disillusionment and understanding many times in my life. And I hope I can make it a little bit easier for you today. So let's talk about monorepos as you've probably guessed from the title. And I work at a company called NX where we built monorepo tooling. And I also work within monorepo all day every day. So that's kind of my perspective on it.

Just to give you an idea of what you're up against, I'm going to start off by talking about polyrepos, what they do well, but also where they fall short. Then I'll talk about monorepos and what issues they solve, as well as why you need good monorepo tooling, especially at scale. So in this talk, every spaceship is an analogy for one repository. So on his first day of work, Dan hires onto a fleet of spaceships, actually. You know, there's some in the front, they're fighting space pirates, they're exploring, they're doing cool main character type stuff. There's some in the middle, transporting crew, handling repairs, and the like. And then there are some ships in the back end of the fleet, you know, handling supplies with back home and communications. So all these ships, they have their individual responsibilities, but they're also all connected in some way.

And I would say that this is a perfectly logical way to organize a fleet of spaceships. And it also makes sense in the world of code. In the world of code, I would call this approach a polyrepo. Where every team or project has their own repository hosted on GitHub or the like. And what we're trying to achieve and why this is such a popular way to develop software is the team autonomy it gets. So every team has experts in their domain and for their projects, right? And they're free to make decisions. For example, they have technological freedom.

In this case, there's two React apps and an Angular app. They share some code with TypeScript, and there's a backend written in Kotlin. Because this is structured so nicely, it's actually way easier to get an overview and grasp what your entire system is doing and how it's structured, versus just having one big ball of mud with no discernible internal structure.

So to summarize, I would say polyrepos are great. Because they offer team autonomy, technological freedom, and a cleanly separated architecture. But if polyrepos are so great, then why am I here? Why am I giving this talk to a lot of repos, right?

Well, let's say the landing page and the web app team, they want to share some code. They want to share some React hooks. How do we go about this? Well, they would create a repository for those hooks. They'd have to set up publishing, building, linting, testing, CI pipeline. They have to make PRs to the repos to consume that library, and then maybe go through those steps a couple of times to iterate until everything works. As you can imagine, that is a lot of overhead. It's a lot of work that developers don't want to go through.

So they will say, you know, just this one time I'll just copy this code over. It'll be fine. And of course, it's not just one time. It happens all the time. And now that code lives in completely separate rounds. If one team finds and fixes a bug, the other team might not even know that it exists. So over time, inconsistencies and bugs start to creep into your system through code duplication.

And as you can imagine, this doesn't scale well, right? People are going to be on different version of libraries afraid to upgrade because upgrading is always a risk. But even if we look at just the front end part of this, let's say one of those updates required an upgrade to a React version. While everyone started out on synced versions, you know, life gets in the way. Some people have blockers. Some people really need to upgrade in order to fix some issue or some bug. And while everyone started out on synced versions, over time, they'd emerge. In this system, there's nothing stopping anyone from importing a React 17 component into a React 15 app. And who knows what might happen when you do that. Those might work perfectly fine, or they might break later on in the most spectacular ways, even the most subtle ways.

Same goes for back-end front-end relationships. Let's say the front-end and the back-end teams, they agree on some shared interface that they use to communicate. If the back-end team changes that without coordinating well and without communicating with the front-end teams, they might just break them completely in production. So you need some way of handling that, feature flags, version APIs, stuff like that. So what this results in, ultimately, is this kind of constant state of fear where everyone is on their little islands, afraid to upgrade and break their carefully crafted equilibrium. So while team autonomy is great, it comes with the cost, in this case, of isolation. And isolation harms collaboration.

To summarize, Polyrepos has issues because of inefficient code sharing, high code duplication, complex dependency management, all of which ultimately results in isolation. So Dan has gone through all this. He is way on the bottom of the pit I was telling you about earlier. And he's ended up. He just spent weeks trying to coordinate between different crews and fixing issues that shouldn't have been there, spending hours on meetings. And he's trying to drown his sorrows, and I can relate. But then he discovers a new approach, the Monorepo.

So instead of having this fleet of spaceships, they all actually work together in one giant spaceship. There are still crews that can act autonomously, but they all work together in one spaceship.

And while that may seem like a daunting approach at first, it actually has a lot of benefits, which we're going to learn about together with Dan. Hello! Right, so before I go too deep into this, I want to get aligned on what a Monorepo actually is. I like this definition, which says, a Monorepo is a single repository containing multiple distinct projects with well-defined relationships. It's quite a mouthful, so let's get into it.

The first one makes sense, right? Multiple distinct projects. If you just have one repository with one project in it, that is not a Monorepo. That is probably a monolith of some sort with no discernible internal structure. So you need multiple projects. But even if you just put three apps in a folder, that is not a Monorepo. That is just co-location. You need those well-defined relationships. So as soon as you start breaking out pieces of your code, defining in how these different pieces depend on one another, relate to one another, then you have a Monorepo. And it comes with a lot of benefits. Let's get into it.

The first I want to talk about is atomic changes. So do you remember everything that was involved in sharing some code in the PolyRepo setup? You have to set up a repo, linting, testing, publishing, make PRs, make more PRs, set up CI, all that stuff. It's a lot of overhead.

In a Monorepo, however, you can just make a change to a library. And because all those apps import from that folder directly, they just consume those changes right away. So this makes code sharing way easier.

We saw all that was required before. Now you can just create a library, you import from it, and that's it. And that means that code duplication will go down by a lot. People never had something against sharing code. It was just the overhead was so high that they didn't do it. If you make it easy for them, they will gladly do it. So code duplication will go down, as will the number of bugs and inconsistencies in your codebase.

Because everything is in one repository, you can actually define the root level React and TypeScript or other versions. So that means every app can use the same version. You don't have to worry any more about those weird inconsistencies. This makes it way easier to add new projects as well to the monorepo. All the infrastructure is already there. All the dependencies, the CI pipeline, you know how to build and publish, lint and test and everything. You just put the new app in there, import from some shared code and get started right away. This kind of brings me to my next point, which is consistent tooling.

So as you can imagine, in the Polyrepo world, everyone finds their own solutions to problems, right? That's good. We want the team autonomy. But certain things could be standardized. If we can get certain teams or all teams to agree on a shared way of building, publishing, testing, linting their code, then the mental overhead for the developers in each of those teams goes way down.

This kind of brings me to my next point, which is consistent tooling. So as you can imagine, in the Polyrepo world, everyone finds their own solutions to problems, right? That's good. We want the team autonomy. But certain things could be standardized. If we can get certain teams or all teams to agree on a shared way of building, publishing, testing, linting their code, then the mental overhead for the developers in each of those teams goes way down.

I can go to any team and will understand how their code is built, how to work on their project. And that means that I will gladly go and contribute all across the stack, as well as developers will have a consistent developer experience wherever they work. So to summarize, I would say that monorepos help with three big things, velocity, safety and mobility. And all of those things I just talked about, they contribute here in various ways. I'm not going to go through all of them. But let's just take Atomic Commits as an example. Because I can make changes with a single PR across the stack, I don't have to coordinate releases anymore, wait for other teams' priorities to align with mine. I can make all the changes in one single PR, which is way faster. So velocity goes up. Because I can test everything in that single PR together, I can ensure that the whole system is in a valid state. So the reliability and safety of my code will go up. And because it is so easy to make a change that affects projects across the entire stack, people are going to be way more likely to do it, and developer mobility will go up. So monorepos are great. And I hope you're all with me on that.

But I do want to put a little disclaimer out there, which is if your teams hate each other, they won't like working together in a monorepo. And that makes sense, right? But what people are very often worried about in a monorepo is being too tightly coupled to other teams, especially if they don't like them. That's actually a misconception in many cases. In a monorepo, you can still have full control over who contributes to your code, what tools you use, when you deploy it. But there is a validity to this point, to this concern. There is a certain level of base collaboration that is required. Right? Do we need to agree on the same monorepo tool? For once, you probably need to agree on the same CI provider. And if your teams hate each other, that might not happen. However, if your teams hate each other, you probably have an organization and people problem, not a technological problem, and I can't help you with that. But it does allow me to make a very important point, which is monorepos are not an all or nothing thing. Most of you probably aren't Google.

You need to put everything together in one gigantic monorepo. We ourselves, we build monorepo tools, right? Yet we have multiple monorepos. There's the NX repo for open source code and plugins and the like. There is the NX Cloud repo for infrastructure and closed source code. And then there's the NX Console repo, which is open source. It could probably be in the NX Open Source repo, but it doesn't need to be. It's version independently, different teams work on it, so it's fine to keep it in there. So the same applies for you. Most likely, you can start adopting monorepos incrementally, group together whatever makes sense to you, and don't let anyone else tell you to organize your code.

Hello. So for this next section on tooling, I'm going to use NX as an example. But there are a lot of other monorepo tools out there, and I encourage you to go to monorepo.tools, which is a website with information where you can learn all about monorepos and the tools to build them. And all of those authors from those tools, they collaborated on the site. So it is as fair and unbiased a comparison as it can be. Hello.

Right. So I'm going to make a bold claim, which is running tests takes time. Okay, maybe it's not quite the hot take I thought it would be, but it's true, right? So if we put more and more projects in our monorepo, and there are more and more tests in our monorepo, then doesn't that mean that the time to test, especially in CI, will go up and up and up and up? Well, yes, but also no, because good tooling can help. So through analyzing your imports, and your files, and your configuration, Enix can build this project graph of your project of your workspace. You can see an example I like to use here. There is a cart application and a product application that probably work together in some micro-frontend module federation setup. And there are some libraries that they share, and there are some libraries that are just for them. So through this, it actually becomes very easy for Enix to see what is affected by a certain change. Right. So I make a change to the products detail page, and Enix will tell that we will need to retest, build and lint, no, not lint, rebuild and retest the products app, as well as rerun and test for the products app to make sure everything still works. But there's no way that I could have broken the cart application because there's no path in this graph from the cart application to the product details page. This makes CI a lot simpler as well. Instead of some long list of steps, we can just say, Enix, please retest, lint, build, run and test for everything that's affected by this change. That's it. That is your CI configuration now.

So with this, we have effectively uncoupled the time it takes to run your tests, to run CI, from the size of your model repo. And instead, we have coupled it to the size of the change you're making. So if you're making a big change that affects many projects across the model repo, your CI is going to take longer. But if you make just a small change that affects only a small subset of projects, it will be very fast, no matter how big the rest of the repo is.

So how else can we make this faster? Let's say we want to run a build using npm run build-dash-dashproduction. There are a lot of things that go into defining what the output of that build is going to be, right? Source files, configuration files, 10S parameters, like dash-dash production, general dependencies. And these are all taken together, some computation is performed, and we get output. In this case, HTML file in the JavaScript bundle.

So Nx adds a computation cache to this. And how it works is this. When we run the build, we compute a hash. And then we put that hash together with the outputs in the computation cache. But the next time I want to run the build, I can first compute the hash, which is very cheap. And then if it exists in the cache, I can just retrieve it, which means my build goes down from minutes to milliseconds. And the awesome thing is this cache can be shared across your entire organization. So in a big org, someone is always doing work, right? Whether that be your colleague halfway across the world, or your CI pipeline running nightly tests. That means that if I log out in the morning, and I want to run my build, chances are someone has already done that for me. So I can just retrieve from cache, get started on my day right away. And this is really powerful because it means that in that Xmonorepo, with the shared computation cache, no work is ever done twice. Ever. Ever. That explains why it's so important to structure things out to libraries as well. They're not just your fundamental unit of architecture, they're also there to let an X make way more granular decisions when it comes to doing effective calculations or hashing. Ever. Ever. Ever. Ever. Ever. Ever. Right.

So in the worst case, we've already been able to eliminate, no sorry, in the average case, we've already been able to eliminate a lot of work through only doing what's necessary with the fact that and caching wherever possible. But there is always going to be the worst case where we make a change somewhere deep down in the project graph that affects a big portion of the Mondorepo, or even all of it. And we need some techniques and some tools to make sure that our CI times and everything still is reasonable. That is the case. If I make a core change and CI takes three days, that is unreasonable, and people can't work with that.

So how do we do that? Through this project graph, an X can also generate a task graph. Here's an example from our VS Code extension. You can see that in order to build it, we need to first build the generate UI, we need to build the language server, and then we combine all of that and build the VS Code extension itself. And with this task graph, an X can actually make smart optimizations in building this. We know what we can parallelize, we know what order things need to be run, and then we can schedule all those jobs before ultimately building the VS Code extension itself. As you can see, this is already a lot faster than if we were to just run everything sequentially, right? But there is a limit to what parallelization can do. Workflows have a limited number of cores, it's not a hot take for me, and if we start parallelizing more and more things and running them together, it's going to start slowing down eventually and there's a limit to the amount of work we can do.

So we need a way to split these things out onto multiple agents, distribute them. So we do that. Let's take our example again, running build, lint, test, and netainn tests for everything. It is affected in our change. And if it's quite a big change, that might be a lot of tasks that need to be run. So one naive way of splitting this up would be to say, let us just build things on one agent, test things on another agent, etc. But this is actually really inefficient because as you can see, while agent three is building and building and building, the other agents are just idling, wasting money, time, resources. And there's another reason why this doesn't actually work. Remember the task graph, right? These have dependencies on one another.

So for example, in order to build the end-to-end tests, I need to first, in order to run the end-to-end tests, I need to first build on that same machine because the end-to-end tests need to build output artifacts from that machine. So suddenly we have created this really complex problem where we need to make sure things are on the correct order while also respecting those output dependencies, what machines they need to be run on, while also trying to make sure that everything is run as efficiently as possible and save time.

Well, NX kind of just solves that complexity for you. You can tell it, please distribute all these tasks across four agents and it will do it for you. Because we have the task graph, we know what order things need to be run at. And because we have the distributed computation cache, we can completely disregard that other limitation of saying end-to-end tests need to be built and need to be run on the same machine as the build. Agent one can run the end-to-end tests and when the time comes, can just replay the build from cache even though it was run on agent four. And it does let you know the difference whether it was run on itself or in another machine. This is really powerful. And because it's so simple to configure, you can start playing around with it. You can say, maybe I want to save some more time. Let's use six agents. I tweak one single number on top and my CI time goes down. Or maybe you're saying, this isn't actually that time critical. I can wait a little longer and save some money. I just use two agents. And nowadays, there's actually a model that we built where it takes your change. You can specify target number of minutes you want your CI to run. You can say, I want my CI to take 25 minutes. Then NX will figure out for a certain change how many agents is that going to need. And this is something that you're not going to get past at a certain scale. If you have a big monorepo, there are always going to be those worst cases when you have to retest everything. And if you have to wait three days for that to be done, then it's just not feasible.

So after learning about all this, Dan is happy. You can see he looks content. He looks focused on his work. He's not drinking for once, which is kind of all we've seen him doing really. And if you want to be focused and content like Dan and get that jawline as well, maybe, I recommend you head over to monorepo.tools to learn all about monorepos. Or you can just trust my unbiased opinion and go over to nx.dev and read our documentation. So now that Dan's happy, that means I'm happy. And I hope all of you learned something. And now all that's left to say is thank you.