Node Monorepos with Nx

Multiple apis and multiple teams all in the same repository can cause a lot of headaches, but Nx has you covered. Learn to share code, maintain configuration files and coordinate changes in a monorepo that can scale as large as your organisation does. Nx allows you to bring structure to a repository with hundreds of contributors and eliminates the CI slowdowns that typically occur as the codebase grows.


Table of contents:

- Lab 1 - Generate an empty workspace

- Lab 2 - Generate a node api

- Lab 3 - Executors

- Lab 4 - Migrations

- Lab 5 - Generate an auth library

- Lab 6 - Generate a database library

- Lab 7 - Add a node cli

- Lab 8 - Module boundaries

- Lab 9 - Plugins and Generators - Intro

- Lab 10 - Plugins and Generators - Modifying files

- Lab 11 - Setting up CI

- Lab 12 - Distributed caching

Rate this content
Bookmark
Github
Video Summary and Transcription
The video explores the concept of using NX for managing Node.js applications in a monorepo setup, addressing the question 'what is nx monorepo'. It explains how NX is framework and language agnostic, supporting not only Node.js but also React, Angular, Java, Python, and more. The talk highlights the benefits of monorepos, such as atomic changes and simplified dependency management, and describes how to 'add nx to monorepo' for efficient build and test processes. The course, referred to as 'nx monorepo course', guides users through creating an NX workspace, generating applications and libraries, and managing dependencies. It also covers NX Cloud, which offers distributed caching and task execution, optimizing build times. The video demonstrates using NX commands like 'nx run' and 'nx serve' to manage executors and build targets, emphasizing the advantages of targeted testing and local caching. Additionally, the talk discusses creating custom generators and using NX Console for streamlined development processes.

This workshop has been presented at Node Congress 2023, check out the latest edition of this JavaScript Conference.

Available in Español: Monorepos de Node con Nx

FAQ

Node Monorepos with NX is a workshop tailored towards using Node.js as the framework within NX, which is framework and language agnostic. It focuses on managing codebases in a monorepo setup, where multiple applications can coexist in a single repository, improving code sharing and dependency management.

The benefits of using a monorepo include atomic changes across multiple apps, easier code sharing, and a single set of dependencies which simplifies management and reduces conflicts between different versions of the same dependency.

Yes, NX is framework and language agnostic. It can be used with front-end frameworks like React or Angular, and it supports various programming languages including Java, Python, and .NET.

In a monorepo setup, NX can detect changes in a shared library and automatically run tests for dependent applications. This immediate feedback loop helps developers fix issues quicker, ensuring that changes in one part do not adversely affect other parts of the application.

NX supports distributed caching through NX Cloud, which allows build artifacts to be shared across a team or organization. This means that if one developer builds a project, the results can be reused by others, significantly reducing the build time across the team.

NX improves efficiency by using techniques like affected command detection to run tests and builds only on affected projects, caching results for faster subsequent builds, and providing tools for controlled code sharing and consistent coding practices across teams.

A monorepo involves managing multiple projects within a single repository with tooling support to handle complexities like dependency management and testing scopes. Code colocation, on the other hand, is simply placing multiple projects in one repository without any specialized tooling, which can lead to maintenance and scalability issues.

1. Introduction to Node Monorepos with NX#

Short description:

Welcome to Node Monorepos with NX. This workshop will be tailored towards Node as the framework used with NX. NX is framework and language agnostic, allowing you to use front-end applications like React or Angular, as well as other languages like Java and Python. Monorepos offer benefits such as atomic changes, easy code sharing, and simplified dependency management.

All right, okay, so welcome to Node Monorepos with NX. This is great. So I am, my name is Isaac Mann. I'm an architect at NX. We've been, I've been working with NX for four years. I've been an employee of NX for four years. I used it for two years before that. And yeah, this is exciting.

So this workshop will be tailored towards Node as just the framework that we'll be using with NX. But NX is, you know, is pretty much framework and language agnostic. But we'll get into that more. So it's, you know, you can use front-end applications like React or Angular or you know, whatever other front-end technology you're using. You can also use other languages like obviously people use Java and Python and.NET with NX. So NX is all about, you know, managing your code base and whatever code you're using is this is node monorepos with NX.

And first, we're going to start with a little discussion of, why you would want a monorepo. So a monorepo is any repository that has more than one application living in that, in that codebase. So what's the benefit of a monorepo? So three kind of big categories of benefits for monorepos. Monorepo give you atomic changes, allow you to easily share your code and give you a single set of dependencies, make it easier to manage those dependencies. So we're going to dig into each of these three things and explain what they are and why they're beneficial.

2. Benefits of Monorepo and NX#

Short description:

A monorepo offers benefits such as atomic changes, easy code sharing, and simplified dependency management. It allows for faster command execution, targeted testing for affected projects, and local and distributed caching. NX provides tooling to help achieve the benefits of a monorepo without the drawbacks of code colocation. It enables controlled code sharing and efficient management of dependencies.

So if you had an application and a kind of a library that is, that is used by that application. And if you have them managed in two separate repos, let's say you make a change to the UI library that's in repo two at the bottom here. If you make a change to it, and for some reason it breaks the homepage application, you, the, the lifecycle of that change would be so somebody makes a change to the common UI library. They push up the commit, they incommit the version at some point later somebody tries to use that new version of the common UI library in the homepage app. They realize there's a, there's a breaking change there and it broke their application. So they file a bug with the common UI library, and they say, Hey, you gotta fix this. And then later once the UI library developer sees that she sees that problem they, you know, they make the change, they push a new fix up new version, and then the homepage app, people later have to you know bump the version to the fixed fixed version, and then see if it see if it fix their application. So that that whole cycle, typically, I mean the best case scenario few days, it can take a week or two if if people are not you know working on these things full time.

So, one of the issues here is, if you were to put both those library, the library and the application together in the same repo, then that that cycle would be, it would be just the UI library creator, making a change, running the tests and seeing Oh, it broke the tests in the homepage app, I've got to fix that. And that so that that cycle took will take like a half hour an hour. Like, even before you've pushed up a PR, you'd notice that you broke the application, you have to make, you know, modify your code. So, that's so you have in one PR, you know, whether you know, all the changes that you need to make to the application or changes that you know, you need to make the UI library to adapt to the application itself. That's that's one benefit of a mono repo. Second benefit is shared code. Let's say you have some logic about you know, what is a valid username? And you know, this function, it's obviously, the different scenarios, you'd have a, you know, more complex function than this, you're handling whether username was valid. But let's say you want to share this across your application across multiple application, multiple libraries. If this ever were to change, you'd have to update that in every single repo that's using it. If you're if you're copying the code from repo to repo, whereas if you're in a monorepo setting, you can just use this function. And whatever that function changes, then your, your behavior changes across the whole, the whole repository, wherever it's being used. So easily sharing code. Third thing is having a single set of dependencies. So if you have a framework like Node, or in this case, and the picture is react, if you have different versions of that framework being used in different applications, you can run into very strange, hard to debug runtime errors. And, and having your code in the same repository will basically forces you to have, well it can you can set rules in place to force yourself to be on the same version for everything that's in that repo. It is possible to have multiple versions of the same mono repo, but it's, it's better to have a single version and make things easier in the long run. So basically in in a large when you have multiple applications, typically you have one application that you're working on all the time, that it's on the latest version of whatever dependencies you you have, but if you, you're going to have, you know, two or three other applications that they get worked on maybe once every couple of months and those inevitably fall behind and you, you have to remember, okay, how did I upgrade that framework? Um, you know, this, the six months ago upgrade, and you have to remember and go through all that same work again. Um, whereas if it was, if you were upgrading all your applications, all three applications at the same time, it's a lot easier than upgrading three different applications, different points over the course of a year and a half. Um, so you just do it all at once and it's a lot easier than, than doing it spread out over a year or whatever, whenever you think of updating those applications.

So, one way of, you know, having a, um, a mono repo, um, so the difference between a mono repo and code colocation and code colocation is when you just dump all the applications together in the same repo without having any tooling to manage that. And that, that can be a nightmare. If you just dump it all together, um, you can end up with a scenario where you're running unnecessary tests. We have no code boundaries and all the code just gets mixed up together and it's hard to maintain. Um, and you can have inconsistent tooling where you have conflicts between how those tools interact with each other like your testing tools or your, uh, whatever other tools you have.

So, NX is a tooling to help you, uh, get all the benefits of a monorepo without the negatives of code colocation. All right. So NX gives you a faster command execution. So, um, executors are basically, um, like, uh, NPM scripts, um, that can help you, um, you run your, um, your build and test and lent and all the actions that you take on your code. Uh, NX affected is the tool that, that lets you run the tests for only the projects that are affected by a, uh, by a code change and not the tests that are unrelated. Um, and then we have local and distributed caching. So anytime you run a command, um, and, uh, if you run that same command again without changing the code, it just uses the, instead of actually running the build again, it just uses the, the cast version of that, um, of that output and, um, without, you know, actually, um, running the build again. So if, if your bill takes, you know, a minute and a half, um, if you run that build again, instead of taking a minute, a half, it'll take a second or two just to pool that, that data from the cache and to replay the terminal output for you. Um, so that's just locally. That's, that's completely free. That's in the open source NX. Um, then with a NX cloud, you can use the distributed caching. So that, that same caching behavior that you get, um, on your own machine, you can share that cache across your whole organization. So if anybody anywhere has run this command with this same code, you can use their cache and replay it yourself. So this, you know, particularly for PRS, you, you run your code, you push it up. Um, the runs in CI and then somebody else checks that branch out on their machine and they run the same thing. They can, um, just use your cache version instead of running that same, um, you know, computation on their machine again. Um, so basically you, you only run, um, the actions for a particular set of code one time across your whole organization, which is, um, and that's also, it's not just for, um, uh, that that's broken up by projects. So if you have, um, one application that depends on five different, um, libraries, um, that's the, the library build step is cached as well as the application build steps. So say you change just the application, those five libraries, their build output is already cached for you. And then, so you can just use that cache and then only do the last application built step, which can be game changing for large code bases. Um, and X also gives you controlled code sharing. So you, you set a, an index.ts file at the root of your, um, each project.

3. Benefits of NX in Monorepos#

Short description:

NX provides consistent tooling to manage complexity in monorepos. It offers faster command execution, selective testing for affected projects, local and distributed caching, controlled code sharing, consistent coding practices, and accurate architecture diagrams. To create a new workspace, run NPX create NX workspace and provide a name. NX sets the directory, path alias, and MPM scope based on the provided name. In lab one, we'll generate an empty NX workspace named BG-Horde for the fictional company BoardGameHorde.

Um, the other thing is, uh, you can have any consistent tooling. So, um, typically with the MPM scripts, um, they, they just get named haphazardly and there's a bunch of flags thrown in there. Um, and nobody knows exactly what all the flags do. Um, nobody knows exactly what all the scripts really do. Um, so you've got, you know, different versions of build and test and, and serve and Lint. And, you know, they're, they're, they're set up for different scenarios and different, um, you know, eventualities. But there's, uh, it's really hard unless you're working on that project every day. It's really hard to know what all those scripts actually do. Um, so it'd be nice to have, you know, some consistent rules and, and, um, uh, some, uh, just conventions about what those things should be named and, and how to easily, uh, know what each flag needs in that, uh, in your scripts.

All right. So you need some, some tooling to help you manage that, that complexity. So that's what, what NX can do. NX is a tooling to help you, uh, get all the benefits of a monorepo without the negatives of code colocation. All right. So NX gives you a faster command execution. So, um, executors are basically, um, like, uh, NPM scripts, um, that can help you, um, you run your, um, your build and test and lent and all the actions that you take on your code. Uh, NX affected is the tool that, that lets you run the tests for only the projects that are affected by a, uh, by a code change and not the tests that are unrelated. Um, and then we have local and distributed caching. So anytime you run a command, um, and, uh, if you run that same command again without changing the code, it just uses the, instead of actually running the build again, it just uses the, the cast version of that, um, of that output and, um, without, you know, actually, um, running the build again.

So if, if your bill takes, you know, a minute and a half, um, if you run that build again, instead of taking a minute, a half, it'll take a second or two just to pool that, that data from the cache and to replay the terminal output for you. Um, so that's just locally. That's, that's completely free. That's, that's in the open source NX. Um, then with a NX cloud, you can use the distributed caching. So that, that same caching behavior that you get, um, on your own machine, you can share that cache across your whole organization. So if anybody anywhere has run this command with this same code, you can use their cache and replay it yourself. So this, you know, particularly for PRS, you, you run your code, you push it up. Um, the runs in CI and then somebody else checks that branch out on their machine and they run the same thing. They can, um, just use your cache version instead of running that same, um, you know, computation on their machine again. Um, so basically you, you only run, um, the actions for a particular set of code one time across your whole organization, which is, um, and that's also, it's not just for, um, uh, that that's broken up by projects. So if you have, um, one application that depends on five different, um, libraries, um, that's the, the library build step is cached as well as the application build steps. So say you change just the application, those five libraries, their build output is already cached for you. And then, so you can just use that cache and then only do the last application built step, which can be game changing for large code bases.

Um, and X also gives you controlled code sharing. So you, you set a, an index.ts file at the root of your, um, each project. And you say everything exported from this index.ts file is available for other people to use. Um, but anything that's not exported from the, from that index.ts you're not allowed to reach in there and grab it and start using it. Um, you can set up a tagging system to say, these particular projects can depend on other projects. Um, and it's very flexible. You can set up your own, uh, structure of how those, how those tags work. Um, so the, these, these libraries belong to this team. And these libraries belong to these other team, this other team. And then these libraries are shared, uh, shared, and they can be used by either. Um, so you can set up that whole structure yourself. Uh, you can publish easily published libraries to MPM and, um, you can use the code owners file to require, um, this get feature GitHub feature, um, and or in any of it, like get lab and other, all the other, um, like code management, get repositories will, um, has the same functionality with code earners, where you force, um, a, a review from a certain, um, from a certain group of people, if you change code within that area. So you can control how that code is managed.

Number three is consistent coding practices. So you can, you know, there's, uh, NX gives you some Lint rules to help, um, you know, manage structure. Um, you can, uh, write code generators to automate the steps that people, the developers take over and over again. Um, there are plugins that are, that, uh, that NX creates and plugins created by the community have these, these generators and, uh, executors, um, written for you. Um, so the plugins help you with, um, basically that you can stop worrying about your build system. If you want to, you can have the plugin manager build system and then just make whatever tweaks you need to for your particular application. Um, plugins also help you with, um, automating the update process. So when a, um, when there's a new version of no that comes out, if there's a breaking change or if there's a new version of express or whatever, um, they, there's a migration script that, that can automatically update your code to, to handle any breaking changes that, that happen. Um, and same thing with community plugins, they, they're community plugins that, uh, that cover, uh, use cases that, that NX does not.

Then the other thing is a NX gives you a, um, an accurate architecture diagram of how your projects depend on each other. So this is useful for the NX effective command, but it's also useful for yourself to really see, okay, what is the actual architecture of my code rather than the architecture that I wish my code had, or the architecture that my code had two months ago when somebody drew a diagram? Um, so NX actually understands your code and can draw the connecting dependency lines between your projects based on what's actually there you or some with some architect actually like wished was there, or thought should be there. So, um, yes, this is a typical NX file structure. Um, really your, your file structure can be whatever you want, but basically, um, the, the way we'll set it up was, we'll have an You can set up a gist folder and a libs folder. Um, and then yeah. Dist is where your, um, you know, applications get built to, um, tools is for like scripts and, and helper things. Um, workspace. JSON is not valid anymore. Nx dot JSON is like your root level configuration for NX itself. Um, TS config that sets up a, a, um, um, a script, um, a, a, um, and then you set up kind of like paths, um, for your, um, for your libraries to automatically, so you can reference them to import them into applications. Um, or all the different, um, kind of configuration that NX has, there's, there's workspace level and project level. So the project level configuration will inherit from the workspace level configuration. Same thing, um, if it has a different, uh, TypeScript, uh, setting that it needs, same thing for Jest and Prettier and all the other, uh, tools that, that NX can set up for you.

All right, so that was a lot of talking. Um, we're going to get started with creating a new workspace. Um, so to create a new workspace, you run a NPX create NX workspace, or you can do the same thing with YARN or PMPM, um, and you give it a name. So in this case, the name is sets three different things. It'll set what directory that you're putting your code in. It'll give you the, uh, path alias, um, for importing code, uh, within things. So if you set your workspace name to my org, your directory will be under my dash org, your path alias will be at my org slash whatever project you're importing. And then if you ever, if you ever publish your, um, a library to MPM, the, the MPM scope will be at my org, um, slash, whatever library you have. So each of these three things you can change after the fact. Um, but when you initially set up your workspace, um, whatever you type in after CreateNX Workspace will be, will be set to those, those two things will be set to that name. Okay. So let's get started with lab one. Okay. So, um, this repo here, you do not need to, um, you don't do not need to clone this repo. Um, cause we're going to be creating a whole new repo, um, for the, um, for the workshop. So we're going to go to lab one here, generate an empty work NX workspace for a fictional company called the BoardGameHorde. And we're going to name it BG Dash Horde. All right. So I'm going to run NPX create NX workspace. And we're going to name it BG dash Horde. All right. Uh, let me, let me make this full screen so we can read it.

4. Introduction to NX and Plugins#

Short description:

For our workshop, we'll use the integrated model repo, which allows multiple applications in the same repo. Plugins add another layer on top of the core of NX, managing the build system and providing code generators. You can run NX list to see installed plugins and add new ones with NPM or Yarn. Generators can be run with NX generates or NX console, and you can create custom generators specific to your organization. In step two lab two, we'll create a node API app under apps.

Um, okay. You know what? I have an old version of create INX workspace. So I'm going to do at latest here. There we go. So this is what you should see. Um, so, um, it gives you some, some options here to pick from a package based, um, is, uh, the setup where you have, um, basically you're managing, uh, your packages with package, JSON files. And, um, it's more freeform and basically lets you set up your own build system the way, whatever way you want and NX kind of gets out of your way, unless you do whatever you want. Um, for our, for our, um, workshop here, we're going to use the integrated. Um, you, it kind of has a setup structure with apps and libs, and is more, um, uh, is the, the, the idea with an integrated amount of repo is that you, you, you buy into the whole NX ecosystem and use, you're using plugins to manage your, your build system and your lens system. Um, and then, um, and it's set up so that you can have multiple applications in the same, same repo, um, the, the, the, um, the, the, um, you know, you the, these last three options, standalone react, Angular, no depth. Um, these are ones where your, you have, uh, where you're, you have basically one application in your repo and then multiple, multiple libraries that are used by that application. Um, and so with, with the standalone setup, you have one application at the root of your repo and then, and then libraries, um, kind of next to it. Um, but for, for our, um, Workshop, we're going to use integrated model repo, and then there's several other apps options here. So apps is, uh, basically, uh, bare bones, uh, setup, um, react, Angular, all these options down here. Kind of set up, preset up a, um, an application for you, but we'll, we'll start with nothing and then add what we want here. Um, let's see, enabled the should be caching. We'll, we'll say no for now. Later on, we're going to add this in. All right. Oh, it failed because I already have that folder. So we're going to delete that folder. Integrated. Uh, we want apps and no to that. Awesome. Um, So, um, if you're new to NX, you can go through this tutorial, but, uh, we won't do that right now. And let me switch over to a separate. Yes. Code window. Uh, let's see. Is this font big enough? Let me try that. Okay. Okay. Um, so this is what it generates. You got an apps and libs folder. Um, you got NX dot JSON, which we'll, we'll dig into a little bit later. And in your package JSON, you have just NX and neural workspace. Okay. Um, okay. So that's just the initial setup. Now let me go back to my, um, my slides here. And we have a little bit more, a little bit more talking to do. Okay. So these first few labs have a lot of, a lot of slides go with them, but later on, we'll be doing more, more hands-on work here. Okay. So NX the, the core of NX, um, manages, uh, your like the caching and the dependency graph and kind of how those, um, those projects depend on each other. Uh, what plugins do plugins add on another layer on top of that, they're, they're optional, um, but they can, they can manage your build system for you. They, they help manage, um, automatically, uh, upgrading things and they give you some, um, code generators to automate some of the repetitive, um, development processes. So in order to see what, what plugins you have and what plugins are available to you, you can run NX list, and that will list out all the plugins that are installed and also a bunch of community plugins and, and NX official plugins that you can, you could add if you want to. Um, so to add a plugin, you just install it with NPM or with Yarn and then, um, and then it's available to you now. Um, uh, so for generators, medics is not the right word here, but for generators, you can, you run NX generates. Uh, and then you give it the plugin name and then a colon and then the generator name and then whatever options you have. So for the, the Angular plugin, you do NXG for generate, um, at NARL slash Angular and then the generator name is app for application and then the name of the application, um, as one of the options here. Um, you can also run generators using NX console, which is a VS code plugin. You there's also a NX console for intelliJ. Um, I think they even have a VIM version. Um, VIM is not a, not as fully featured to this because obviously VIM has no UI, but there's a, um, like IntelliSense kind of features that, that come with the. They come from VIM. Um, so there's, uh, there's cogeneration for, um, you know, a bunch of different frameworks for, um, for backend frameworks like, like node and nest and next. Um, and for tooling like TS lint and prettier and, uh, testing tools like protractor and cypress. Um, you can also create your own, um, custom generators. You make a plugin inside of your repo and then set up generators that are specific to your organization. So you can either add on to like build off of the, the annex provided generators from, uh, from a plugin, or you can, um, you know, build up your own generators from scratch saying, Hey, create this file and, and move these, uh, this code around in this way. Um, and kind of instead of having a to-do list in a README file, that's like 10 steps long, you just have a generator that says, okay, enter in the name of the new component you're making and, or the new route you're making. And then everything gets set up the way it's, the way it's supposed to. Alright. Um, so it's not an Angular app, but that's fine. So step two lab two, we're going to make a, a node API here. So when this is done, the file structure will look like this. We'll have a, an API app underneath apps. Alright. So let's do, let's see if I have enough screen space to do this side by side. So if I run NX dash dash version, uh, what's going on here? Let me run npm install. Let's see what's, see what's happening here bamboo. Okay. Okay. Uh, the latest, uh, node 18 should work, but I'm, I'm having issues with node 18 right now. Let me just try, um, let me try 16. Or it could be my, um, it could be my machine is having weird issues. Let's try this. Nope. Let's try it on here. No such file or directory. Okay. I was doing this yesterday. So this is the curse of the demo. Everything fails when you run the demo. Let me try restarting the terminal. Okay, there we go. Good. All right.

5. Creating a Node App with NX and Plugins#

Short description:

Something was weird with my terminal, but now it's better. We can use any Node version we want. Custom generators show up in the NX console. We can add plugins using the NX list command. We added the node plugin and Express. We used the narwhal-node plugin to generate a node app called API. We updated the main.ts file and added endpoints. To serve the application, we can use the NxServe command. We fixed an error by adding another file. The API is working, and we committed our changes. We created a new application quickly without worrying about configuration settings.

Something was weird with my terminal, but now it's better. I'm using Node 18, but you can use whatever version you want.

Yes, good question. So Naya Kunin asks, do custom generators show up in the NX console? And the answer is yes. NX console. So we'll get to this a little bit later. But so you click on generate here. And this looks through your your repo and finds all the generators. So in this case, we only have our workspace installed. Later when we add other things, there'll be there'll be other other things in this list. And if you add your own custom generator, it'll also show up in this list. You know, whatever options you set up for your for your generator, will show up here as fields that you can fill in and then you click run and it shows up. Yes. Okay, cool.

Alright, so. NPX next we do this. OK, so let's do an X list to see what plugins we have, and this generates a whole bunch of code here. So I'm going to do an X list. What is the cat or is it? Settings Nope. An X list less. Pipe to less. There we go, so this is just the top here, so it shows you what's what's installed? Just an X is installed, and so we've got all of these plugins available to us. So these are the official NX plugins. And then there's a whole list of community plugins, with a short description for each of them. And there's a lot here. So what we want to add is the node plug-in, so we'll do NPMI-D, narwhal-node, and we'll also add Express. Because we'll use Express. So this is steps three and four here. So now we're going to go into step five, and we will use the narwhal-node plugin to generate a node app called API. So we're going to use that generator syntax, nxg at narwhal-node, and then colon, and then the generator we're going to use is called app or application, and then we give it a name of API. So if we wanted to use the NxConsole for this, so that's what the command line would be. So I can use NxConsole here and run generate, and so now instead of just having narwhal workspace, we also have narwhal-node, and the dependencies narwhal-node, the linter and JavaScript. So narwhal-node would create an application here, and so there's a bunch of options we can pick from, all we want right now is to set the name to API. And it does a dry run of this. And so let's do click run here, and it creates a bunch of files, it updates our package.json and updates NX, so let's look at the changes that it made. So in package.json it updated, just formatted that I guess, it added some dev dependencies to set up node for us, added Axios and TSLib, added ESLint for linting, some Prettier for formatting Jest, so all of these set up for us created an API application with some configuration options and an end-to-end application to test, to do run end-to-end tests on this node API. Okay. main.ts is kind of our entry point here. And so project.json here is the configuration that controls what build looks like and what serve looks like and lint and test. And so we can see here in NX console here, if we look under API here, we can run build, serve, lint, and test. So if we ran build here, it builds it and if we run serve, that launches it in kind of wash mode. And we can lint and test it as well. Let me go back to the lab here. All right, so we're gonna update the main.ts file to add some endpoints here. So we're not really doing, this is not a Node workshop. So we're just gonna copy and paste the code here. So this is using express and setting that for us. And here we've done that.

Okay, so now to serve the application, we can run. So we can either use the NxConsole or we can just run NxServe API. Oh. Okay. There's an error in the code that I copied. Okay. Main.ts. I think. There's another file that needs to be imported. Yeah, there's another file I need to add. I want to see where I added it. Here it is, this one. Games.repository.ts. So underneath this, I need to make a new folder, app, and a new file, games.repository. Yes. Paste that in. And save it. And then I'll rerun the search. Okay, so now that's good. Now if we go to localhost333 api, slash games. Sorry, let me move this over here. This is the JSON that we expect to see for the API. All right, so this is the excitement of backend development. You see some JSON and you're happy. But the front-end developers will be very happy to see that JSON coming through. Awesome, so we will go back. Okay, so that's working. And so let's commit our changes. So call this lab two. And that's working. Okay, so, what we did here is we created a whole new application. And it took us, you know, very little time. Can I, can I drag this over? There we go. It took us very little time just to spin that up. So we didn't have to worry about whatever's in here. We didn't have to worry about setting up ts-config or any of that stuff. And, you know, we could spin up five more applications in the another five minutes if we wanted to. So we're worrying about the content of our code rather than about the, all the configuration settings. All right, so let's go on to the next slide. Questions about that real quick. Let me look, look in Discord. All right, not seeing questions.

6. Understanding Executors in project.json#

Short description:

Let's talk about executors in the context of the project.json file. Each project has its own project.json file where you define build, serve, lint, and test targets. Executors are defined for each target, and you can easily switch them out for different tasks. To run an executor, you can use nx run or nx console. In lab three, we build the app using nx build API and explore options in nx console. The serve target uses the build target, and you can modify the configuration in the project.json file. That's how you can set different flags and configurations for your executor.

So that's good. See, do we still have 32 people? So we have, yeah, about the same number. Great, cool.

Alright, we will go back to Keynote here. Let's talk about executors. I briefly showed you the project.json file. Each project inside your repo will have a project.json file. Underneath your projects, you're going to define build, serve, lint, and test, or any other name you want. Each target has an executor defined for it. For example, if it's a plugin, it has the plugin name and then the executor name. In our case, we have Narwal esbuild as the executor for the build target. You can easily switch out the executor you're using for different tasks like building, serving, or testing. To run an executor, you can use the nx run command followed by the project name and target name. You can also use nx console, which provides fillable fields for the options.

Let's try it out. In lab three, we want to build the app. We can do nx build API from the command line, or we can use nx console. The console allows us to see the pre-filled options from the project.json file and make changes if needed. We can also set flags like bundle to true or false, which determines whether the TS files are bundled into a single JS file. The serve target uses the build target, so it first builds the application and then runs node on the output. You can change the configuration to production or development by modifying the serveTarget in the project.json file. That's how you can set different flags and configurations for your executor.

7. Running Executors and Configuring Targets#

Short description:

To run an executor, use nx run or nx console with the project name and target. You can also use nx serve or nx build with the shorthand notation. The options can be specified after the target name. The nx console provides pre-filled options from the project.json file. Executing nx build API bundles all TS files into a single JS file. The bundle option can be set in project.json or via the command line. The serveTarget uses the buildTarget to run the application after building it. Different configurations can be set for build and serve targets.

So to run an executor, you can do nx run, and then the project name which is API, and then the target which will be serve or build, and then whatever options come afterwards. So like nx run MyAppServe, or for the shorthand you could take the target name and put it in front. So nx serve MyApp, or nx serve API, or nx build API, and then the options come afterwards. So this works for whatever your target name is and whatever application name is.

The same thing, you can also run it with nx console, and it'll give you all the options as fillable fields in here. Okay, so let's try that out. If we go here on lab three here. So we've got executors. So we wanna build the app. So if serving the app was nx serve, so to serve the app, click here. There we go. All right, so serving the app was nx serve API. So to build the app, we'll do nx build API. Since we've already done it from the command line, I'm going to try doing it from nx console, just so you see that. So I can either click here and just click, like run here, and so that'll just run it without giving me the form, but I'm gonna go down to build here and do build API. And this gives me a bunch of options here. So these are already pre-filled in. So these are the values that are in the project.json file. So the main.ts file is here. So that's what's pre-filled in here. So it gives me the default that's filled in in the project.json file, but if you want to change it, you can change it here and then run it somewhere else. Looks like, this looks like an NxConsole error here. Interesting. But we're gonna try and run this. And so it built it and it put it in the, in the output path that we defined here. So dist apps API. So dist apps, dist apps API is here. And so this is our, this is our output.

So we built it. So we put stuff in the dist folder. Let's look under apps API projects at Asan, and it has an executive option that is there. And we want to, we want to change this to be, to be minified. So to bundle everything up. So let's, instead of saying bundle false, let's say bundle true. And see what that does to the, to the output. So if I run NX, NX build API again. So now if we look under dist here, so now instead of having that folder underneath here that had the, the game's repository, everything is built into one, one JS file. So this has, let's see what's the size of this? Oh, I don't know, 23,000 lines of code in here whereas if I went to project Json and set this to false, run this again, run this again, in here this main.js file is only 36 lines of code. And then there's the games repository and then that's it. So and then there's other stuff that is like the dependencies that don't get bundled in. So that's the difference. So you can set that in the project.js Json file or you could also set it in the, from the command line. You can do dash-bundle equals true here and that will bundle it into a single file like this. Okay. So that's how you can set you whatever, whatever flags you want in your, in your executor. You can also get. Let's see. Okay. Looks like autocomplete is not working right now for Nix console. That's fine. Nope. Okay. So let's look. So we've, we set the bundle to true. Instruct the executor to bundle all the TS files in a single JS file, which you did. So the serveTarget uses the buildTarget. So this build has, has options here. So let's close this down. So serveTarget down here has a buildTarget of API build. So basically what this is doing is saying, first build this application using whatever's defined up here, and then run node on the, on the output of that. So that's all service doing. So if you wanted to change this and do, so you can have a development version and production version. So this just runs the, the build with the production configuration, which is set up here, and then this serveTarget, serve with development configuration, runs the, the build for development and then runs node on it. So if you, if you want to set the NxBuild API, if you want to set the configuration to production manually, you can do it like that, or you can set it to development this way. Go left, and this way, maybe it's configure, sorry. I think you have to type out configuration all the way. There we go. So that ran it with development. So the default configuration is development here, and for build, the default configuration is production. So you can, you can change that, these names are totally arbitrary, so you can, you can set these to whatever names you want them to be. So that's how you can set up different kind of settings.

8. Setting Up Settings and Creating Libraries#

Short description:

You can set up different kinds of settings and overwrite options in configuration files. Janet asks about using the global version or NPX. NX can use either, automatically using the version set in package.json. Libraries are designed to be used by applications or other libraries, while applications can be deployed or run directly. NX operates on the library or application level. Splitting code into libraries allows for more caching and infrastructure. The dependency graph is on a project basis. NX can call package.json scripts. Get little tech is asked to share their screen to troubleshoot. Lab four involves creating a library.

So that's how you can set up different kind of settings. So every option underneath, underneath production is the same. So all these options can be overwritten in these configuration files. So what this does is it takes everything under options and keeps them the same except for whatever is specified here. So this just overwrites this ESBuildOptions and sets SourceMap to false and OutExtensions to the same thing. So it just, all it does is overwrite SourceMaps to be false instead of true. But if you needed to change something else, you could change something else.

So Janet is asking whether you should use the global version or using NPX. I like to use just NX directly. If you use NPX-NX, that's fine as well. NX will automatically use whatever version is set in your package.json. So if I have a global NX of... So right now, 15.9.2 is what's installed locally. And I'm sure my global one is not that. So if I CD out of here and I do NX-V, oh, it doesn't even tell me. But if I do NX-V inside a bg-hoard, it is... Maybe after you do NX-V. So here inside of bg-hoard is 15.9.2. If I CD out of here, NX-V, it still doesn't like it. All right, well... Yeah, so it says it can't find the global version, which is weird because I'm definitely using NX globally. But so NX will just use whatever's run locally. Whatever's there inside of this folder. Build for command line. Somebody's calling... Ah.

So let's do a... I'm gonna go through the slides and then get little tech. I will maybe have you share your screen while people work on Lab 4. Because I'm curious why it's not working for you. So let me do some slides for Lab 4. And then... Okay, so far, all we've done is created an application. We're an hour in, okay, we're good. So far, we just created an application. Next, we're gonna create a library. So the difference between an application and a library is a libraries are designed to be used by applications or other libraries. So they're not designed to be deployed. They could be published to npm to be used in outside the repo, but besides that, besides that, they're expected to be used by, the code is expected to be used rather than deployed directly. Whereas applications are, you can deploy them to host them somewhere or you can run them directly. That's what an application is. So typically you can have libraries that are features like a route in a front end application or their UI focused or data focused or just kind of like lower level utility libraries. So you can make libraries as granular as you want to. But NX operates on the library level or the application level for each project. If you want to use more of the NX Caching functionality then it helps to split things out into more libraries. So libraries can have basically whatever directory structure you want. You can have a folder, you can group them within folders. They don't have to be inside the libs folder but that's what the default is for this workshop setup. So you can have them at the root level if you want to or you can rename the libs folder to something else. You can call them packages or libraries or whatever name you want there. Typically, we see for large organizations, for each application you typically have a folder inside the libs folder. So libraries that are specific to that application would go in that folder. So you'd have like a libs slash API folder and then a lib slash, I don't know, front end or libs slash store if you had a store application, like a front end application. And then you'd have a libs shared folder that libraries that could be used across multiple applications would live in. So basically that whole structure is pretty much up to you what makes sense for your codebase. So generally speaking, the more you split code up into new libraries, the more caching will help you and the more kind of indexes infrastructure will help you. But there is a cost to creating new library. It's another, you're the code is being separated out into different locations. So you have to kind of know, you wanna group code together that makes sense to be together. But if it's, the same pros and cons that splitting code into separate folders or splitting code into separate files or separate components. It's the same thing with libraries. You can definitely go overboard on one side or the other. Just, and you can always change to the back. If you have a library that's too big, you can split it up. If you have too many libraries, you can combine them back together. So your dependency graph is on a project basis. So each library or application is a node in the dependency graph. And that's what we have for libraries.

So let me go back and answer some of these questions in Discord here. So I saw Toon is asking, can NX call package json scripts? Yes, yes you can. So this workshop is showing you an integrated style repo that's using Project JSON. But if you were to have, let me just show you real quick here. Let's see, file, package.json and default package.json. That's fine. I think I want to call this. I think it'll get confused if the name doesn't match. So scripts. So if you... So Project JSON is optional. You can just have a package.json instead of a project.json, and so let's see, or you could have both. So let's say, call this hello and we'll say echo hello world. Let's do hello from package.json. So if I run nx hello, because hello is the name of the script here, nx hello and then API is the name of the project. It runs hello from package.json. So whatever you put in here, it'll run for you. So yes. And then there was another question. And then yeah get little tech. Would you like to share your screen and show us what's going on with your VS code? For everyone else, feel free to get started on the lab, lab four, I believe is what we're on, next one. Yeah, lab four, where we're creating a...

9. NX Workshop Node Migrations and Lab Steps#

Short description:

In lab four, we dig into migrations and updating dependencies. We add an old version of NX Workshop node and then migrate to the latest version. The migrations.json file contains the code generators needed for each lab step. By running the migrations, we can complete the setup for all the labs up to a specific step. However, there may be some issues with the migration process that need to be resolved. We can also run specific lab steps by commenting out the unnecessary code in the migrations.json file.

Oh, lab four is actually, if you get stuck, so we'll go on this. Okay, so you've click run. What happens when you click run? Nothing is showing up. Yeah, exactly. Kind of doesn't seem to work, but maybe it doesn't. Try clicking on build in the lower down under generate and run target. Okay, click on super node, and then click on run here. Does that work? It doesn't work, but it actually ran from the terminal. Okay. I don't know. Yeah. I can figure it out later. Yeah, okay.

And Jannik is asking, if I'm building a front-end app, what's the best practice for structuring adding new features? Yeah, so typically, if you're adding a page or part of a page, you want that to be a library, not an application. And then you'd use that library within the application. So let me go over Lab 4 here. Lab 4 is digging into migrations. And also, if you need to jump to a particular, if you get stuck on a particular section of the architecture, of the workshop, you can jump to that step. So one of the things that NX plugins give you is the ability to automatically update your code, update your dependencies. And then update your code to account for the breaking changes that might have happened. So what we're gonna do is we're gonna run, we're gonna add NX Workshop node, an old version of it. And then we're gonna run, Yeah, and then we're gonna try migrating. So let's do this. Npmi-d. Actually at narwhal instead of the branch. Okay, so this is an old version of NX workshop node. So this could be any any plugin that we have. So that at narwhal, at narwhal slash node or at narwhal slash react or whatever plugin you're using. Let me delete this package. Jason here. And okay, so we've added this. Okay, and then we're going to commit our changes here, just to make sure we've got a clean setup here. And so then we're going to migrate to the latest version of NX Workshop node. So, NX has the NX migrate command at narwhal nx. So if you just do NX migrate latest, that will migrate NX itself and any plugins that are installed. So if I ran that, that'll do everything basically. And so let's see what it updated. So all of these are already up to date. And All I did is formatted it. Let's see, NX Workshop node is there. So let's do NX migrate narwhal NX workshop node. That should find the 1.0.0 version. Let's check that one. Let's see if I manually zip it up. Okay, there we go, so it found it. Okay, I think part of the issue there why that wasn't working the way I expected is when I set up the workshop, I published 1.0 first and then 0.0.1 later. So normally you just say NxMigrate and it'll find the latest version. So what that did is it updated, why did that not update then? Okay, so typically what it'll do is it'll update this. Oh, I see, it did update it. I just didn't, there we go. Good, it did what it was supposed to do. So it updated this to 1.0 and it also created this migrations.json file. And so these are all the code generators that are needed to move up to that version, that next version. And so we're kind of using a Frankenstein version of this. So typically this would be like modifying your codes to make any breaking, fix any breaking changes basically for that dependency. But in our case, what this is doing is each of these generators is completing the step for each lab. So this is completing lab one, this is complete lab two, three. Lab four is what we're doing now, so there's nothing to do there. And then this will skip you up to lab three and five and all the way up to lab 13 here. So if we were to run all of this, you can run index migrate dash dash run migrations. So that runs all of these steps set up. Oh no. So that does not, it's got some old code in there. Why does it have old code in there? Got no, Live Debugging. I should not need that. There's no Angular involved there. Or not. What if I had two? Migrations for that. Oh, just run npm i. Mm. Okay, um. There's something wrong with my migration, so I'm just going to run. I'm just installed that. There's no reason that that should be there, um, but just to move forward, will run this. Okay. Yeah. Uh, okay, so this ran through all the steps up to lab eight where it stopped. I got stuck on something, but you can see that here it made a bunch of changes, adding some apps and um adding some libraries in here. So this is all stuff we haven't gotten to yet. Um, so if you want to just run up to up to lab three here, I'm just gonna comment that out even though it's not valid. Jason. Um, gonna run this again. So this will just run lab one, two and three and get us back to the place where I was supposed to be. Mmm. Okay. Let me Remove those. All right. Nx console has a stale. I'm gonna run the run of the command. Nx G rm cli bansheet e.

10. Creating a Library with NARL JS#

Short description:

We're going to create a new library called off using the NARL JS plugin. Libraries and applications are similar in NX, but libraries are typically at the top of the dependency graph. They can have serve, build, and deploy targets just like applications. To generate the library, use the generate command with the NXG flag and specify the NARL JS plugin. This allows you to create reusable logic, such as authentication, and use it in your applications. NX helps keep your repository up to date with migration scripts that can be run to update your codebase when breaking changes occur. It's important to commit your changes before running generators and review the changes made. Now, let's move on to the next lab where we'll create the library and use it in our application.

Remove that. and remove the mootle interface library. And. And we are back. Okay. So this set us up, up to lab three. If we want to go to lab five, we can comment this out. So we can either do it this way or there is a generator that comes with the, um, with the. Workshop. Um, plug in here and next workshop. And next workshop, no to complete labs. And I want to complete lab, um, five. So if I complete lab five, you can do a range or you can do completely the lab I'm going to run that. And what that does is that updates my migrations as Jason file to just have completing lab five here. And then I can run an ex run migrations here, run that, and that will just complete lab. Five. So the idea here is that your, um, this helps you to, to keep your repro up to date. Um, basically the, the library maintainer can write these migration scripts. So when they know that they're making a breaking change, they can say here, anyone who's using this library, run this migration script. And, um, and you know, you can update without, without needing to know, um, all the nitty gritty details of the different, um, API changes that are happening. Um, so this created a new, a new, uh, library that we're going to make in the next lab. Okay. Let me go back to here. Um, start generating a some version, then change some things to my liking and then update. So for our case, um, the, um, so for this particular scenario, this, this NX workshop node, um, these migrations scripts, um, the, the first, um, complete lab, one is going to reset everything, delete all your code basically, and then, um, And then start from scratch. And then start from scratch. Um, so if you have code that you don't want to update, uh, you don't want to be, you know, deleted, then don't run these scripts. Um, or you could try just running the, like the, the single next lab. And that'll just try to add on. Um, code. Um, and always, if you don't like what a generator has done, just make sure you commit before you run it. And then you can always revert it here with your, your get history. Now that's always a good practice when you're running a generator, make sure your, you have a clean good history and then you see what it did and then you can undo it if you need to, um, okay. Alright, so let's move on. Let's do, we'll do one more lab and then we'll take a break. Think about it, maybe a 10 minute break. Um, okay. So let's, let's do the, um, Let's see, do we have that for. Okay. I'm curious if there are any, no. Okay. It looks like we're doing slides. Good. Alright. We are done with slides. Let's just do the next, the next lab. Okay. So far we've just made applications. Um, next we're going to make a library. Um, so let's say we have four API app. We w we have some authentication that we need to do, and we want that to be kind of reusable logic. Um, so this, this auth. Whatever authentication logic we're doing, um, we're going to keep that inside of a library. And, um, and then we're going to, um, Uh, then we're going to use that code in the application. Um, so, um, so that this is a good time to answer Jenick's question. Um, the, um, does, does NX, uh, differentiate between lives and apps? Um, the only difference, um, Basically basically, no. Um, basically they're, they're the same thing. Um, the main, the main difference is for, for the users of your applicant of your, of your code base. Um, so it's helpful to be able to see this, this is like the entry point for our, um, where the code base. Like, if you want to understand from a high level, what's going on in this code base, you look on the apps folder and you see, you know, everything that can be deployed. Um, and that's like a high level understanding what's going on. Um, You can make, um, libraries that have a serve target. Um, you can make, um, obviously you can make, have a build target on libraries. You can even have a deploy target on, on library so that you can do everything you can do with an application. You can also do with libraries. Um, but just typically, um, applications are at the, at the top of your, I haven't even shown this yet. Let me, let me show the Nx scratch here. Um, Typically libraries are at the top of your. Um, of your, your dependency graph. Let's zoom out a little bit. Um, so API is here at the top and then the library API auth that we're about to create, um, is at the bottom. Uh, these things we'll we'll make later. Um, but, um, the, you know, even in the, in the graph here, they, they have the same like symbol. Um, so basically there, you can be, you can treat them as the same thing. Yes, basically apps are, have entry points. Um. Okay. Alright. So let's move forward. Um, so to, to make a plain JavaScript, um, library, we're going to use the NARL JS package, NARL JS plugin. And, uh, that was already installed because we installed the, the NARL node plugin and NARL node depends on NARL JS. So you should already have NARL JS available for you. Um, let me get rid of these. Thank you for things. Those don't need to be there. Um, and the, so we're going to make a new library and we're going to call it off. So to generate it, we're going to use the, the generate command again. Okay. With that NXG, um, so the plugin we're going to use is NARL JS. So to know which, um, which plugin to use, um, So I'm going to run do generate here.

11. Creating and Using Libraries#

Short description:

There are two types of libraries available: JS and node. JS is for low-level JavaScript that can be used in a node or front-end context, while node is specific to node applications. To create a JavaScript library, use the NXG plugin with the library generator. If you need to remove a library, use the remove generator or omit the plugin completely. The API auth library is created and copied into the API folder. The auth endpoint is created in the main.ts file. Restart the TypeScript server if you encounter errors after updating the tsconfig.base file. The project graph shows the dependency between the API application and the API auth library. NXGraph is a useful tool for tracking dependencies in larger repositories. Commit the changes as lab five and proceed to lab six and seven.

So there's two different kinds of libraries that are available to us. They can do JS or node. Um, JS is for kind of, you know, low level, um, JavaScript that could be used in a node context or could be used in a front end context. Um, where is, you want to use node for things that are, you know, node specific? So node things can use JavaScript react, could use JavaScript, angular to use JavaScript. Any, any job, anything that's written with JavaScript to use a plain JavaScript library, whereas node would be specific for node.

Um, so to make a JavaScript library, we'd use, um, NXG at narwhals, that's JS and we'll call it. We're going to use the library generator. And we're going to call the library, uh, off. So I'm going to type in here. Um, what unit test test runner do you want to use? Um, I guess we'll use Jest, but you can use V tests if you wanted to. Um, what bundler will say, I guess, TSC. Um, But you could play around and use whatever you want there. Um, Uh, why so serious? You mentioned that your migration.json is not created. Uh, that means you need to run. You need to run NX migrate command. This NX migrate. Um, you can either say, migrate latest or migrate NX migrate at Narwal NX workshop node, um, at. 1.0.0 that will create your migrations.json file. Okay. So we've, we've made the, the, um, off folder. Oh, you know what? I made a mistake here. We've actually already created it. Um, and, but we want it to be underneath the libs API folder. So let's actually, I'm going to show you how to delete a library after you've made it. So you could, you could just delete this folder, but the problem with that is that, um, and X also created another entry in here. Um, and potentially has modified some other configuration files, um, with that, with that library. So if you just delete the folder, um, you'll have, you know, extraneous configuration to other places. So in order to remove it, there's a, a, a remove generator under NARWAL workspace. Um, R.M., um, you can do R.M. or do remove. Um, but, um, you can also just omit the plug-in completely. And if it, if there's no, um, so you couldn't do this, well, you could do this. Lib. If you do an NXT lib, it lets you pick which one, which library you want. Um, but since there's only one remove, um, remove generator, it knows exactly which one you want. Um, so if we do NXT, uh, lib, um, and we want to set the dash dash directory flag here, set its API and the name, which lib is the one that we want to call, which lib is the one that we want to call for icon. And the name of the directory flag, um, is the one we wanna use to set reply. So you don't have to go and re-write the command. Um, to really make it look like the same thing, you can just, you know, name is gonna be off. Or the other thing you could do is you could do, um, API slash off like that, and then it knows that the directories API, um, and it asks you which kind of library you want. I'll say JS and all of this. Um, and then it sees, oh wait, it already exists. So. We don't, we don't need to create it. Cause I already, already made it with the migration. When I ran the migration scripts with complete LabVive, it already created for me. So we'll leave that there. And then this is copying into API auth.ts. I'll show you what that looks like. So this is what was copied over. And then to use it in your API, auth endpoint, so back up here under api.src.main.ts. Here. So this auth endpoint here is created. So this has an error because we need to update our, we need to re-run the TypeScript, restart the TypeScript server, re-run the TypeScript, restart the TypeScript server. So every time this tsconfig.base gets updated, these paths get updated, the TypeScript server, it doesn't notice that it's been updated. So you have to run restart TypeScript server. I just did a command chip P and restart, and this shut up, and then that fixed that that Reds quickly there. This is how it imports the doAuth function from our library. So it's defined here under API auth, and it's exported here under index.ts, it says everything that's in API auth, I'm exporting and saying that this is available but to be used. If I were to comment this out, then this would fail because it's saying, Hi, I can't export anything. So this is your public API for your library. Okay, so what's next? And then let's launch the project graph. So let's do, if ever NX is confused about something, you can do NX reset which just basically tells it to recalculate. Okay, let's do this. I ran an NX graph. And so here we have our API application and the API auth library. So this connection here, it says this API depends on API auth because of the contents of source main.ts. Because of in here, because of this import line here, that's where that dependency line comes from. This is a very useful, especially in a larger repo where you don't know why there's a dependency line being drawn. It can help you track down, okay, why is there a Circuit Dependency? Why is this depending on that which is depending back on itself again. You can figure out what files are causing that to happen. Okay. So I did not change any of like project JSON files or I did not tell NX that the API is depending on the Auth library. All I did was write my code. And so if somebody else later on comes in and removes this line of code, then we run NXGraph again. Okay. I have to do a reset here. So there's some issues, I think. And let's refresh this. There we go. And so now NXC is, oh. They're not connected anymore, because somebody's updated the code to break that line. All right, so we'll put that back. And let's commit this as lab five. So let's commit this as lab five. Cool. All right, so we finished lab one through five. We'll be starting on lab six here. And then we'll dig in. So six and seven is finishing out the code portion.

12. Creating a Node CLI and Sharing a Library#

Short description:

In lab six, we add a node CLI. We create a new node app called CLI NXG app CLI. In lab seven, we share a library between the CLI and the API. We create a JavaScript library called util interface and import it into the API repository file. The games objects are given the type of game array. The build process runs without errors.

And then eight through 13 is focusing more on kind of like the architecture kind of structure of the repo.

Let's see, Andrix saying, Yes, so Andrix we will answer your question in lab eight. So we'll get there. Right, so let's get started on lab six here. And the answer is yes. So Lab Six, we're going to add a node CLI. So basically we're making another node application. Keep the instructions on the side here. So create a new node app, call it CLI NXG app CLI. So I'm not specifying the plugin so it should ask me. Oh, yes, so there's only one app generated right now in our plugins. So as to what framework do you want to use? I'm going to say none because I just want to be a CLI. And so it sets up with the name CLI. Let's update the main.ts file with the, here, copying it. So now we have two main.ts files so that we have the CLI one. Update that. And so this is not using any other libraries, so if we were to run nx-graph again, refresh this, so you notice the CLI doesn't depend on anything else, we just have an ntn application and then the CLI itself. Okay. And that should be, okay. That's not sick. That's lab six. Awesome.

Let's move on in lab. Oh, hold on. We skipped. All right, that link was wrong. We want to go to lab seven, not lab eight. Okay. Lab seven. Okay. Let me commit this. That was lab six. Very quick lab six. If we wanted to... Let's let me just show off real quick to run the the main.ts file. We would do nxserve, would run it. So if I do that nxserve-cli, oh, okay. It's trying to make a, trying to make a request to localhost 333, but the API is not running. So if I, I just served the API, and then in a separate separate terminal do serve-cli. Okay, so here we go. So it listed out, it made it, made a, API call and then listed out the, the name and description here. So it did work the way we wanted to. Looks like the slash end isn't working in the console here. So if, if it, so it's listing all the games here. So that is working close enough for a demo. Let's move on to lab seven here. So now we want to share a library between two different applications. So sometimes you, you want to have some kind of connection between the, a front end and a backend application. So the CLI is our, is our front end. And when that interface changes, we want both the front end and the backend to make sure that they're matching that interface. So we're gonna make a TypeScript type to define what a game is. And both the CLI and the API will use that TypeScript type. And so that way they can be kept in sync. So whenever that contract changes, they're both in sync. So that's what this Lab 7 is all about. So stop serving it. So we're gonna make a new JavaScript library, we'll call it util interface. And we're gonna make sure to stick it underneath the API folder. Then later we'll move it around to move it somewhere else. So let's do, so nx generate a library. We're gonna call it API util interface. So here's my shorthand to make sure to stick it underneath the API folder here. Okay and then because I didn't specify the plugin, I'm gonna ask me which one I want. So I wanna use js, not node. Then I don't care really which of those. So under apps we've got that, so now under libs we've got libs api-auth and libs api-util interface. Okay and now let's see what's next. Okay, so when I make the game interface here, okay my link is not correct. Source lib api-util interface and then slash maybe. Nope, okay. I have to go find it. Samples lab7. This is the one, api-util interface. If someone wants to get bonus points, they can go in and fix the broken links in my code here. Okay, so update that. And so we've got game being exported for api-util interface. In here. Okay, so we made that. We want to import it in the API repository file. So basically we want to import it with this and then use it on this line here. So we'll copy that import statement. So the games.repository.ts, that's not what I wanted. This, so this is under the API. Okay, and then we're gonna give this games objects the type of game array. Okay, so now I'm importing from the library that I just created and then using the type that is defined there. Right, so that one is the player. Then we're gonna build it and make sure there are no errors. So, NxBuild API. Okay, good. So, notice here, when I ran NxBuild API, at first it was building the util interface library because util interface also has a build task to find.

13. Building and Moving Libraries#

Short description:

The build process for the API takes a longer time compared to the build for other parts. Moving the util interface library up a level allows it to be used by both the API and CLI applications. The API and CLI applications are now dependent on the util interface library. Lab 8 introduces the concept of module boundaries, which define rules for dependencies between libraries and applications.

So it does the build for this first and then it does the build for the API itself. NxBuild API is that. And then if I run it again, it read that from cache and I read this one from cache. So it took no time at all, whereas this other one took, I guess, two seconds. So it's not, it doesn't make big difference for this small example, but for a larger app, it'll make a huge difference. Your cache version will still come back in what, 49 milliseconds. Whereas, you know, the real build step could take, you know, two minutes up to 30 minutes, depending on what you're doing.

Okay, so we built that and then let's do the... Let's check out the project graph here. Okay, so here we go. Let's refresh this. Okay, so now we've got this new library created and APIs depending on it. That's good. Now what comes next? Let's commit everything, okay. So we'll commit under lab, lab seven. Hm-mm. Okay, so CLI makes calls to the API, but it's using type any, so let's use the type in our CLI as well as the API, but the issue is, we could just use it, but the folder that it's in kind of doesn't make sense. The idea here is that everything in this API folder is only used by the API application, so we want to move this util interface library up a level so it's directly underneath libs instead of underneath the API folder. So if I were to just move this folder, there would be problems with that because a lot of the configuration options wouldn't be updated correctly, so we want to use the move generator to move our library, so let's do this. So you can, so I can do NXGMOVE or just MV, and I'm just gonna run it like that. So it says, so it's running that, so it needs a project name, let's actually do this with NX console so I can figure out what I'm doing here. So MV here, MOV, there you go. And NX console has a automatic dry run feature, so I wanna move this util interface, and I wanna move it instead of API slash util interface. Let's see, okay, here we go. The issue was that this dropdown here had a stale list of projects, and so I had to go into NX console and click refresh here, and then reopen the generate tab. So, typically, you don't have to do a lot of this refreshing stuff because you're not making new projects every 30 seconds, but in this workshop we are, so. So, API util interface, and then we wanna move it to util interface, and then it's running this dry run here, and so you can see the files that it's creating. So, it's deleting that and moving it, so API slash and deleting that, and moving up a level. So, that looks like what we want. If we wanted to we could also change the name here, so just like the Linux MV, that's how you change names, so that would make it util interface change name here, or you could move it into a different folder here. So, you can do whatever you want. So, if you're stressing out about what you're naming the library or where you're putting edits, don't worry about it, you can always move it later. So, this is what we want, we're going to take it from the API folder and move it up a level. So, we'll run that, and now we can see the changes that were made. Um. Okay, so I moved it from the API folder, now it's down here at the top level, which is good. And now we're going to start using it in the CLI. So we're going to take this and then I assume, formatting issues here, but that's okay. Here. And in the main.ts of the CLI, let's see, so this val is not an NA, we're going to call it game. And we need to import game here. Okay. So now if we go back to our instructions, let's look at the games repository.ts. If you notice in the, this import path was automatically updated for us. They used to be API slash, and that got updated when we moved our Util interface. Also, the path here got updated. So that got taken care for you, for us, by the Move Generator. Okay, now if I run, okay, I'm gonna fix this alignment, because this is... Change, nope. Is it here? Right-click. Three dots, nope. Nope, that's not it. Let's see if I can find... I don't want that. Hmm. Here we go, line panel. Panel position, bottom. Okay. All right, what was I doing? Run the next graph. This one. So now I refresh this. Okay so now the CLI and the API. All right. I have to do an NX reset here again. So this does not typically do this. So this, I think there's an issue with the latest version of NX. But so now the API is dependent on the API, so now the API is depending on the util interface and the CLI is also depending on it. So if this ever changes then both of these need to be updated to reflect the new interface, which can solve a whole bunch of issues that you can find in production or whatever. All right. We got that. Anything else in this lab? No, that's it. That is lab seven. I'm gonna commit this lab seven. And all right, now let's check out lab eight. So lab eight, we do have some slides for it. Okay, so let's say you've got a setup like this, where you've got multiple different frameworks being used in your repo. So you have Angular, you've got Remix, you've got, I'm not sure what, maybe this is Nest. I'm not sure what that... what syntax is, but some backend API, and you've got libraries that are all, they're all used. What we want is to have some kind of rules saying that the Remix code can only use Remix code and plain JS code at the bottom here. Angular code uses Angular code and this stuff, but you want to have some structure to this and have rules about which libraries can depend on other libraries. So that's what module boundaries take care of for us. So if the API starts depending on something in Remix, we want to be able to say, hey, don't do that. Or if the plain JS starts depending on API specific stuff, that's a no-no. Those are apps, those are libraries. So these are like our named apps. So this is an admin app, this is a store app, this is an API app. And then we've got shared stuff at the bottom. So apps can only load libs, libs can only load libs. So we want to set up these rules.

14. Setting Up Module Boundaries and Tags#

Short description:

You can define tagging systems for your repository to group things by type, name, or other dimensions. The Lint rule 'enforce module boundaries' in NX allows you to set up constraints on dependencies. You can define rules for specific tags, such as allowing a library with the 'util' tag to only depend on other libraries with the same tag. You can also set rules for external imports and banning transitive dependencies. By defining module boundaries, you can prevent circular self-dependencies and control the dependencies between different parts of your repository.

So whatever tagging system we have, we're gonna be able to define for our repository, like group things, by type, or by name, or by different dimensions. And then be able to set up rules about what those constraints are, about which things can depend on other things. So there's a Lint rule, called enforce module boundaries, that comes with nx. So you can say things like, if a library has this tag of type colon util, which is this can be whatever string you want in here, you can say type util can only depend on libs that also have that type util tag. And they cannot depend on anything with react, like any dependency, like NPM dependency, anything with react or anything with the Angular namespace. So that's what this Lint rules is for, enforce module boundaries. There's three different kind of sub categories within this Lint rule, you can say only depend on libs with a specific tag, you can do the inverse of it, cannot depend on libs with a certain tag. And you can do, have rules about external imports, which is like NPM dependencies, things outside of the repository that they're not allowed to depend on. You can have a rule about banning transitive dependencies, say if you're using, if you're depending on, I don't know, express, which is using some other dependencies like I don't know, choked or something. And then you're using choked or inside of your own, inside of your application. It could work, it could build and compile and work correctly. But then if express stops using that dependency, then you'll end up with a broken application without realizing what happened. Let's see. So you can say that don't do that. You can check for nested external imports. I'm not sure exactly what that's about. So if you want to, by default, it doesn't let you do a circular self-dependency. So if you depend on a library that depends on the first library, it'll stop you from doing that. But if you already have those and you're trying to adopt Nx, you can say, OK, just allow that for now until I figure out how to solve that issue. So that is the slides for module boundaries. Let me look back here. Yeah, so Jannick, if you're seeing things that shouldn't be there, you can run NxReset to reset that. But it looks like you figured it out. Anyway, Jannick says that Nx seemed to know it was a library because it kept it in the libs directory. Yeah, so typically, if things are already libraries, then it'll keep it in the library folder. There is a property here under Project JSON, project type, this can be application or library. And that just determines which folder it goes in when you're using the integrated setup. But you can just freely change this if you want to. So here, this is the project library. That's where Nx knew it. So let's do the next lab where we set up some of these module boundaries. So to set up tags, we'll go into the project.json file for each project. And we're going to set up some scope tags and some type tags. So for scope, we're going to set up a scope of CLI, a scope of API, and a scope of shared. And then for type, type could be application, type app. And then for libraries, we can do type util. Let's see. So this is going to be a type util. And then auth, what should we call this? We can give this a type of probably data access. So in here in this tag here, we can do this, you know, whatever strings we want in here. So I'm going to give this a scope of API, give this a type of data access because it's accessing the authentication data. And then this one, this is the util interface library. So this is a scope of shared, and give this a type of util. And then we've got two more projects to update. So this is a scope of CLI, and a type of app. This is the API app, so we'll give it a scope API. Type app. We could do the same thing for the end-to-end applications, which we haven't actually used at all. Does not have tags. OK. If there is a project here that does not actually import, so this end-to-end application does not actually So this end-to-end application does not actually import any code from the API app, but we want to run the end-to-end tests whenever this API app changes. So you can manually add a line in the NXGraph by defining these implicit dependencies here. This is just manually saying, I want a line where API ETE depends on API. So there are cases where you need to do that. So I'm going to add a tags property here. And so you notice in here, we had some auto-complete. Because this is coming from an NX console, reading the schema for the project JSON and knowing what properties go in here. So tags, this is, I guess, scope API. Because it's an API and test. And then type, well, it gives us a type of ETE because it's different from an app. But you can name these whatever you want, whatever makes sense for you. Basically, based on whatever you want your rules to be. That's how you should set up your tags. Scope CLI and type. Let me actually run the NX graph again. And in the graph, it actually shows you what the tags are. So if you click on these things, it shows you scope API, type ETE. And this one is scope share, type UTIL. So this can be useful. So now I think it's going to have us do the. So we set up all the tags here. Next, we're going to define our dependency constraints. So in the root esland rc.json file, we're going to add this. We're going to modify this rule and set up some dependency constraints. So I'll copy this one. And then we'll ESLANDRC.JSON. OK, so this is our rule here, NX Enforced Module Boundaries. And dependency constraints. So this default setup here is saying any project can depend on any other project. So we want to be more restrictive than that. So we want to delete that and add in our own rules here. So if it's the scope of CLI, let me minimize this. So the scope of CLI can depend on CLI and share. OK? So then let's add some more. If it's a scope of API, then it can depend on other API things and shared. If it's a scope of shared, it can depend on other things that are shared. That's it. Now we'll set up some rules for types. So if it's a type of app, that actually should be able to depend on. It can depend on anything.

15. Linting and Disallowing Imports in CLI#

Short description:

We tested the linting and made sure it is working. We also demonstrated how to disallow the import of Express in the CLI. Target names are arbitrary except for the build target. When splitting an existing application into libraries, you would run 'index init' to set up a base index.JSON file. Then, you would generate a library for the desired folder and manually copy over the code. Finally, you would update the references in the app. This process requires some manual work due to the complexity involved.

So we can just leave that unspecified. Yeah. So we can just do a type data access. Can depend on type data access or type util. And then type util can only depend on type util. All right, so let's just do a sanity check here and do So I could do an Nx Lint API to run the lens on one particular project, or I could do Nx run many with a target of API, and this by default will run, not targeted API, targeted Lint. And this will run the Lint target on everything in the repo. You could also specify a list of targets with a list of projects with a dash dash project flag, but we want to run it on everything. So let's do this. Okay, it looks like everything is still happy, so let's see if we can break this Lint rule and make sure that it's actually working. So let's make a util library use a data access library. So we'll go to the the util interface here, and in here we're going to add an import, we want to import the doauth function. Okay, so it imported it, it found it, which is nice. So now let's run the Lint rule on, actually, yeah, it wouldn't, in the nxLint util interface, we have an error. So it says buildable libraries, cannot import or export from non-buildable libraries, okay, that's a different error than I expected. So that's an issue where this one has a build target, whereas API-Auth does not have a build target. We'll just add something here, it'll skip past that. So we added a do-nothing build target here, so this is the error that I was expecting. The project tagged with scope shared can only depend on lives tagged with scope secure. Well, that's not exactly the error that I was expecting, but it also is valid because this has Util Interface is has scope shared and API-Auth has scope API. So that should not be allowed because of that reason. This rule was also violated, but I found this one first. So we could do the same thing. Yeah, we could have a CLI importing the do-auth function and that would also violate this rule here. These rules are arbitrary, you can set them up however you want. But this is a powerful feature that can impose some structure to your codebase. Yeah, okay. So could you disallow express from the CLI? Let me answer that question first. If you did, let's see. How did you do that? Yeah. So you do this. So source tag scope CLI. We can add another property on here. I guess it doesn't autocomplete. We want to add a band external imports. I think, is that it? Let me look up the documentation here. Nx.dev is the documentation site. I want to go to packages, and I want to find the linter. Linter here. And then, I'll just do a search for in force module. Sure. I'll do a search for band external imports. So this is under packages. Oh, here it is. So it's the eslintplugin.nx package. So here are the options here. So we can add only dependent lives with tags. That's what we've been using mostly. But we can also do band external imports. So this is an array of packages. So here, band external imports. So that's correct. And then, it's an array. And then, I could put in express here. You can also use a wildcard in there. Express star. So now, if I go into cli here, and I want to import from Express. It doesn't like it. So it says, hey, don't do that. Not allowed. Yes. So Janik says target names are arbitrary. They're arbitrary except for the build target. There is some extra logic around build. I think that's it. We have a term called buildable library, which just means it has a build target. We also talk about publishable libraries, but I don't think we have any special logic around that. It just refers to libraries that have a publish target. So I'll just do a shorthand for that. There was another question up here. So Janik was asking about how do you recommend splitting up an existing application. So for an existing application, you would run. So just to get started, you'd run index init to just install index and set up a base index.JSON file. And then once you have a folder in here that you want to turn into a library, what you would do is so if I wanted this games repository to be its own library, I would do basically I would index generate a library and call it a repository, maybe games repository. And then I guess it would be JS, whatever you want it to be. And then you just basically have to copy over this code into that newly created library. So you'd copy that over and then all references to that you'd have to manually update inside of this app itself. So there's a little bit of manual process involved there. Because there's too much complexity to automatically do that for you. OK. So we added that. We created this. We did run many. We tested that. We tested that. We tested that. OK. Let's just make sure our linting is all working. Nx run many. The other thing we could do, I want to show this up actually.

16. Nx Affected Command and Setting Up CI#

Short description:

I want to show off the Nx affected command. First, I need to make sure that the linting is working. After fixing an issue with the lint rules, the linting is now working correctly. The Nx affected command allows you to run a certain target on projects that were affected by a code change. It looks at your git history and can be configured with a base branch. This can significantly speed up your CI time by skipping unnecessary tests. We have time for one more topic, and the audience voted to focus on setting up CI and distributed caching. We create a CI workflow file that runs npm install and tests for the CLI and API. We commit the file and push it to main. That's it for lab 11, and now we can move on to the next topic.

I want to show off the Nx affected command. All right. Let me just make sure that the linting is working first. Nx run many. Target equals lint. One failed. I need to fix this. So the VS code hasn't caught up with our new lint rules. So if you want to fix that, you restart the ES lent server. And now it has the correct squiggles. That's because we changed the rule. And it just needed to be updated. Okay. So if I fix that, now our lint should be working. Now, you notice how quick that was? That's because most of these things were cached. These five lint targets were already cached, and it just needed to run this one that I just changed. So if I run it again, it's instant again. Okay. So there's that. Now let me commit this, and then I want to show off the nxaffected command. How does this lab 8? So let's say if I were to undo this, I could, instead of doing nxrunminny to check and see what I need to run, I can run nxaffected. And so this command runs a certain target. Target equals lint. So this is saying to run the lint command on any projects that were affected by a particular code change. So right now, I've affected this file. So it looks at your git history. And you can set up a base branch that you're comparing against, if you want. You can set up the base and the head. But default head is that. And actually, I think default head is your current working setup. And then base is head, I think. Yeah, head is I don't know how to say in Git whatever's currently in your file system. But so if I run this, it ran. Let's see, nx run, nx affected. So it ran. So this util interface, so it ran it for a CLI. Ran it for API end to end and API end to end. So ran it for five projects, which makes sense because this game's repository is unaffected. And let's see, what else is unaffected? API auth is unaffected. There's no reason for API auth lint to be affected by this code change. But these other ones, they could potentially be modified by the lint could fail because I've modified some code here. So that can significantly speed up your average CI time because you're not running tests they don't even run. So that's lab eight module boundaries. We have a half hour left. Let's look at what's left and be strategic about what we want to do. So I think we have time for let's do a, I'm going to stop sharing my screen. And for the people who have stuck it out so far, you get the power to vote on what we talk about next. So we can either work on making your own generators that are local to your repo, or we could work on setting up CI and distributed caching. So we're going to do the use the Zoom, raise hand. Let's see where's the Raise Hand feature. Thought it'd be a reaction, but there it is. Yeah, under Reactions. So you can raise your hand like that. So raise your hand if you would like to do. So the options are generators, custom generators, or CI and distributed caching. So raise your hand if you want to do generators. Custom generators. So I've got two votes for generators. OK, three votes for generators. You can unmute yourself if you're frantically trying to find the Reaction button. All right, let's do votes for the CI and distributed caching. 1, 2, 3, 4, OK, it looks like distributed caching wins it. All right, so if you want to look at the generators, you can do that on your own. That's labs 9 and 10. But I'm going to skip to lab 11. So lab 11 here. Let's set up CI. OK, so this is setting up distributed caching. Setting up CI. Actually, hold on, did I? Yeah, that's right, OK. Lab 11. Hold on. OK, so we're on main. We need to commit it. So we're going to copy this and we're going to create a workflow.ci.yaml. What was all of these? That should be under.github. There. New folder. Here..github, and I think it's workflows. And then a new file. Ci.yaml. Paste that in. OK. So what this is doing is, it is, whenever there's a PR, we're running npm install. We're checking out the latest. We're running the tests for the CLI and running the tests for the API. OK? So we do that. And then we commit that to main and push it up. All right. So let me get to, is this 11? I don't remember.

17. Pushing Changes and Updating CI#

Short description:

We push the changes to the repository and create a pull request. The unit tests are failing, but it turns out there are no tests. We update the CI configuration to use nx affected instead of specific targets. We commit the changes and run the tests. The tests pass, but it's clear that our unit tests are not sufficient. We discuss running tests in parallel and add more steps to the CI configuration. However, there is an issue with the main branch not being checked out in CI. We make the necessary adjustments and continue with the process.

Push it to. OK. So we push that up to my. OK. That's working. That's pushed. Add a CI. OK. So we're going to open up a new branch. We check our dash b. Modified. And then new branch and update our main.ts. Make some change. Let's do, actually, the cli, main.ts, because this would actually affect something. OK. So that's the new message that'll be displayed on the cli, the terminal, and committer changes and push it. OK, changes made. These are really great commit messages. It's like the Hands. If you've seen that xkcd comic. So we push it up. OK. And then we're going to go to our repo here, bg work. First, use the slash. OK. So we're going to make a PR here. Great employee request. So we've made that. And then we'll see this. Unit tests are failing. Let's see. So it's running this. So it's running the test CLI, running the test API. And we'll come back and check on that in just a little bit. So we're setting that up. OK. So we want to update this to only use index affected instead of specifically. Instead of doing two different jobs here. So yes, thank you. The xkcd link. So I'm going to update the CI.yaml. So instead of test-cli, we're just doing test-affected. And instead of running npx nx-test-cli, we're going to do nx affected dash dash target equals test. All affected. And then we don't need this any more. So that's the change we'll need to make. Let me see if I should make it on main or in our branch. Let's. Ok, so let's double check this. What's that, it passed. Interesting. It looks like our unit tests are not very good. Um, so it ran yeah, the x, it next tests cli, Oh, no test found. That's what it has passed. Well, good job. Good job. We have succeeded at not running any tests. That's the best kind of testing. So, um, you know, annex effected is, is not helpful. If you don't actually, uh, we don't actually have any tests, then it's, you know, you're living in the wild west at that point. Um, okay, so let's keep going. Okay. Okay. So we made that, that job. Um, testing, I've been running sequentially for each project. You can run them in parallel. Um, so by default, um, dash dash parallel is set to three. Um, if you want to make that more, you can, you know, change the number. Um, or I think you do parallel equals one. Make it in sequence. Um, so let me, let me commit this and. Okay. So I'm committing that and then we'll see, um, let's see what it does over here in just a little bit. Um, okay. So only those testing, so we can also, we have it do Lint ET and build. Um, so we can add more steps in here. So ideally we'd want to do, um, that and then do, uh, Lint and build. Um, I'm not sure if this is in the latest NX, but soon you'll be able to do this. They'll, um, instead of having four different minds. Um, but, but right now I think you have to do this, all targets. Let me just check the, this one here. Okay. Okay. So this one has updated and so not target test is failing or is it failing numbers are get main on revision or path not in the working tree. Oh, okay. So that issue is because we don't have the, the main, main branch checked out and CI, so we need to, let me see in here. Make sure that this is set up correctly. That's right. That's right. Let's check it out.

18. Setting Up CI and Distributed Caching#

Short description:

This lab focuses on setting up CI to use NX affected and running only the necessary code tests. Another topic covered is distributed caching using NX cloud. The lab demonstrates how to specify cacheable operations and provides an access token to connect to NX cloud. The results of the run, including cache hits and misses, can be viewed in NX cloud.

Whatever branch the PR is using and CA I probably don't have last branch backwards. You don't want to check out every branch. I think that's what this does. So in your CI, you probably want to do something differently. But Go back here. And okay, where is the? Where are we getting the actions? Get hub actions. We'll go. For some reason, it wasn't showing up in that invalid workflow file, 913. OK, so it doesn't like this. So I need another indent. Maybe. Let's look at that one. OK, so it likes the YAML file. That's good. Still called test CLI, and that's because I never didn't change that name. But there's only one of them, which is what we expect. And it failed because, OK, so it's still not checking out the main branch here. The base needs to be available to it. OK, well, we have to update that. Before you run NXEffected, you need to do this. OK. So I found this from the docs here, looking for GitHub actions. I found this from setting up that, setting up that, and then there's this node here. If you're using the action in the context of a branch, you need to add this, get branch track main, origin main, before running NXEffected. Since the origin main don't exist. Let me just make sure we're also using that. So this action here will help you be able to check which you can run affected based on the last successful deploy if you're doing CI. In our case, we probably don't need it because this is just for running test and letten build. But for a deploy action, we'd want to do based on the last successful deploy, not just the last mid on main. Let's push this up. Right, so between our deprivation and user and user only, we'll push the first two. This is the hour. Let's skip a little bit. And this is a good one, because there is a different Joker that you can send instead of the last delegate and deploy. But let's just say we've got a Joker and we want to put it in our build, and then we'll return our response. So we're going to send her. Okay, let that fail. Okay, something's weird here. I'm going to bail and go to the next lab. So basically, this whole lab is about setting up your CI to use NX affected. And then running the, running only the, only only testing the code that you need to test. So, that is one aspect of setting up CI. The other thing I wanted to show you to make sure we don't run out of time is in the next lab. Lab 12, distributed caching. So, you can either run this or you can run NX connect to NX cloud. And, so it's basically saying yes to the thing you said no to before. And what that'll do is that adds a, another package here, this NX cloud package. And it adds some settings to your NX.json file. It changes the runner from the local task runner caching system to using the NX cloud distributed caching system. So, and then you also, I think these are just the default cache operations. So you can specify which things make sense to cache here. So build typically is cacheable. Basically it's anything that is, will only change based on, the output only changes based on the inputs. Things like serve don't make sense to be cached here because there's no output to actual cache. Things like anything that's like making a network call. So sometimes ETE tests should not be cached if they're making a network call to some other server that you're not, like controlling and the intent test. But it's up to you how you, depending on how you wrote your intent test, that could be cached or could not be cached. I don't know, there is some other things that don't make sense to be cached. Like a deploy, you don't want to cache your deploy step. So you define what should be cached here and then this, the adding in X cloud also gave you this access token. And so what we're gonna do is, so when we ran this, it sent us this link here to claim our workspace on NX cloud. And so I'll just create a cloud. I'll just continue with Google and I'll use my normal account. Yes. So this is the access token. So this is the one that's from the NX.json file. It's this access token. Say, hey, this is my workspace. So let me connect it. Okay. So now what this does is when I run, I can run NX build CLI here. So it successfully ran two targets. This message was not here before. So if I click on this link here, bring it back over, I can see the results of this run. So this means it was a, let's see, it was a cache miss here. This check means it was a successful run. They all took less than a second, even though it was a cache miss. But that's the information about it. Now I could also, I could run this again, and this time read the output from the cache for a three out of three tasks. So I'll look in here again. It keeps opening it up in a separate window here. Still a success, but now this icon here changes. It was a local cache hit. That's what this icon here means. So it's read it from my machine and loaded up. Now, I can, I'm going to do this. Strict in caching Say long commit, sure. Push that, push that up. Actually, let me check out main. Um, Well here, I'm going to merge this in and then check it out.

19. Using Remote Cache and NX Cloud#

Short description:

Merge and confirm the merge. Check out the code in a different folder and demonstrate the cache mechanism. Run NPM install and Nx build CLI. Download my repo and run Nx build CLI. Set up a new token. Commit and push package lock changes. Run Nx build CLI again. Clear the cache using NX reset. Run Nx build CLI to fetch the remote cache. The report shows that two tasks were done remotely. NX cloud is a paid product for smaller repos.

Merge that in, confirm merge. Now I'm going to, what I'm gonna do is I'm going to check this out in a different, um, In a different folder. Copy that. And I'm going to show you the, The cache mechanism. It clone this and then BG hoard copy. I'm going to code, BG hoard copy. Okay. I'm gonna run NPM install to get everything set up. And what was the command I ran, ran? I think it was build the CLI. Yeah. Nx build CLI. Okay. So I ran that and now I'm going to run Nx build CLI here. And so this is a completely different, repo here. I could run this on a different machine, but it'd be harder for me to set up. But if you, if you want to test this yourself, you can download, download my repo and run Nx build CLI. Now that's a full repo. Download my repo and run Nx build CLI. And so successfully ran the target build and one task depends on the zero run run from cache. Let's see. I wonder if that's because I have a read only token set up. Okay. So this was a cache miss and it ran it all again. So let me see. Okay. So I'm going to set up a generate a new token here. So the two different kinds of tokens, EG horde settings, manage tokens. There's two different kinds of tokens. Now this is a read, write token. Why is that different? Hmm. I wonder if it, oh, there's a change here. What's different. My package lock changed. Okay. So that broke the, um, the next build CLI here. Huh. Okay. I'm going to commit this. Package lock changes. Push this up. So because the package lock changed, um, it, it should, it correctly broke the cache is correctly said, I need to run this again because, um, if there's different, uh, a different package lock, then that means we've got a different. Different inputs coming in the system. Um, so I'm going to run this index we'll see lie again. Okay. So that's correctly ran it, read it from cash. Let me go back to the now. Now we're on, I'm on the original repo here. Um, and I'm going to. Um, get checkout main. Oh, I think that's, that's the issue. Um, I was on a different branch and there's some other things that were different, um, and if you install, okay, so nothing, nothing is different here. Okay. Um, so by default are, um, the cache is stored inside of node modules. Um, dot cache is NX folder here. Um, and this has a hash for each, each command I've run and then the outputs of them. So the terminal output and the. You know, the dist folder outputs for each of these things. This is where things get red from. Um, so I'm going to run NX reset which will clear out that cache here. Show it to you, I didn't. Um, NX, uh, there's a command for clearing the cache. Let me find it. Um. Or I can just delete the folder, but, um. I wouldn't. Maybe. Maybe it's just. Okay. That was just my file system. The VS code thought it was still there, but it wasn't. Okay. So NX reset did, did get rid of it. Um, so deleted that folder. So now if I run NX build CLI, it should go fetch the remote one that was created, created here. Yeah. And, and loaded in. So the next bill CLI, um, there we go. So this one was loaded for the remote cache. Um, so read the output from the cache instead of running the command for two out of two tasks. Um, it reused two tests. So it did two of them were, were done remotely. So if I look at this, this command, this, uh, report here. Oh yeah. This report. So this, this icon here says it's a remote cache. It's not, instead of locally on my computer, it was remote pulling it down from, from an expert and X cloud. So, um, and then you can, you know, get the terminal output if you want. No terminal output for that one, this one had termed up. So NX cloud is a, it's a paid product. Um, but it's, um, for a smaller repos.

20. NX Cloud: Distributed Caching and Task Execution#

Short description:

NX Cloud offers distributed caching and distributed task execution. Distributed caching helps with the average use case, while distributed task execution parallelizes tasks in the most efficient way possible. NX Cloud is the only paid portion, and NX itself is completely free. The community offers resources such as the Community Slack, Twitter accounts, YouTube videos, and GitHub interactions. Thank you for your attention and for watching until the end of the video.

It's, um, there's, there's a, a large free tier. So if you have like 10 developers, um, you'll, you'll probably never, never go past the free tier. I think we got, uh, 500 hours of, of compute time, um, of, of time saved per month, uh, free. Um, so typically. You know, a small organization, you'll, you'll never hit that. And it's free for whatever size, if you're, if it's an open source repo. Um, so for, for larger, um, organizations, there's a, there's a paid paid version. Um, and there's also an enterprise version. If you want to. If you want to host the cash on your own servers, um, that you can enable that way. Um, so this, this sets up so that, you know, your across your whole organization, if you, if anybody ever, um, runs a build on a particular set of code, um, you can share that work for anybody in the whole organization.

All right. I've been talking for a long time and haven't looked at, um, looked at the questions. So let me see if I've missed anything. Um, All right. I am not seeing questions. So, um, we, we've got like a few minutes left. Um, what, uh, what other questions do we have? Is there any way to do distributed caching without NX cloud? Um, so theoretically you could roll your own, um, there, I mean, NX itself, itself is open source. Um, NX cloud is not open source, but if you wanted to write your own. Um, distributed caching mechanism. Um, you could that part is not too difficult. Um, the, the other piece that I forgot to show you is the, um, so distributed caching is great for your, um, for your average use case. Let me share my screen again. Um, So you, you could theoretically, um, I dunno, in, in a week or two of develop development time, write your own. Distributed caching just like find, find the API and the index source code and, and write it yourself. Um, but I don't know why you would need to, uh, frankly, um, because the pricing is not, not bad. Um, but the, the other feature that I wanted to show you that NX Cloud also offers is, um, distributed task execution here. Um, so distributed caching helps with the average case. Um, and so, you know, if, if you've, um, if you, you know, modify something that somebody is already, already done, then you just use their, their, um, their results. Um, but what if you modify something that's at the root of your, of your repo or something that's nobody has, has new calculator already for you. Um, then you'll need something like distributed task execution. And what this does is it, um, automatically helps parallelize all your tasks in the, the optimum way possible, um, because NX knows your dependency graph. Um, So typically your, any attempt to parallelize things in the CI looks like this, where you've got, if you've got, you've got three different runners, you have some idle time and then you run tests and some idle time on your builds and then some idle time afterwards, and then you land and then a bunch of idle time, so, you know, manually setting up which, which tasks run on which servers. Um, but what DTE does is it. You just tell it how many agents you want and, um, and then it automatically. Splits up tasks among those agents in the most efficient way possible because it knows, uh, which tasks need to be run before which other tests. So it says, I'm going to run those first on this agent. And as soon as that's done, I'm going to be adding in other tests on top of that. So it makes the most use, the best use possible of how many agents you have. And if you ever need to scale up, you just change a three to a 10 and it's instantly scaled up to 10 agents. Um, whereas if you did it manually, that's a lot of work to try to set that up. Um, and that, that's the, that's the feature that it would take you, I don't know, I don't know, three or four months of development time to. To get it to where it is now. And by that point we'd have, we'd have moved on to something else. Um, to add more features and stuff. So, um, and that's, that's a more killer feature than distributed caching is even. Because that, that helps you with your, with your worst case scenario. Something that, you know, would take two hours, we'll take a half hour. Whereas distributed caching basically it'll take your average of average of a half hour and make your makeup be an average of 15 minutes or something. All right. Other questions? You can unmute if you want to talk verbally. Um, or if you're zoned out, that's fine too. So NX itself is completely free. NX cloud is the only paid portion. Um, and, uh, yeah. Um, if you have, um, let me actually point out, um, some of the community stuff. Um, so if you would like to, um, chat with other people who are working on, uh, NX, or, or, you know, trying to use NX, um, if you go to the NX.dev, this link on the right over here is the Community Slack. Um, and you can follow people on Twitter. We also publish lots of videos on YouTube, um, or just, you know, interact on GitHub too. So this, this area over here has all of our community areas, uh, community things. Um, yeah, well, it's been great. Um, presenting with you. Um, thank you, um, for your attention. And, uh, hi to everyone who comes back and watches this later. Um, good, good work, uh, watching all the way to the end of the video. Thank you very much. Yeah, you're welcome. That was great. All right. Thanks everyone. Bye.

Isaac Mann
Isaac Mann
160 min
06 Apr, 2023

Comments

Sign in or register to post your comment.

Watch more workshops on topic

React at Scale with Nx
React Summit 2023React Summit 2023
145 min
React at Scale with Nx
Top Content
Featured WorkshopFree
Isaac Mann
Isaac Mann
We're going to be using Nx and some its plugins to accelerate the development of this app.
Some of the things you'll learn:- Generating a pristine Nx workspace- Generating frontend React apps and backend APIs inside your workspace, with pre-configured proxies- Creating shared libs for re-using code- Generating new routed components with all the routes pre-configured by Nx and ready to go- How to organize code in a monorepo- Easily move libs around your folder structure- Creating Storybook stories and e2e Cypress tests for your components
Table of contents: - Lab 1 - Generate an empty workspace- Lab 2 - Generate a React app- Lab 3 - Executors- Lab 3.1 - Migrations- Lab 4 - Generate a component lib- Lab 5 - Generate a utility lib- Lab 6 - Generate a route lib- Lab 7 - Add an Express API- Lab 8 - Displaying a full game in the routed game-detail component- Lab 9 - Generate a type lib that the API and frontend can share- Lab 10 - Generate Storybook stories for the shared ui component- Lab 11 - E2E test the shared component

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Levelling up Monorepos with npm Workspaces
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Levelling up Monorepos with npm Workspaces
Top Content
NPM workspaces help manage multiple nested packages within a single top-level package, improving since the release of NPM CLI 7.0. You can easily add dependencies to workspaces and handle duplications. Running scripts and orchestration in a monorepo is made easier with NPM workspaces. The npm pkg command is useful for setting and retrieving keys and values from package.json files. NPM workspaces offer benefits compared to Lerna and future plans include better workspace linking and adding missing features.
End the Pain: Rethinking CI for Large Monorepos
DevOps.js Conf 2024DevOps.js Conf 2024
25 min
End the Pain: Rethinking CI for Large Monorepos
Today's Talk discusses rethinking CI in monorepos, with a focus on leveraging the implicit graph of project dependencies to optimize build times and manage complexity. The use of NX Replay and NX Agents is highlighted as a way to enhance CI efficiency by caching previous computations and distributing tasks across multiple machines. Fine-grained distribution and flakiness detection are discussed as methods to improve distribution efficiency and ensure a clean setup. Enabling distribution with NX Agents simplifies the setup process, and NX Cloud offers dynamic scaling and cost reduction. Overall, the Talk explores strategies to improve the scalability and efficiency of CI pipelines in monorepos.
Federated Microfrontends at Scale
React Summit 2023React Summit 2023
31 min
Federated Microfrontends at Scale
Top Content
Watch video: Federated Microfrontends at Scale
This Talk discusses the transition from a PHP monolith to a federated micro-frontend setup at Personio. They implemented orchestration and federation using Next.js as a module host and router. The use of federated modules and the integration library allowed for a single runtime while building and deploying independently. The Talk also highlights the importance of early adopters and the challenges of building an internal open source system.
Scale Your React App without Micro-frontends
React Summit 2022React Summit 2022
21 min
Scale Your React App without Micro-frontends
This Talk discusses scaling a React app without micro-frontend and the challenges of a growing codebase. Annex is introduced as a tool for smart rebuilds and computation caching. The importance of libraries in organizing code and promoting clean architecture is emphasized. The use of caching, NxCloud, and incremental build for optimization is explored. Updating dependencies and utilizing profiling tools are suggested for further performance improvements. Splitting the app into libraries and the benefits of a build system like NX are highlighted.
The Age of Monorepos
JSNation 2022JSNation 2022
25 min
The Age of Monorepos
Today's Talk is about the world of monorepos, their history, benefits, and features. Monorepos address challenges in web development, such as slow build processes and unstable connections on mobile devices. Collocation in monorepos enables easy sharing of functions and components among projects. Speed and efficiency in monorepos are achieved through collocation, dependency graphs, and task orchestration. Monorepo tools like Learnr offer features such as caching and distributed task execution. Monorepos provide code sharing, consistent tooling, and automated migration, resulting in a 10x developer experience.
Remixing Your Stack in a Monorepo Workspace
Remix Conf Europe 2022Remix Conf Europe 2022
22 min
Remixing Your Stack in a Monorepo Workspace
Let's talk about remixing our stack in a Monorepo workspace, which allows for incremental migration and is suitable for transitioning from a Next.js app to a remix stack. Refactoring may be required for feature-specific and Next.js-coupled components, but the process is simplified because the features have already been moved out. Configuring the Monorepo to reference packages locally and linking them to the Next.js application is necessary. Nx provides benefits like fast refreshing, pre-configured setups, and features like local and remote caching.