Building a modular monolith with Fastify

So, happy to be to be here at Node Congress. I have been... Look, I've known about this conference for a while, and it's so great to finally be here in Berlin. It's been a long time since I came to Berlin, and I kind of missed it. So, look, I am here talking about Fastify. So, it's great to be here, and so on.

So, a couple of things about me. I have a newsletter too, and in the UK, if you don't matter, just subscribe there, or follow me on Twitter, at Matteo Collina. And I also am also streaming on Twitch, so if you like that, you can join. Also, I need to correct Liz, because since I put in the bio, it's actually 17 billion in 2022. Whatever. Like, it's so much fun.

So, okay. So, today, we are talking about Fastify. And if you don't know Fastify, you probably should, but you should probably use it if you're using Node.js, right? This is not Congress, so I don't know. You should probably use Fastify to build your web applications and backends and APIs. How do you use it? Very easy, you require import whatever. It has a logger built in. EpiDays, you want a logger in your application, right? It's called Pino. And you can use it to create your routes using AsyncAwait and all the other shenanigans that you might want. It's probably 4 billion million times per year or something, downloads per month.

So, okay. Let's talk a little bit about why you should not use Fastify or anything, or how to not use Node.js. How many of you have built a model view controller system in your career? Great. What's the problem with this system is that ultimately it doesn't scale. In what sense it doesn't scale? Well, it doesn't scale well in complexity. If you are building an application, and you're building it using following model view controllers, okay, let's start with the view. Most of the time there's no view layer anymore, we just do build API, so that's gone. So if you are writing a piece of code, it either goes into the controller or the models, and if it, you know, a 50% chance. After a bit of time you have 2000 models and a routes.js file that is 10,000 lines long.

Truth. This is actual number from real code. So it doesn't really work well in that way, and it doesn't scale well in complexity. What do you do instead? Because the question is, if model view controller is not good enough, what do you do instead? And how a lot of few major companies and a few major system can actually build monoliths, because model view controller is not good but they still ship monoliths. So there's a few article every now and then about monoliths around, and why and how, what they are doing.

Well, to be honest MVC leads to Carbonara and spaghetti code, but I like the Carbonara and not the spaghetti code. That's it. So how do you do it? You need to structure your application by domains, by feature. You need to identify what are the key features of your application and segment it. And in between those systems, you tend to communicate over well-known APIs. The important part is, do not have one set of features, read the database of the other set of features. Because the moment you start dipping into both, then you have a lot of, you don't know how you end up with your relations. Or another term to say is if in order to fetch a data, you need to do a 10, you need to join 10 tables, you probably have the wrong schema. Okay? So think about that and what you're doing and reflect on your life choices.

So what makes a domain and how do we build monolith? Like it is something that we want to do, right? It's a good thing. And well, you know, it typically, it's a specific subject that is being developed, okay? Typical cases is the catalog or the order management system or a cart or whatever you want to call it, okay? In some of the stuff that they're recently been developing, it's called, we have a domain which is the organization and the teams and other stuff is about the applications and the code that you run on our cloud. So anyway, it's domains, and you want to avoid the spaghetti code. So what does Fastify as for helping out building with this stuff? Fastify offers a concept called encapsulation. So it enables you to structure your application in a set of plugins that you can use to segment your data, to segment your code and in that way they don't share anything, essentially. Now a few few may ask but you could do this before, right? There's other techniques. Yes, but this has no cost has no overhead whatsoever. Okay, it's super fast, so you don't have anything to pay for it. And it's great. It has also concept of this called the decorator. So you can actually add stuff to your to your code and application. So how does encapsulation work? You basically can create context within context within context within this context, like kind of from a big matrioshka, and each layer does not share anything with the previous one. It's great. It's something disappeared. Okay, fun. Some missing.

I don't know. Okay, fun. There was an example there. I don't know. Okay, it arrived. I have no clue. I've not done nothing. Great. What happened is that... What you can see here, we can use this register call to create these encapsulation context, and we can decorate our request or response objects with data and information, and we have a concept of hooks.

Here we have hooks, and we have the onRequest hook where we set the data, and these will lead... Given the two routes, you will have different value. You see that one and two. And those will have the same different values that have been populated based on the concept... Based on the metadata, on which the route is being executed. It's great, because in this way, we could actually create our stuff better. So how do you... But you know, every encapsulation system, there's a way to break encapsulation. So how do you break encapsulation? Well, there is an utility called Fastify plugin that lets you do that.

The key concept about it is you want to provide, to create, to structure your application as little bits of sub-apps. So that each one of them contains their own domain and their own sub-domain. And in that way, you can scale your JavaScript development to different parts. And each one of them has their own plugins, their own routes, and all the structure of their code. How do you break the encapsulation term? You use these nice, you can use a skip override or the Fastify plugin again. We are missing some slides, so I am, okay, it arrived. Also, you can use plugins to do metaprogramming. When you register a plugin or a route, you could automatically start creating new behavior. Automatically customize the way the system works. Let's skip, I'm a little bit late. How do you structure your normal JavaScript application? You use plugins and routes to structure your thing.

These are typical structures for a monolith, right? This is not modular at all, it's just a monolith. Okay, fine. This is not the slide that I was supposed to. Okay, now it's green. Okay, so how do we build a modular monolith? First of all, it's one, and it arrived. You need to identify the domain that you want to do, and then two... Okay, you need to structure your coding folders so that you can lay out your domains one at a time. And three, it's the same. That's going to be fun when I do the live coding. You jinx me, Paolo. This is totally you. So three, you can use plugins and decorators to share code between domains.

And we are going to do a quick demo now because in fact, it's demo time. And this was supposed to be a nice picture of Rome, and the demo gods should be with me. But, apparently, okay, this is starting super well. So let's see how it goes.

Okay, so here we go. So we have this nice folder. And in this nice folder we have a few things. So these are typical Fastify application. I have my server.js that starts the things, and we configure my, we have a build method to build the app. And then I can close my, listen to something to port and host. And then close the app, cool. And in my app, what I do? Well, in my app I load Fastify. I use an utility called Fastify autoload to automatically load all my plugins, or my routes. And then I return the app.

What are my plugins? Well, I have a few different plugins. I have one for managing the error, and one for doing the handling a not found. And look, you want a not found and an error management, right? So, okay, and we have something called the inventory. What is an inventory? Well, the inventory is kind of this object that I use to store data, okay? At some point, you need an entity, somebody that you want to get some data from the database, okay? And you can use whatever you want. You can use Prisma. You can use Next. You can use just the database driver. But at some point, you will need to have some utility that reads data from the database from you. So that's what this is about. And it completely faked it, okay? It just returned true if this SQ is for tissue, okay?

So what are our routes? We have two routes. One is the product. So we want a route for products, and that returns. And then we also query to get our inventory, okay? And also, we have an inventory which has just returned the number of items that are in store for that SQ. Why we are doing this API structure, who knows, okay? But that's not the point. So, okay, we have this structure. Let's run this and let's see how it goes.

So we do Node server, and then we could do... let's do that, okay? We can do curl. And we can go and query for the catalog of the products, okay? And you can see the result. And then you can go inventory, and product 42, and you get the data. And then if it's 43, you don't get anything, okay? Note that you see that there is a prefix, and the prefix is the folder. So Fastify autoload really gives you that nice Fastify file system-based routing to structure your thing.

So this structure is pretty good, okay? This is a good monolith, essentially, what you would call MVC architecture, very close. But it's something that can take you very far. How do we migrate this to be a modular monolith? And how can we have multiple teams communicating with each other, and sharing code in a better way? Okay, let's start with doing a little bit of refactoring to this.

So first of all, let's take our magical routes folder, and let's call it modules, because we like it, right? So it's not route, it's modules. So, and now we are becoming a modular architecture, right? That's how you do it, okay? So, and you have still have your catalog and your inventory, but now we create sub folders for routes, and you move the stuff in there. Okay, and then inventory. And you move the other routes. Cool, we have, maybe I've made my move at this stuff, and why do we do this? Well, it will be clear in a second. So we have done this, these two migration, okay? How do we separate them? So the key interaction point between the catalog and the inventory is this inventory.js file, which is a plugin. So we need to do something about it because this truly, it belongs to the inventory subsystem. So what we do is we open it up and we create, here we create an index.js file. And in this file, literally, I'm just cheating badly. I'm just putting stuff in here, okay, copying these up. Well, we won't need this anymore. I'll show you in a second why. So we have this and great. Now we need a few more things. So we want to load our routes. So we go and copy these two things. We get the autoload and this utility called DSM that I wrote. Don't ask why because it's a bad answer. So it's you don't want to join routes. So here you take you got that and oh you just copy this one. Okay, so here you go into inventory and after you do that you have set up the autoload thing and to load our routes.

Okay, so this is done. Okay, now what we need to do is we need to essentially do the same thing for our catalog. And here we go and we don't need the inventory though. Okay, we will use the inventory of the upper layer. Cool, we need one more thing though. Okay, we want to set the prefix. So we need the options and here we need options and prefix, ops prefix. Cool. Why we'll do that? Because our autoload magic will need, will set up our prefix and we just need to pass that through. So we do that and we have, what is it? Okay, great! So we have created, we have set up our stuff. Now here we can delete this inventory file. And in here, oh, well we don't need, this is not routes anymore. It's modules but we just need to change something. We don't want to, we want to break encapsulation. And we just want to load things at one level. So we have refactored everything. And now we can restart this app and it's crashing. Good! And here you go and it's up instead of fastify. Great! Okay, and it again is the same thing on the other file. Look here, I'm just changing this fastify. Cool! So here you go and it's... Come on, I keep forgetting stuff. Okay, here you go. Great! So now we can query it and you can see that this still works the same as it was working before. Okay, and the demo gods have been mostly with me. So let's be very happy about this. So we have been doing this nice refactoring and setting things up and it seems good, right? However, can we do something better? Can we improve our things? Well, we can. I have been for a while working on a new company called Platformattic. Here comes the pitch, so be careful. So what we can do here now is that we can just create a new Platformattic.service.json config file and we could literally open up a server.

Let's see if Copilot is happy enough. So platformattic.dev, schemas, well, yeah, not exactly what it should be. Schemas, and here we have V20 and here we go. And this is DB, okay, cool. And now it's complaining because we are missing some stuff, so we want to add the server property, and we want to listen to the port, and which go to 1000, we want the hostname, and we want... Okay, cool, we need to add a few more things, so one thing we want to add is plugins, and we want to tell it to load stuff from some path, which is a path, and we need to load plugins first, so we just do load plugins, cool, and we just don't want to encapsulate this, and this is false, cool. Now the next one is we want to load our modules, and we want the modules, we don't want to encapsulate, and we want to just go one one level depth, with one level deep, and now I can just delete this file very happily, delete this other file very happily, so way less files to maintain, and now I can just skill this NPX-PLT service start, and now everything is still working as it was before. So, and now we have simplified a lot of things. Bonus point of this approach is that I could just do, for example, service, add more config, and just turn on OpenAPI. True, okay. And what mistake did I make? Is not allowed... Oh, it's DB. Service. Sure, he's right. Okay, so here it just restarts. Okay, it was restarted automatically. Cool stuff. So it restarts automatically as you've seen. So what we could do now is go in here and look at lost 3000 documentation and you will see all our OpenAPI docs automatically generated and set up correctly. It also supports automatic metrics, automatic permissions integration and a lot of other things that you want in your production Node.js application. There's another component called PlatformaticDB that automatically generates your APIs from a database, but I'm not talking about this in this presentation. What I wanted to say at the end is, building a modular monolith is all about creating and separating concerns and at some point you might just want to switch to microservices. But if you've done that, you see all the code that I've done is just a config file a way to start with so you could easily migrate to microservices now and you have almost a share-nothing architecture. So I just wanted to say at the end that we also just launched our cloud. So if you want to deploy your Fastify or Node.js application it's out there and it's free. So thank you. Are the startups snapshots taken after function optimization? I don't think this question was for me. Yeah I don't think so. But if you want I can answer that I can try answering that question. I probably have the answer.

I don't think this is for you. Okay no. Wait. This seems to be something not working. It's something not working with the Q&A. Do you have any questions folks? Yeah just raise your hand and then we just go. Old school. Ooh. Yeah they're just... Okay there is a question over there.

What's your opinion on dependency injection based on ShareCode? ShareCode, Fastify, what are they? Yeah, so what I've just shown is a very simplistic way of dependency injection. Essentially. The question was, what do you think is what you showed. Dependency injection mechanism based on sharing code across Fastify plugin. If I understand the question correctly, right? Perfect. It is exactly that. It's a very crude way of creating a context and attaching things to that context. And to be honest, it is a dependency injection mechanism. It's mostly a very dry, don't repeat yourself type of system. And it's not even config based. You just create your plugins and you can even specify with Fastify autoload, you can even specify a little bit what other values need to be decorated and everything will load correctly. So, yes, it's a very simple dependency injection mechanism. And from my point of view, it's a dependency injection mechanism that just works, essentially.

Okay. Thank you. I have one question here. Oh, it worked! High five! So, how can we debug that our Fastify microservice is working at its best? Yes, this is a phenomenal question. Okay, the first thing that you want to do, you want to create... You want to make sure that you have specified schemas for both the data coming in and the data going out. In that way, you're both protected and you're also rendering your JSON as fast as possible.

It's great. Then, the second bit is you want to use a module. It's called under-pressure, and under-pressure protects your service from being overloaded. So, you probably have heard about event-loop trashing, and it automatically protects you from that and essentially ensures that if your service receives too much load, it starts returning a nice 503 and, at least, is not crashing or going into a situation that's unrecoverable. Oh, by the way, that module is also part of platformatic, so you get it for free, if you use our stuff. Cool.

And the JSON file was very specific. How would I know what to put there if I am not Matteo? Oh, there is auto-completion if you didn't notice. It's actually very straightforward to put in there, it's JSON schema-based, and the JSON schema provides a guide. Finally, there, yay. That's my next question. It's great, and you can put it there, it's fully documented, but also there is auto-completion. So if there is auto-completion, it's great.

So, do you hate TS? Nope, nope, that's the answer, nope. Nope, I don't. In fact, the platformatics support full TypeScript compilation, so you can just write your plug-ins in TypeScript and everything will work fine as expected. So yes, no, I don't hate TypeScript. What happens to the plug-ins when the application is converted to microservices? Oh, you know, microservices is just a plug-in, right? So, that's what it is.

And when modularizing by domain, where do you place code that doesn't fall under a particular domain category? Some helper to be reused across API endpoints. So, okay, that is two bits. So, when you're building this kind of system, you have two levels of reuse. One level of reuse is typically at the code level. And when working larger teams, especially on a modular monolith, you tend to use a mono-repo. So each one of your modules, as I showed, instead of being structured, instead of just being a single folder, it also has a nice package JSON attached to it, right? So you could very practically have a utility library that you are actually importing and requiring in there. So if you just need code to reuse the utility, you can just, you know, share code. And that's it. If you want to set the key part is keep the domain separate, mostly keep the database separate. Keep the data of each domain contained so that it's not shared between boundaries. Cool. Thoughts on separating the transport HTTP GraphQL PubSub from business logic to allow business logic reuse between them.

OK. I probably I missed a few. Yes, it's like that. OK, so, yes, my point here is it would be nice to do so if you could separate them. And the harsh reality of software development implies that it's actually very hard to actually separate the business logic from the transport. And you can, but it requires a gargantuan effort. So of course, if you are able to, but do it. But at some point, you're moving it from being an HTTP server that expose some API to being a queue exposing some API, OK? And what it's important is not just the code, but also the way the code is invoked. And I tend to put the stress on things on people, not on software. And when you define an API, an API is not code. An API, it's a contract between people. So we, developer of the back end offer you front end developers some access to data and business logic. And the API and the interface of that API is actually the thing that matters the most. Because it's what, it's how we can speak and communicate with a common language.