Building a Sustainable Codebase with FP

Hi, folks. Today I'm here to talk about how you can build a sustainable architecture. First, I'm going to present myself. My name is Karolina Pascal Campos. I'm from Brazil. I'm a software engineer that works at Briza. And here you can follow me on my social networks. And I love books, coffee and running. So let's go to the presentation now.

So the main topics that I'm going to talk about today are functional programming, tests and hexagonal architecture. So let's start with functional programming. We know that functional programming is about pure functions, right? So given its inputs, we are always going to return the same output and we're not going to have side effects. So when we talk about side effects, it could be like saving to a database and an email, so anything like that. But it's important to have in mind that we will also need to deal with improved functions because we know that in order to have like a useful code base, we need to have improved functions. So our programs need to have input and output. We want to be able to interact with it. And those interactions always happen in perfunctions. So after we write your database, for example, we're going to change its state. So when we are going to read from the database, after saving to it, the result is going to be different. And how are we going to handle that? So we need to talk about side effects. When we are like functional programmings, we love to talk about the pure things. But we also need to talk about the effects. And that's a really important topic to be discussed, how to deal with them. So we need to be really careful about the impure functions because they change states. So if I run something today, tomorrow the results can be different. So this is the hard part to manage, but we need to manage that. So that's the reason we need to carefully structure our database to not mess with it. So here's an example. Imagine that you have an impure function inside a pure run. So you no longer have a pure function.

This is why we need to be careful about structuring our code. We want to maximize the pure functions we have. So we cannot call an impure function inside a pure run because we have just lost it. So we need to separate what we built. We need the effects from the pure functions. The effects can infect everything. And our goal here is to maximize our functional programming and to minimize and isolate the impure functions, the imperative code. So what we need to have in mind always is that if a function could be pure and is not, we are doing it wrong. So let's try to refactor this.

Another powerful thing about functional programming is when we talk about immutability versus mutability. Let's start with the image that we have on the left. We have a book with many pages. This book is the identity. When you want to change something in the draw, you just turn the page and draw a new thing in the end. So the act of turning pages represents the state of the draw during time. So if you stop on a page, you can observe the state in a certain point of time.

But if we look now to the right, we have a mutable example where the book is only one page. So if you want to make a change, we need to erase stuff and redraw things. So we have just lost the notion of time here. And state and identity are now blended. So when we talk about immutability here, we have the animation. You know all the steps your draw has been through to get to the end. You know the whole story.

So when talking about pure functions plus immutability, we are going to have functions that are easy to reason and easier to debug. This is what we want. Because with pure functions, you know that given an input, the output is always the same. So it's really easy to reason about it. And also with immutability, you always have new pages when you change something. So you don't need to go deep in the code to understand which function is mutating the page. So we want to be mindful about the things we want, that's pure functions and immutability when developing, so we can seek them. Because this combo is going to give developers a lot of power and speed. Nice.

So when programming is functional, we want to focus on data, calculations, functions and not classes, interfaces and boilerplate code. So the intent of our code is clear, and you can try new things faster. So what we want here is to be as functional as we can to be able to move faster and with confidence. Now let's talk a little bit about tests, because if you want a sustainable codebase, we need tests. So tests are important because they give developers confidence to change or to extend or change our code. So when trying to understand for example a code we didn't write, we can check the test to understand what was expected from that code when it was written. So it's also like documentation.

If we change code, we need to change tests because otherwise it would get stale, but they don't, because it's a live documentation. But how do we know and check if our program is doing what it's expected to do? How can we check the quality of this feedback? And here I'm not going to talk about 100% coverage.

So let's start with accuracy. If your tests fail, can you determine exactly which portion of the code has failed? How long do you take to know where it failed? So what we want here is functions that are really granular, so as granular as it is, the more accuracy you can get. So it's way faster if you have small functions with no dependencies from the outside world, so that remember pure functions.

Also about speed. The faster we find a test failing, the sooner we can identify the problem and the smaller their impact will be. So what we need here is tests that are simple to write and to maintain, and also that are fast to run. If we have a lot of complex code with a lot of dependencies from the outside world, it's going to take a lot for us to maintain these tests. And also we need tests that are reliable. We need to trust the outcome of our tests. If the outcome changes from one execution to another, if we don't change the code, the configs or the dependencies, why do we keep these kind of tests? Because we don't trust them, so it makes no sense to keep them.

Here we are usually talking about end-to-end tests. End-to-end tests are a problem because you don't walk anything, you really save stuff to the database. It's really common for them to fail for obscure reasons, like latency, garbage collection, asynchronous operation. So we need to think in a different way. Here we need to think a little bit larger. We know that unit tests cover the function levels, integration tests cover the flows, and we need to guarantee that the conversation between services is working. So usually we use end-to-end for that, but we can just think a little bit larger and test the conversation between two services at a time, for example. So here we use contract tests.

So every contract test is going to have a provider service and a consumer service, and for each communication between them, we need to define a clear interface called a contract. So now we have like automated tests that are sure that the data generated by the provider can be consumed by the consumer. And if we change anything, the contract is going to break. So it's important here to have in mind, this is not a one-to-one replacement with end-to-end tests, but most of the errors that end-to-end do catch, those tests can also catch them. So with that, we can have accuracy, speed, and reliability. And that's what we want.

So now, I'm going to talk about how to design a record. Because okay, we have the functional programming, we have the test, but how to design the code to leverage this? What we want is high maintainability with low tech debt, because we want the application that are used to give maintenance and have low tech debt. We need them to be simple and easy to work on. And this is hard because maintainability is a long term concept because in the beginning, it's really easy.

Your project has just started, we have a few lines of code, three people working on it. So it's just a couple of dependencies. But as soon as your team and your code grows, things get harder. And the bug can result in a whole architecture refactoring. So that's the reason you need to think about maintainability since the beginning.

What we are going to say here. So the first attempt we had was try to use hexagon architecture. So here we have just a simple hexagon architecture. The domain model where we have the implementation of our domain. So here we're going to have the pure functions and the mutable data. That's why it's green. After that, we have like the second layer that there's where we have the controller that's responsible for connecting the ports with the adopters in the logic, the domain. So here is all about coordination. But here we already have some functions that are impure, right. And some mutable data. So it's not all of them, but some these whites, yellow. And in the third layer, here is where we have is called the IO or ports. And here we interact is external words. So here we're going to have called for other services, we're going to have calls to the database to third party APIs. So here it's okay to have impure function and data mutation we expect that in the IO layer.

So what we want to take second hexagon architecture is to be able to separate the business logic from the IO. So we want to structure the code in a way that we protect the domain from the technical stuff. So we can define what are the business rules, what are the educational rules and what's the infracode and how do you do it testing this architecture? So we want to test in isolation with mocking steps because we don't want to rely on external database or external APIs. We really want to mock and step things for that. We use dependency injection. Right? So these dependencies here are going to be passed as arguments. So when we run in production, we use the real database and for test, for example, we can use a hard coded file, but what was the first discovery with this architecture? So remember that in hexagon architecture, we have our functions in the boundaries, right? In the IO layer, the ones that generate side effects. So the HTTP request, the emails, the message, everything is there. So it's nice to realize that in this architecture, we can isolate and control the side effects in this layer. So we have the boundaries here, as we can see.

So let's talk about this concept. What does isolation mean? So isolation is when all the info a function has from the outside world is passed through arguments. So we can come to the conclusion that every peer function is isolated. So a peer function is a subset of isolated functions. Not all isolated functions are pure, but all the peer runs are isolated. And isolation is the dual of encapsulation. So encapsulation means that the object has a state and the outside world knows nothing about it unless it's made explicitly available to it through getters and setters. So isolation means a function knows nothing about the external world unless it's inputted on it. So it's the dual, right. And isolated functions are really easy to test and given for free when using functional programming. But in object-oriented programming, they are a problem because of the encapsulation principle. So when developing with object-oriented code that's going to be tested, we need to find intersection between isolation and encapsulation. And that's really hard because it's way easier to work in a subset than an intersection, right? So it's hard to maintain the balance between isolation and encapsulation when dealing with object-oriented programming. But with functional programming, it's no problem at all. The minimum expected from a good functional design is to have pure functions. And they are a subset of isolated ones. So if you're doing pure, you are doing it isolated, and you get the easiness of the test for free. And we want to have as many pure functions as we can to get all of those benefits, because we know that if you have pure functions that cause an impure run, it would be impure also. So why is hexagonal architecture in FPP natural? So one of the languages that helped to understand that is Haskell, because Haskell is a functional language that expects all functions to be pure. And so when we are imputing Haskell, the compiler helps you in that function, checking if that function is pure or not. Because if you wrote an impure function and forgot to declare its type as IO, it's going to throw you an error. So in order to have as many pure functions as you can, you push the impure function to the boundaries. So the impure functions can call the chain of pure functions, but a pure function never can call an impure, otherwise you get a compiler error. So we are going to have a shell of impure codes. So this is a reminder of hexagonal architecture. So this is why Haskell is a language that helps you discover that because it expects all functions should be pure. And so good functional programming design can be an eSports and adopters. And this is a great discovery. But remember that at the beginning I said that we want to leverage functional programming to its maximum. And as I showed you, we still have controllers with imperative code and side effects.

So why don't we want that? Here I'm going to show you an example. Imagine that I have this functional call that's called block. And I'm passing a card, a database, and a producer. So here, when I look at this functional call, I have no idea what's the output. It could be true. And if it's true, what does that mean? OK, let's enter the function trying to understand that. So as you can see, I have the card. I'm doing some logic on it. And then I'm calling the update in the database. And after that, I'm calling the producer card status changes here, right?

So here, I needed to enter the function to understand what it's doing. And I'm not returning anything here. So I have no idea what happens before I enter in the function. It's also harder to compose when I have imperative code. So here, imagine I'm calling block for 10 cards. Imagine that at the third card it failed. So now I have called the debate to update the producer to send messages, but only for three cards. So I cannot just rerun stuff because I would duplicate effects. How would I handle with this composition? So it's hard because I'm just doing the things on the fly. And what we have here is not the permit that we want. We have a permit that we have more integration than unit test because as you can see in my controller, I have those kinds of functions that are calling the debate, that are calling the producer. So this not what we want. Those functions are not pure. The tests are not that easy, and also the maniability is not going to be good. So let's try another approach for that.

What about if you use hexagonal architecture with dependency rejection? So what does that mean? That we enter the couple, the decisions, from the facts. We decouple the intent from the execution. So the same function, but now when I call block, I only pass the card. I'm not passing the database or the producer. And as you can see, I have a return. And my return is an object that has a key called the facts data, and it has its value and a key called the facts message, and it has its value.

So now I know what the function does. But sure, I can enter to better understand. And as you can see, I'm just calling the logic here that's a pure function, and I'm just creating. What do I want to do? What's my decision? And I want to update the card, and I have the data that I need for that. And also, I want to send a message. I have the data that I need for that, but I'm not doing anything. This function is pure. So I can test using unit test, for example. It is really to reason about. Also, it's going to be easy to compose, because before we were iterating and saving things to the database and producing messages. But now, I'm not doing that. I'm just creating an object.