Forget Bad Code, Focus on the System

We hope you enjoy the rest of the talk. Thank you, and I'll see you next time. Bye-bye.

So I'm going to try to convince you today that you're focusing on the wrong thing. You're talking too much about good or bad code, and what you should actually be focused on is how you set up your entire system, your architecture.

Now, has anyone seen this slide from NextJS Conf that kind of made the rounds on Twitter? This is a server action, and what's going on is that you have a React component with SQL right there in the React component, which is a bit weird. And earlier today, David from Redwood also announced that they can now do the same thing, where you have SQL stuff mixed together with JSX stuff. Now how does this make you feel when you see this? How does it feel? Does anyone feel angry? Yes, all right. Does anyone feel upset, like kind of weird, like this is a weird, bad thing to do? All right. Now, what if I told you that's exactly how we felt about JSX 10 years ago. React launched, they came up with JSX, and all of the greybeards on the internet were like, oh my God, you're mixing HTML and JavaScript? What the hell is wrong with you? Why would you ever do that? It's separation of concerns, all of that. And then CSS, CSS in JS came along and the internet went wild again. Why are you mixing all of this together? And what if I told you that the whole point of all of this is, in fact, separation of concerns.

Now, it feels, you know, I say separation of concerns, and you might be thinking, wait, but you're mixing all of the concerns together. You have styling mixed in with database logic, you have fetching mixed in with all of the ‑‑ can I actually see my notes? All right. So you're mixing everything together. That doesn't feel like separation of concerns. But there's a different ‑‑ there's another way of thinking about the separation of concerns. What if you separated your concerns by business domain, not by the technology that you're using? The word business is doing a lot of work here. We like to ‑‑ I don't know, who here has worked with like fancy pants architects and people who have like really high sounding titles like principal engineer and stuff like that? Just give a scream. Okay. A few of you. Okay. Who here has heard them talk about domain modeling or business this, business that, that sort of thing? All right. Perfect. So this is a lot of work and ‑‑ oh, damn, I'm missing ‑‑ there was supposed to be another slide there. Okay. So the word business domain is doing a lot of work there. What the business ‑‑ what I mean by that is a good example of that is Lego. So these are two Lego sets.

They both happen to be my first big Lego sets. So on the right you have what large Lego sets used to look like in the 90s. Here's the Lego super car. It comes in a really big box. I loved it. As a kid this was like oh, my God, you get 1,300 pieces and the first thing you do, you open it up and you sort every little piece into its own little box. It takes a while. And then eventually you can start building your Lego and you have ‑‑ you can find all the pieces. If you need a gear, it's in the gear box. If you need a wheel, it's in the wheel box.

Now, later, when I realized that, hey, I'm now an adult, I can just buy Lego whenever I want. I know, like crazy idea. I bought the Saturn 5, which was my second Lego set. That one has 1,969 pieces, which I think is carefully calibrated to coincide with when Apollo 1 landed on the moon and all of that, because it went up in the Saturn 5. That one didn't come with a box that you sort Legos in. It came in a bunch of baggies. You build the Saturn 5 rocket by going stage by stage. For those of you who don't know, rockets are multi-stage. It launches. The first stage burns up its fuel. It drops the first stage, and the rest of the rocket keeps going. That's how you build the Saturn 5. In each baggie, you get exactly the pieces for one stage of the rocket. You build it up from the ground, and all the pieces are mixed in. You have pieces that are from the outside panelling. You have pieces that go for the inside structure and all of that. It's all jumbled together. What's really cool is that it's much easier and much more fun to build than the supercar was because you never have to search through more than like 200 or 300 pieces, which is much easier to do. You can spread them out on a table and you're like, oh yeah, that one, that one, that one, and you just build it and you're enjoying the building process rather than spending a lot of time searching for the right LEGO. Now I know some people say that's the fun part.

I personally hate that part. And this concept of separating by business concerns goes even further than just LEGOs. This is, there's a new, actually I don't, I'm not in construction. I don't know how long this trend has been around, but they have this thing called modular construction where they prefab entire pieces of a building and then just hoist them up and stack them on top of each other like LEGOs and you get a building. It's much cheaper, it's much faster, and what kind of blows my mind, if you see on the bottom there, that is a prefab bathroom. You get an entire bathroom with all the plumbing, with all the electricals, with the tiles inside, with the bathtub, with the sink. Everything is in there and you just plop it on your house and connect, I guess, a sewer pipe, an electrical outlet, and a source of water and you get a full bathroom. The idea is that it's fast. They have prefab kitchens, etc, etc.

You get units contained and they fulfill a function. Another more close to home, if any of you have been around for a while, this is a famous slide from, I think, 2017 by Cristiano Rostelli, I don't know if I pronounced that right, where he talks about how the old separation of concerns was JS, CSS, HTML by layers. The new separation of concerns is button, modal, list, etc., and you have a component that is self-contained and fulfills the function and knows everything it needs to do.

So, the reason people do this or the reason people are excited about this modularity is architectural complexity. Now, architectural complexity sounds like a super-fancy term but I promise you it's pretty easy to understand. The idea is that if you use self-contained units that aren't just jumbles of bits that you have to artisanally construct every single time you're building something, you are going to have an easier time dealing with your code base, lock shipping features, etc.

There was a, these are stats from a 2013 study, it was actually a PhD thesis by Daniel Sturtenrand. He did this study at MIT in 2013, it shipped in 2013, I don't know how long he was working on it, but they were looking at an industry piece of software that's actually used by real people, developed in industry, so it's not like a grad student's project, it's not an open source project, it's actual code that makes money. They didn't reveal who or what the company was, so I don't know. They tracked the software through eight major releases, so a very long-running study. It went for a couple of years, and what they found is that high... Okay, I have to be careful where I put my head. High architectural complexity leads to a 50% drop in productivity, a three-time increase in defect density, that's bugs, and a 10x increase in staff turnover.

That last one is a little iffy if you ask me, because it's like... It turns out that the people who work on more architecturally complex code are also more experienced, people who are always more likely to leave because they've already been in the company for more years, and they have more opportunities elsewhere. Sometimes people just get bored. You don't want to work on the same thing for ten years. But the 50% drop in productivity and the three-time increase in defect density was a very solid The way they studied this was... They compared people who work primarily on... Actually, no. Everyone was a self-control. So they just looked at people, and they looked at how much time while working on a feature you spent in files with high architectural complexity and how much time you spent in files with low architectural complexity. And I'll explain how to actually measure that in a bit. And they saw that the more time you spent working on more complex files, the less productive you were, and the more likely you were to make a bug. Which in normal people's speak means if you work in code that's hard to understand, you're gonna make more mistakes and it's gonna take you longer. Which sometimes it's like... Why are these researchers trying to research such obvious stuff? So my latest obsession has been finding actual research that confirms my preconceived notions and things that I've noticed in reality. Because then I can be like, this isn't just a hunch, there's research. Which works really well in online debates. But let's move on.

So how do you measure architectural complexity? Well, first you got to start thinking about your code as a dependency graph. So let's say this is a bunch of code. Each of these shapes represents an element or a thing in your code. It can be a function, a class, a file, a constant, whatever. Just elements of your code that have different complexity, different sizes. And most of them are about average.

If you look at how these elements of your code call each other, you can draw lines between them. So this is a dependency graph. Imports are a good example of a dependency graph. But also when you call a function, you are depending on that function to do the right thing for what you're trying to build.

So in this example, you see that the lines, there aren't many of them. There's what? Five, six lines? Five lines. And it already looks like a mess. That's because the lines are going across the code base. So I have the triangle on the left going all the way to the triangle on the right to make a function call or whatever it's doing. And that increases your architectural complexity. It's going to be really hard to understand how this code works when you just look at a file, because it's all jumbled together. And I tried doing it with more lines, but then it really doesn't make any sense.

And what if you took the elements that are actually related, that are calling each other, and group them together? And then you can redraw the lines, and you see that you have a couple of areas that have a lot of internal connections and that are very loosely connected to the outside. You can call that a module. So if you look, that's how you can identify natural modules in the code that you already have. So here I increased the number of connections. And you can see that circled in blue are modules that have tight internal connections, tight internal coupling, but are loosely connected to the outside.

And what's great about this is that you can start thinking of those modules as a higher level component, as a higher level element. And one of the things that Sturtevant or Sturtevant found in his study is that the more experienced engineers actually already think about code this way. They don't even read the individual elements. They just see, oh, this is a four loop going across this thing, and they don't really read the body, they just understand the higher level component or the higher level code element that does the thing. And I forget what I was going to do. But the point is, you have modules, and you can identify them by looking at your dependency graph. So, what the hell am I talking about, right? Like I'm just blabbing out here.

So, think about a car engine. This is a W16 engine from, I think the Bugatti Chiron. This thing is fucking amazing because it produces a bazillion horsepowers. It has 3,500 parts. And if you look at that picture, you have no idea what's going on. At least I don't. It's all jumbled together. It's tightly coupled in a very, very small space. They have a lot of things that they have to pack into a small space. But if you look at it from a higher level, you realize that the API for a car engine is actually pretty simple. Air and fuel go in on one side, and exhaust and torque come out on the other side. So, if you think of an engine as a module in your code, you can kind of just plop, pop it in, put it in, connect the API points because they are clearly defined, and you get a working thing that can help you achieve a task.

Now, I know, this is a far-fetched, far-fetched, I forget the word. But anyway, they have actually found these structures in large code projects as well. So, this is from a 2008 study by McCormack, Rasnack, and Baldwin who they study architectural complexity and how real people organize software and what leads to better outcomes in the software design, blah, blah, blah. But this is a network, it's called DSM, which stands for Something-something diagram, but basically the way you read this, each axis represents files and the dot between, so each axis of the matrix, each side of the matrix is a file, each dot inside the matrix represents a dependency between those two files. The reason you get that diagonal line down the middle is that every file depends on itself or they also call this visibility matrix, where the idea is which files can see into other files or into other functions.

Now this is when Mozilla some time before this study did a re-factor and you can see that before the re-factor, the code was very interdependent, there was a lot of architectural complexity and then when the engineers went in and they cleaned up the code, you see a lot fewer dependencies because they were focusing on the architectural complexity, not on the actual bad code inside each module because that turns out to be more important. And you can also see this in smaller example, in smaller codes or smaller examples. This is actually a dependency graph I pulled from our code base just last week, where we built a new feature and the way we like to build stuff is to just throw things at the wall and then later Swiss comes in and cleans it up and makes everything nice and organized. And on the top, you can see, very confusing code, there's a circle of dependency in the middle that's marked by the red circles and like everything is kind of jumbled together. And then, and you can also see that we have components folders and hooks folders and like a very child component is going three levels up to a parent component to get to a parent file to get a hook, et cetera. And then after, I just organize things by business domain. You get nicely named folders that tell you, hey, you know, build summary, inside it is an index and other components and it has a hook and it's kind of independent. You can take that folder and wherever you plop it in your code base, it's going to just work because it can take care of itself. So, look, the point of what I'm trying to tell you is that you should contain your bad code, put it in a little box and then give the box a nice interface so that other people can use your bad code without worrying about how the code itself actually works and without going to read it. This is also called an abstraction. And the main thing you should make sure when you're building abstractions is to make sure they don't leak. If you remember the slides that I showed you from the very beginning, that's my thesis why React server components and server actions are not quite there yet is that the abstraction is a little leaky but I really like the direction we're going with that where you could have self-contained components that do everything in one file.

And real quick, the easiest way to notice architectural complexity is if you use a tool that draws a dependency graph, you can see that it's all jumbled together. If you have dot, dot, dot slash, dot, dot slash, dot, dot slash imports, that just screams architectural complexity. If you have circular dependencies, something has gone very wrong. And usually if you can't tell where which code works together and which code is a public interface, that is also a bad sign. Most often I've seen that happen when people do, when people export stuff so that they can have unit test and shit. I have one more slide. So the way you fix it is you organize things by business domain, put it in a folder, make sure that anything that is tightly coupled lives together. And then one big tip that I have is to clean and validate your inputs on the boundaries of a module. And then just prop drill the data down, and that actually works really well. And you know, if you ever are in doubt between copy-pasting some code that looks similar and separating concerns, always go for separating concerns. Even if it looks similar but it talks about a different concept, it is different code and I promise you that putting it together and making it, going for the DRY principle will shoot you in the foot later. And that's what I had to say about architectural complexity. If you use that QR code, you can go, I have a newsletter, stuff like that, some further reading for any nerds who want to read some papers.

And that's it, thank you. There we go.

Okay, that was fun. I'm going to say something that's never been said in the history of React conferences. Organize your flamingo. Keep it in, I want to talk Dutch. Keep it under control. I don't want to be hit in the face by a flamingo here. Bad flamingo. Safety first.

Coolio McSwag is asking, so you mentioned in your presentation, that's how JSX felt years ago. Are you calling the audience boomers? I'm calling myself a boomer. My first reaction to React was, why would you do this? And then I tried and was like, this is amazing. I also remember my first exposure to React. I had already accepted the job and I didn't know they were using React and I was like, no. And after a few days of playing around with it, I never looked back. I honestly, I hope one day we get browsers with native JSX support. One day. Well, we just heard from Ken that you can build your own browser with Canvas. Uh-huh. So, why don't you make your own? Yeah. I'll try. Like, what are you doing this weekend?

All right. Next question from Noah. Why the flamingo? Yeah, great question. Why the flamingo? You know, flamingos are really great and if you didn't know, they aren't actually pink. They get that from eating shrimp. Uh-huh. So, don't eat a lot of shrimp, I guess is the lesson. Unless you want to be pink.

Unless you want to be pink, yes. Yeah. She's really successful. She sells at stadiums, so pink is a good thing. Swag McCoolio.

Another question. Do you use any particular tools to analyze architectural complexity of a React feature? So, I've tried to find tooling for this. There isn't anything really good out there. The best thing I've done is a tool called madge that can visualize your imports and dependencies that way. I'm trying to find something that can do it on the level of function calls, because that would be really useful. I haven't found anything. But, one interesting avenue of research, you can ask chat, you can give a JSON dump of your dependency graph, chat-gpt, and ask it what are the modules in here, what are the areas that I should organize my code in, and it will just tell you. It actually is able to find that. So, one thing I'm hoping to do this weekend is make a VS Code plug-in or something, where it could be like, hey, my code looks like crap, can you organize it, and have itself organize that. I think that would be really cool. We're not there yet. Yeah, probably you can use an apt-ast abstract syntax. Yeah. What's it stand for again? Abstract syntax tree. Abstract syntax tree. Yeah. To do that. Which is a lot of fun. And Steven from AG grid, who spoke earlier today, he knows a lot about that. So, we can work together on him. With him, on our Canvas AST React dependency browser. Perfect. We have the dream team. Yeah. All the knowledge is in here.

Alright. A question from Anonymous. What are your thoughts on atomic design? Atomic design. So, I like atomic design from, so atomic usually, if my understanding is correct, you have atoms, and then you have molecules, and then you have organisms, or something like that? Something like that, yeah. So, the atoms are like the building blocks. I would put those in a library or something, but then the molecules and the organisms are basically what I'm talking about here, where you have a more higher order component, or a higher level component that you can use when you're actually building features, because you don't want to always be artisanally assembling every single atom. Okay, that makes sense. Yeah.

Alright, next question is from Anonymous, also, two questions from Anonymous. Any recommendations for tools for showing dependency graphs in React or Next applications? Yeah, I would go with something like Madge, or I think TreeSitter can also do that. I haven't tried it yet. But one thing I like about Next, especially if you use their new slash app paradigm, they really promote you making self-contained pages that do everything. So that's kind of like what I was talking about here where once you have a page that knows how to load itself, and it knows how to load its data, it knows how to organize itself, you can then very easily move that page anywhere, make it a sub page or whatever, because it's self-contained.

Alright, then we have time for a really, really quick question. That's not a quick question, but we're gonna do it anyway. How would you recommend organizing a large monorepo with high architectural complexity? You have six seconds. I would look for the natural modules that already exist because they are in there, and then just, I hope you're using TypeScript, just start moving the code around, and the TypeScript will especially in VSCode, you just grab a file, move it into a new folder, it fixes all of your imports, and I found that that usually just works. You get an engineer, give them an afternoon or a week, and let them reorganize your files, and it's probably gonna be fine. The hard thing with that in my experience would be like, if I look at the mirror code base, we have like 900 engineers working full-time on it, and if we're gonna move things around, it's gonna be hell, right? Because there's like 200 PRs a day, even more, and if you're gonna reorganize stuff, that's really painful. So while it's possible, like you said, you have to have a team to manage also. Any tips on that? I would start small, do it with small things, and then, a happy thing, a happy accident that I found is that a moved file and changes in that file don't cause a merge conflict. Git is smart enough to figure that out and move your changes, so as long as people aren't also reorganizing files at the same time you're reorganizing them, it should be fine. Then you should be fine. What's also fine is that our time is out.