What's Inside Biome's Linter?

Hello, everyone. How's it going? So today, we're going to talk about the Biome Analyzer and what's behind it.

So before going forward, who I am. So my name is Emanuele Stoppa. I'm Italian. I live in Ireland. I like open source, games, traveling. And I'm also so into open source that I'm into two projects, Astro and Biome.

Today, we're going to talk about the Biome Analyzer. So what's really curious about the Biome Analyzer? Well, Biome Analyzer is so fast. I'm going to look at why it's that fast. It's not just a linter. It's much more. A linter is just a smaller thing. And it's not just a CLI tool. It's also something more. So, let's do it.

So, why Biome is so fast? So, there are, among other things, there are three things that I want to explain to you. And so, why is Biome so fast? Like, it takes advantage of multi-threading. So, it spawns multiple threads for each file. It uses this kind of channels to keep the communications among the different threads. And we use, like, aggressive caching during the parsing phase. Now, multi-threading. So, when each command that you run from the CLI, like formatting, analyzing, checking, this kind of stuff. So, each command that crawls your file system. What Biome does is that once it identifies those files that are eligible for being handled, let's say, Biome spawns a thread. So, each thread is responsible of its own file. And it parses it. It analyzes it. And it emits some signals that could be, like, if there's a diagnostic, if there's a code action, and more.

Now, all these threads, when they are spawned, they are not aware of each other. Like, they just do one job. At the end, they have to report something, like if there are errors, or if not, like any kind of information. In order to do so, we use channels.

So, you have all these files. For each file, we have these threads. There are n threads, depending on the operating system. Then we have the main thread. So, the main thread waits for all these threads. And it starts collecting information from all the threads.

So, using these channels, we have multiple channels with multiple senders, which are essentially the threads. And one receiver, which belongs to the main thread. And once there are diagnostics, we collect them. We collect if there are warnings, errors. If, like, we skipped like some diagnostics due to some restrictions or options and things like that. So, that's how the communication happens. And once all the threads are died, the main thread can resume its work and report everything to your console.

And then we have reusable tokens. So, essentially, what does that mean? So, once biome parses your file, it creates tokens and nodes. These are essentially pointers to a block of memory on your operating system. And these pointers are saved, like the references are saved into a caching object. Okay? Once a reparse of the same file happens again, let's say a code action occurred and that action changes this snippet from let to const. We do a reparse to make sure that there are no more triggered rules. When we reparse it, essentially, the nodes that belong to msecret equals and the string are reused. So, instead of creating a new node, we have it there already. So, we have that reference that says that msecret points to that block of memory. Let's just use it. Let's not create a new one. So, that's how for each document we reuse the same thing. So, like memory wise, there's no waste at all.

Neat, right? Now, before I said that it's more than a linter. Exactly. So, a linter is just a small part of the analyzer. Biome's analyzer is able to complement the parser. It's also able to provide other features like import sorting and even more.

So, how does it complement the parser? So, depending on the project, the project can decide that some things regarding the parsing phase can be out of scope, non-goals. For example, semantic validation or other things. And we can actually use the analyzer to complement the parsing phase by emitting new diagnostics using the analyzer.

In Biome, for example, we have a few syntax roles. That's how we call them that are emitted and triggered before the actual, for example, the actual linting or the actual formatting. So, if the code that you try to lint has some syntax errors, they are emitted and the linting phase actually stops. We can't or the formatting phase stops. We can't do that. So, that's how an analyzer can help even more.

Import sorting. Imports are actually coming from Biome analyzer. It's not a lint role. There's nothing about the linter. It's part of the same infrastructure. We just use it in a different way. And that's how it works. Like, you just enable your IDE to call the import sorting. And once you save, it just does that for you. No need for diagnostics or anything like that.

And even more, like, the analyzer is actually put the foundations to, like, transpiling the code. So, for example, transforming and transpiling JSX to JavaScript. So, some kind of a bubble thing. And assist. Like, nice trivia, the import sorting is actually an assist. So, in the future, in the near future of Biome, you will be able to see more of this analyzer.

So, the analyzer is actually LSP ready. So, like, you can configure it with a small pane with your IDE or any editor that supports the LSP. You will be able to see automatic sorting of JSON keys or JSX element attributes or JS object attributes. So, anything that can be, like, formatted that is opt in. Okay?

And since Biome is also a CLA tool, via configuration, you can actually impose these refactors and make the CLI fail, if, for example, the JSON keys of a certain JSON file isn't sorted. So, like, the best of two worlds.

And everything comes also to this video. Like, here you can see how fast is Biome. Like, I downloaded the webpack repository and the rollup repository and the TypeScript codebase of Rome tools. And see how fast it is. Like, we don't print diagnostics, because printing diagnostics is expensive. But see, like, 13,000 files. Even really big. Because the old TypeScript codebase contains a lot of big files for parsing purposes and testing. And it's freaking, freaking fast. So, like, great DX, I would say. Thank you.