Supercharging Your Tooling With Rust

Hi folks, my name is Mael. I work in Developer Efficiency in the NGTX team at Mistral.ai, and today I want to talk to you about Yarn in Rust, a project I've spent most of my time these past years. I'm going to tell you a lot about it, including why we did that and how it went. So, let's get started.

Before going further, a bit of history about the project, Yarn originated in 2016 in 0.x Moniker version. It's only in 2017 that we felt confident enough to tag Yarn as 1.x. I was there, I pressed that damn button. However, about two years later, in 2019, I started to notice that we had learned so much ever since we started the project that if we were to create a new package manager now, we wouldn't do it the way that we did before.

And that was a problem, because the way I saw it, we needed to maintain Yarn not only now, but also for the 10 years to come. So at the time it was decided to create a new release that would take everything we had learned about making a good package manager and would put it together under a new implementation of the same package manager. And that's how Yarn 2 was born. Of course, this version came with some issues and we're going to cover some of them in the next slides. But overall it succeeded, because now we are in 2016 and we are still using Yarn 2.

Of course, it evolved. Now we are at Yarn 4, actually. But the codebase itself stood the test of time. And now in 2026, we are building something new. We are building Yarn 6, the next major, major version of Yarn, which is going to be written in Rust.

And now you can ask yourself, why are we doing that? We already rewrote Yarn once. It was already a bit painful for everyone, for the ecosystem. So why are we doing? That's a very good question. I can't wait to tell you all about that.

First, the thing to realize is that today Yarn is stable. Yarn is an extremely stable software. It is the poster child of no surprise. If Yarn tells you that something doesn't work, then something doesn't work. And by fixing it, you will not have this error message or in the future. And your colleagues are not going to have this error as well. Yarn is deterministic to a fault.

We rarely had any install bugs. We installed reasonably fast. Of course, PMPM was sometimes a bit faster than us. Sometimes we were faster than PMPM. It was kind of a good challenge. But Yarn was very fast at making typical installs. And building features was easy because we knew the codebase in and out.

So all of that made good reason why we should stay on the codebase that we're familiar with, right? Yes, but we had somehow reached the ceiling of what we could build with the codebase that we had about Yarn 4. We supported, for instance, monorepos with hundreds of packages, which was great. But we couldn't support workspaces with thousands of repositories. Well, we did, but it was much slower than we anticipated.

A thousand workspaces in the same monorepo may seem excessive, but many large companies are doing things like that privately. They approached us, mentioning that Yarn was working fine but starting to feel sluggish. In some repositories, Yarn could have more than one second of latency, which, when compounded, led to significant delays in user experience. Our goal was to enhance the user experience for everyone and implement new features in Yarn.

Who does that? A lot of very, very large companies are managing monorepos privately. They approached us, expressing concerns about Yarn's performance. In some repositories, Yarn experienced latency issues, affecting user experience. Yarn's subprocesses and complex processes led to significant delays.

Improving user experience was a priority. Ideas like lazy install required high performance. Lazy installs needed to be fast to maintain a seamless user experience. Yarn aimed to be exceptionally fast, surpassing existing ecosystem performance. Yarn's ambition extended to being a package manager for multiple ecosystems beyond JavaScript.

Yarn's vision included support for Python, Rust, and other ecosystems. The goal was to create features applicable across different programming languages. Having diverse language support was essential for accommodating various developers' needs. Each ecosystem's unique requirements demanded a versatile and efficient package manager for a seamless development experience.

Another thing is that we had ideas. We wanted to build new features in Yarn. But some of them needed massive performances. One of them, for instance, is lazy install. The idea is that when you're running a script, if your repository is not up to date, if your dependencies are not up to date, then the package manager should or you install all the missing dependencies without you noticing that something had to run. This is especially important for people working on a git repository. Because when switching from one branch to another, you're frequently going to have differences between your dependency tree. And the package manager handling this mental task for you means that you can focus on what really matters.

But in order to make that possible, we needed lazy installs to be really fast so that running Yarn run would not appear sluggish to our users because they would spend some time installing things under the hood. So, we wanted Yarn to not be just fast, but very, very, very fast. Much faster than anything that existed in the ecosystem. Another reason why we did that challenge and why we thought that writing Yarn was the right time is that I always envisioned Yarn to be a package manager not just for JavaScript, but also for multiple ecosystems. What about Python? What about Rust?

When we are building a package manager, when we are writing features for them, it's so frustrating that we are building things that will only work if you work on a JavaScript project. And in my company, we have people working in JavaScript, but we also have people working in Python. Other people working in Rust. Other people working in Go. And each of those ecosystems has a different package manager. And any feature I could implement for the JavaScript folks would not be used by anyone else in the company. And that's somewhat frustrating. Because when you think about it, there aren't really a lot of reasons why the code that we wrote for workspaces would not work for people writing Python.

And similarly, there is no real reason why the code that we were writing to run audits would work on JavaScript, but not for people doing Go. So, I really envisioned a package manager that could support multiple ecosystems. But Node.js didn't really make it easy to do so. Primarily for distribution problems, of course. Because if you're distributing Yarn in JavaScript, then it means that you need to have Node.js installed, which means that in order to develop Python or Go or Rust programs, you need to have Node.js installed, which doesn't make a lot of sense for our users. And then, that was one extra prompt to rethink how we envisioned Yarn to be developed.

So, for all those reasons and more, I have to go fast here. We felt it was time to reinvent ourselves, to try something new, to go the next way. To prepare Yarn, not only for the 10 years that came, but also for the 10 years that were coming now. The first time we did a rewrite in 2019, we learned a bunch of things. Something went well. Some things didn't go well. And I didn't want to create a new implementation of Yarn without thinking carefully about the problem that we faced then and how could we mitigate them now. Now, there were two main reasons why Yarn 2.0 had such a strong reaction from the ecosystem.

The first one is that it had too many breaking changes. At the time, we felt that, oh, since we are writing code base, now is the right time to try to do something, to do things differently, right? Wrong. That's not the right time. When you're making a rewrite, things are already very complicated. So, if you're on top of that, you're also making breaking changes, you're reaching a critical mass of pain for your users, and they are going to resent that. Now, the second issue is that there was way too much friction. Part of that was due to our attempt to make things as seamless as possible for our users, which is a bit ironical when you think about that.

For instance, we didn't really change the binary that was installed by npm install dash nt Yarn. It kept being Yarn 1.x, but it means that a lot of people were asking us, how do I install Yarn 2.0? It was difficult to give them a straight answer without actually doing something that would risk breaking people using Yarn 1.x. We stayed that way for a long time, and only now are we going to be able to provide a good answer to that question. So, that was something that was very important to not get wrong this time.

Also, another thing is that, in 2019, we rewrote Yarn from scratch. We rewrote even the test suite. In 2026, things are different. We are not actually rewriting entirely from scratch, because we have all our tests that we wrote in 2019 that we can reuse now, which means that it's kind of a data-driven development, because now, thanks to a test architecture that we designed well in 2019, we are able to just run the exact same test with Yarn 6, even though the implementation is completely different, even though now it's a Rust binary. Our tests are just running a binary, passing its argument, its command, and then checking that the results on disk are by spouting Node.js or the expected one, and this pattern has worked so well for us.

The entirety of our test has been portable from one codebase to another. That has been super helpful for this rewrite. Because then, we could also build a neat UI to allow us to track progress, and I'm going to go over that later, but tracking progress is very important when you're making a change of this amplitude. Now, thanks to all that work, we know that out of the 1,500 tests that were originally in the Yarn test suite, almost 1,400 are now passing. That's great, because not only does it give us some very large confidence that the new rewrite is going to work for our users, it's also very motivating to see those numbers go up as we make more peers.

That's great, because not only does it give us some very large confidence that the new rewrite is going to work for our users, it's also very motivating to see those numbers go up as we make more peers. So, it's really too much one stone. So, how did it go for this rewrite? What did we learn from that? The first thing is that it involved learning a new language. You know, the project started early during the AI boom. At the time, we were not using AI for everything, and that changed a bit how we approached this rewrite. Still, even at that time, TypedJPT and other tools already existed, and the question tell me how you would do that type of code, but in Rust, was already useful. I remember that I shared frequently snippets of JavaScript and asked how would that be written in Rust? Because I don't know anything about Rust.

And then sometimes it was really bad, and then I was able to see that it was very bad, because I still have some common knowledge of what a good code is supposed to look like, and I was able to look for more pointers on the internet. It gave me leads to understand exactly what was happening. It was also very interesting to see my personal Rust proficiency grow over the course of the project. I remember that in some cases, I rewrote the same part of the codebase multiple times because I wrote them one month, and then the next month, I came back to that code and noticed, oh, but I could use this new construct I learned during this other task, and then I improved it.

That didn't make the development faster, but it made it much more fun. I also learned a couple of things from Rust itself. The first one is that Rust has good tooling. Cargo is really nice, and it's actually a good package manager. I came a long time ago from the C++ ecosystem, and in C++, you don't really have a package manager, a native package manager. You have some options, but they are all nonstandard, meaning that every project is doing different things. In Rust, you don't have that, because they actually have developed an official package manager, and they are maintaining it well, and you can feel that.

Another thing I learned from Rust is that there are so many very interesting experimental features that are stuck in limbo. Sometimes you're searching, okay, how can I do this kind of thing? And then you go to GitHub, and you find an issue about someone saying, yeah, that has been implemented, but it's experimental, so you can't actually use it. And then you look at when was the last time that this thread was updated, and you see that it was, oh, five years ago. And that's quite frequent when you're learning the language, and it can be a bit frustrating.

That said, Rust was a great language. It prevented a lot of concurrency or shared memory pitfalls, especially as we were writing more concurrent aspects of YARN, like, for instance, fetching packages in parallel or running resolution in parallel. All this kind of thing has a fairly high amount of dangers that could arise if you're following bad code patterns, and Rust allowed us to avoid a lot of those issues. That said, it didn't come without cost, and it required a lot of mental model shifts trying to understand not only how the code was meant to work, but also how Rust wanted us to write that code.

The idea in Rust, of course, is that the compiler is helping you enforce rules, but those rules don't always make sense at first read, and sometimes you really have to think hard to understand why the compiler is feeling a certain way. Usually the compiler is right, and I'm wrong, but in some cases, it happened that the compiler just wasn't smart enough to do something, and I just needed to do it another way. Now, an elephant in the room, is it an AI forward rewrite? As I told you, the YARN rewrite originated about a year and a half ago, and back then, AI was nowhere where it is now, so we actually used AI sparingly, unlike BUN for instance, that a couple of weeks ago announced that they had completely rewrote their implementation from ZIG to BUN.

That's not what we did. Here, most of the code has been written by hand, by humans, and considering Rust's complexity, I actually feel right about that because since we want to be maintaining this code in the future, I feel more comfortable with a human-driven rewrite that I made, and that I understand.

That said, AI was still used in places, it was used for code reviews, some investigation around very weird bugs, so that we could find reproductions, also building some experimental features, especially near the end of development, and porting some easy tests.

Now, we are reaching the conclusion of this talk. Did anything go wrong during the rewrite? That's a very interesting question. Any rewrite is a very difficult work. It's a brutal endeavor. You are probably preparing, you have to prepare yourself for a lot of problems doing that.

That said, it worked well in our case, but only because we were very perseverant, and that we knew where we needed to be, and we had all those tests that we could trust to know where we were in the reimplementation process. Other things that could go wrong, well, if you spend your time writing the things, when can you actually integrate it? That was somewhat a concern when I was doing that, because at the time, I was working at Datadog, and the thing is, either I was building this Yarn rewrite, or I was integrating the Rust version of Yarn inside the Datadog's infrastructure. And doing the two at the same time proved very difficult. In the end, I managed to do it, but it required more time than I had anticipated.

Another issue with the rewrite is that can you afford freezing your main branch for multiple months, the time for you to actually spend a lot of time working on that rewrite? Because your competitors are not doing that. They are still cranking features, they are still fixing bugs, and if you stay frozen for too long, then your user may start to resent that. In our case, this was offset by trying to spread the rewrite over a long time, that's why it took one year and a half, so we didn't completely freeze Yarn 4.x, and we kept doing features and improvements there, but it can be a danger, and not all projects are ready to do that.

Finally, the main issue perhaps of doing a rewrite like that is that for the first few months, it kind of feels like walking in the desert. You don't have any results, and you're not entirely sure whether it's going to work. Having the test that we had, we kind of knew that we could just figure the test one by one and we would end up getting something. But still, we didn't have those results at the beginning, and so it was a leap of faith to go into that project and expect everything to work out of the box a couple of months later. So, given all that, should all projects be rewritten into Rust? Since Bion did that, the NPM is doing that, they are rebuilding part of the package manager, especially the fetching and linking steps to be in Rust. You have all the other package managers that are trying to go Rust as well. There were talks about ESLint going into Rust at some point. So, should all projects be Rust? I would personally be cautious about that, and if you're a maintainer, I would tell you that Rust is not a silver bullet. It will not make magically better. It's going to come with its own trade off that we are going to have to tell how you develop whether you're using AI or not. If you're not using AI, then you're going to have to be familiar with how the Rust compiler expects you to write code, and that takes a lot of time. If you're using AI, then you have to be comfortable with the fact that you never truly know your code base. Is it something that you will be fine with? That's up to you to decide. In our case, since I wanted to maintain this code base for a very, very long time, I figured that the choice I had made not to use AI to make a full rewrite, choice that was partly enforced by not having good options to do that at the time, was the right one.

We made a whole talk about YARN and a rewrite, but I'm guessing that you have questions. What are the performances? Did it deliver on the perf side? The answer is yes. YARN is faster than everything else, including YARN 4.X, including PMPM, including, of course, NPM. YARN is much faster now. We are still not done. There are still low-hanging fruits that we haven't touched yet because we are still focused on making all our tests fast, then making the Windows implementation, all kinds of things. I'm sure that we are going to bring the install time even lower.

Something else to keep in mind is that this isn't the time I showed, or the average, across many scenarios, but some scenarios outperform. Like, for instance, here, where YARN is only taking less than a second in order to do some small installs, and those scenarios are important because installing a single package is something that a lot of commands are doing. When you're adding, when you're making an auto-import in VS Code, you're making just a very small install, so making it fast and not have to spend one second and a half doing work is going to make your life much better in the IDE as you're writing code.

Was this project the right choice for YARN? Was this rewrite the right thing to do? The thing is, we built the package manager we needed for the next decade. I don't imagine us doing another rewrite for a very, very, very long time. The thing is, motivation and resilience were key to make this work, and if you're not ready to do this gruesome work, then perhaps rewriting your project in Rust is not exactly the thing I would recommend you to do. The thing is, you may remember at the beginning of this talk I told you about how YARN was envisioned to support JavaScript, but not just JavaScript, also Python, Rust, and all the things.

The thing is, we are still working on that. It's not entirely clear. It will probably not be at release date, but YARN is moving towards this direction, and that's incredibly exciting for me because I always wanted development to be universal. Your tooling should be universal if you want to really be able to dig into other ecosystems. The thing is, one last thing, I'm writing a bunch of devlogs which are going to be shared on both BlueSky and Twitter, and our Discord channel, of course. So, feel free to join our communities there, and I would love to have your feedback on what you thought about this talk, but also this rewrite, whether you're interested in Rust yourself, whether you have questions, feel free to shoot them, and I would be happy to answer all that. Have a good day.