Video Summary and Transcription
Let's talk about React and TypeScript, Yarn's philosophy and long-term relevance, stability and error handling in Yarn, Yarn's behavior and open source sustainability, investing in maintenance and future contributors, contributing to the JavaScript ecosystem, open-source contribution experience, maintaining naming consistency in large projects, version consistency and strictness in Yarn, and Yarn 4 experiments for performance improvement.
1. Introduction to React and TypeScript
Let's talk about React, a framework introduced in 2011 that has become essential for web and mobile developers. TypeScript allows you to write JavaScript the way you want to and improves the development experience.
And now, let's talk about React. We're bringing free articles, videos, courses, and tutorials to help you learn all about React. Introduced in 2011, the framework has become essential for web and mobile developers alike, enabling modern apps by look and function. And while other JS tools have expanded or revised its original notions, React's popularity is still the one to beat
Did you know that TypeScript lets you write JavaScript the way you really want to? Explore the variety of TypeScript courses we compiled from our speakers and collaborators and find out what you can do with this language and how it improves the development experience
2. Introduction to Yarn and Its Philosophy
Hello everyone, my name is Mael, and today we're going to talk about Yarn, a package manager that has been thriving and will continue to do so in the future. We will focus on the project itself, its philosophy, and why it's a safe bet for your project. Yarn is designed to make developers' lives easier by allowing them to focus on writing products instead of dealing with tools and infrastructure. We will also discuss how to evaluate projects and make conscious choices. Let's start by exploring the Zen of Python and how it can be applied to Yarn. Yarn is here to stay for the next 10 years, ensuring its long-term relevance and coolness.
Hello everyone, my name is Mael, and today we're going to talk a bit about young. So first, let me tell you a bit who I am. So I work for the Japanese and social networks and I work at Datadog and part of the developer efficiency team. And our job is to make sure that developers working at Datadog, product developers, can focus on writing products and don't have to deal with maintaining tools or infrastructure or deployments or this kind of stuff.
As part of that I've been also contributing to the Yarn package manager, and in fact, leading its development since 2017. Let me ask you a question for this talk. How do you evaluate projects? All tools have their own strengths and weaknesses and it's your job, as developers, to decide which one you will want to use on a project in order to benefit the project itself. And as maintainers of open source projects, our job is to give you all the information you need in order to make a conscious choice that will allow you to move forward in your implementations.
So in order to do that, I could tell you the feature list of Yarn. But I don't think it would be as useful as many people make it sound. Indeed, a feature list is transient, it's just a point in time. If I was to tell you the feature list of Yarn, all the nice things it can do for you, it would quickly become obsolete when we are going to implement new ones. And in fact, we are working on Yarn 4. So, as you can guess, new things will come in the next version. So, instead of doing that and making this talk obsolete as I'm speaking it, we are going to focus on the project itself and how it works, why does Yarn thrive, why will it keep doing so in the future, why is it a safe bet for your project? That's what I think would be interesting to discuss.
In order to do this, I remembered something called the Zen of Python. You might not know it, but in Python, if you are doing a special type of import, you're going to get a poem printed on screen. I put the lines there, like, beautiful is better than ugly, explicit is better than implicit, simple is better than complex, you see the picture. The idea is that all those statements are actually the philosophy of Python code. So if you write Python code, it's supposed to be simple is better than complex, it's supposed to be flat more than less, this kind of stuff. I really like this format and I wonder what would it look like if it was on YARN. I wrote those statements. I'm leaving it on screen for a few seconds, but you don't have to read it, we're going to go over each and every one of them, so don't read it, just screenshot if you want.
Ok, let's start. But before we start, one last thing. YARN is there for the 10 years to come, that's our goal. So all the slides that I'm going to follow have to be put inside this context. We are working on this project not only so that it's cool now, but also so that it remains cool for the future. So, first statement. Uniform is better than varying.
3. Uniformity in CLI Conventions
Uniformity is important in our CLI. Previously, we had inconsistent conventions for CLI flags and configurations, making it difficult for users to remember them. We are now ensuring consistent nomenclature for flags, settings, and comments.
Uniform is better than varying. Our whole CLI used to have widely different convention from one comment to another. For example, you had an option that was called Noble Links, another was called Ignore Scripts, another Disable PMP, and that's only for the CLI flags. The configurations themselves had different conventions in how settings were called. The whole thing made it very difficult to remember them, even for us. For our users, it was even worse. By doing that, it also increased the chance that further deviation would happen because people contributed to the YARN codebase would have no idea how they should call things. They would have no idea about the right nomenclature.
So, now we are doing our best in order to make sure that all the flags, all the settings, all the comments all have a consistent nomenclature so that everything is neat and tight.
4. Stability, Readability, and Error Handling
Stability is a priority for us. Yarn ensures cross-platform compatibility for scripts and surfaces errors early. We prioritize readability by restricting the information shown on screen and using colors to convey meaning. We also focus on vertical space to avoid overwhelming users. Errors and warnings are valuable tools that guide users and should be actionable.
Stable is better than unstable. We want your application to have only two states, as far as we are concerned. Either it works everywhere or it crashes everywhere. Of course, it can be difficult to do that in every case, but we really do our best for this.
For example, scripts are run by cross-platform shared implementation. So, even if you're writing scripts that are meant to be run on Linux and one of your contributors is working on your project from Windows, there is a good chance that it will just work, even if you're using pipe, even if you're using redirection, because Yarn is applying an indirection so that everything is portable.
Another example is that Yarn will do the best it can to surface errors early. So, let's say that you are in production, you are removing all the dependencies, there is a good chance that you will be missing a few of them, because of the null hoisting issue of ghost dependencies as they are called. Yarn will do its best to tell you when that happens so that you can fix it. It will tell you hey this package is depending on something but it is not installed, please install it.
Readable is better than messy. So as a package manager, we have a lot of information that we can tell you. But only a few of them are actually really interesting. So we are making a conscious effort in order to restrict the information that we are printing on screen and only show the most important ones. Even more than that, we are also trying to highlight them so that they are easily parsable just by a human by using colors in semantic places. So for instance, all the package names have the same color, all the branches have the same color, all the references have the same color. You see the picture. We are using colors not to make things prettier, but to give them a meaning. Additionally, we also care a lot about vertical space because I don't have a very large terminal so it really is a problem when, for example, when you run an install, you have, let's say, 300 lines of nonsense being printed on screen. That's something that we don't want to happen. We are really making efforts to make sure that none of the PR that we merge has this kind of problem.
Errors are great tools. Don't be afraid of them. Warnings are errors or ways to include the user into the decision process. So we don't shy away of printing errors. When we are not quite sure what to do, we ask for guidance and when something doesn't feel safe, we ask for confirmation through settings, usually. But they should be kept actionable. That's the important thing about errors. If we print an error but you have no way to fix it, then it will just be frustrating for you. So too many errors will make our users blind to them.
5. Error Handling, Benchmarks, and Default Behavior
We prioritize showing errors and warnings that you can actually do something about. We have a list of error codes with detailed explanations on our website. Benchmarks are not the best way to evaluate a package manager's quality. Speed is not the main focus for us, but rather improving the developer and user experience. We aim to do the right thing by default and are working on making Yarn more secure by enabling the immutable flag on CI and adding additional checks for pull requests.
Once you start having errors that you cannot do anything about, for example peer dependency warnings, you will just not read them anymore. That's not something that we want to happen. So we try to only show errors and warnings if you actually can do something about them.
And to help you with that, we have a list of all error codes on our website along with detailed explanations for each of them. Like kind of small articles, really, that explain what is the problem and how to fix it. And frequently we improve the wording based on the feedback that we receive from our users. So we really are making conscious efforts to really write something readable.
A bit is important, benchmarks not so much. Benchmarks are usually about forget managers. Years ago they were in a very different place. A lot of them were much slower than they currently are, including YARN. But nowadays speed is rarely the real differentiator between them. YARN has automated benchmark, which we use to spot dramatic regressions, if it comes to that. And as an example of why benchmarks are not the only... are not the best way to see how good a package manager is, the same benchmark that we use ourselves on our CI don't give the same results whether we run them on CI or on laptops. So it's really difficult to give a winner in terms of speed nowadays, because it really depends not only on the package manager themselves, but also the environment they run it. Overall speed isn't a goal in itself. It's a mean to an end. We are doing this for a reason. We are doing this to improve the developer experience, your user experience. So while speed is something that we take into account, it's not the main thing that we have into our mind.
Aim to do the right thing by default. That's something that we are currently starting to adopt more and more. So until now, Yarn was a regular tool and it had the same behaviour in every environment, but the problem is that configuring a package manager is a bit difficult, especially if you have to take into account different environments, if you have to take into account security, this kind of stuff. For instance, would you spot a P.R. that would add a malicious dependency to your project? So called, supply chain addicts. It's not clear. But now we are trying to do something about it, for instance, Yarn will enable the immutable flag by default on CI so you cannot commit a logfile if it's not up to date. In the next major, we are going to add additional checks on P.R. when we detect that a branch is a P.R.
6. Yarn's Behavior and Open Source Sustainability
Yarn's behavior is influenced by npm and Yarn 1. For example, packages with a binding.jp file have an implicit post install. Some features have been deprecated, while others are kept for compatibility. Saying no to certain features is necessary for the small team to maintain code quality. YARN.NO provides hooks for custom behaviors, allowing the community to develop features independently. Users are seen as collaborators, and open source sustainability is an important topic.
So that we can tell you, we can check for, for example, one package into the cache being corrupted or this kind of thing. All without you having to configure Yarn itself. Being that status quo doesn't justify itself.
This is an interesting statement. A lot of the behavior in Yarn have been generated from npm or Yarn 1. And for instance, did you know that if a package has a binding.jp file, it has an implicit post install? Even if it doesn't declare it, Yarn and npm will still run nodjp install just because that's how it always worked.
We sometimes re-evaluate features and modernize or deprecate them. In this case, the nodjp post install has been kept because it would have breaks too many things to deprecate it. But in other cases, we have decided to deprecate a few features. For instance, bundle dependencies have been deprecated a few years ago and we've been doing just fine without that.
Saying no is both hard and necessary. Many contributions that we receive are one-shot. It's not a problem per se, but we have to assume that we're going to have to maintain the code that we're merging ourselves. Given that we're a small team, it's a lot of pressure on us. We need to assess usefulness, complexity costs and maintenance cost before deciding to merge something. Sometimes the bar isn't met and it makes the feature ineligible for core adoption. But that shouldn't be a blocker either. YARN.NO provides hooks that application can use to implement their own behaviors. For instance, YARN plugins can create comments. They can access our APIs, our JS APIs. They can work with a project. They can manipulate dependencies. They can do all kinds of things. As such, us saying no rarely prevents a feature from being developed independently. For instance, one community member created a plugin that is essentially made to replace Turbo or these kinds of tools. So it's really interesting to see what the community comes up with just by using plugins. Users are collaborators, not customers. Open source sustainability is a large topic. Perhaps you will see talks about this in the future.
7. Investing in Maintenance and Future Contributors
We work for the benefit of future contributors and invest in maintenance. We improve the release and merge experiences, advocate for good practices, and surface mistakes early. We prefer to tell you what doesn't work and let you find ways to fix it based on the context. Our team has contributed to third-party projects and answered questions.
Perhaps you will see talks about this in the future. As we said, we don't have the resources to investigate everything. We don't have the support team. We don't have the resources to do lengthy back and forth on support requests trying to figure out what's unique about your environment, this kind of thing. What we can do, though, is to share context and keep the code base healthy so that you can investigate your problem yourself. That's the only way we can make sure that all bugs can be addressed. By making sure that you have the information you need to dig into them if it comes to that.
And of course, the positive side is that some people who actually made the effort to understand what are their problems may sometimes become recurring contributors and help us maintain YARN. This would already happen. We work for the benefit of future contributors. So remember what I said about YARN being alive for 10 years? There's something called the pulse factor. If only one person is handling a project, then if this person disappears, the project has a big problem. And as I said, we want YARN to survive 10 years or more. I don't know if in 10 years I will still be working on YARN, but I know that I will probably still be using it, but working? I'm not sure. So what I will be doing, and what our whole team is doing, is to make sure that we are investing in maintenance, to help future generations of contributors.
So, now and then, we are improving, for example, the release experience, so that it shows the button on GitHub actions, the merge experience, so that we can automatically resolve the conflicts just by clicking a button on GitHub. This kind of thing. It's not entirely selfless, because the more active contributors we have mean that we have more bandwidth ourselves, less pressure, and more time to spend on R&D and fun features and things like that. We try to be teachers, and advocate for good practices. There's a school that advises to shield developers from consequences of incorrect usage to always try to do the thing that they probably want to do. Because then, it decreases the barrier to use the tool, and it's a good thing for beginners. But, there's another school, and I think I'm more of this one, that aims to surface mistakes early and explain how not to repeat them. JARN is more about the second one. It's difficult to know exactly what users want to do, so rather than make guesses and be wrong, we prefer to tell you what doesn't work, and then you can look by yourself what does it mean, and what are the ways to fix that, depending on the context. Because the context can sometimes change how you are supposed to fix a problem. Right? So, that's what we do. Benefit the ecosystem, not only direct users. Our team made dozens, and when I said dozens, I wasn't entirely sure whether it shouldn't be hundreds, but let's say dozens of keywords and comments to help improve third party projects. We have been on all the repositories, we have answered questions. And we are keep...
8. Contributing to the JavaScript Ecosystem
And we keep doing that. Plug and play have tangible effects on ghost dependencies. We've been working with third party projects to solve their missing dependencies and improve packages in a short time. We advocate for new features to other package managers and even implement them ourselves. We work with the Node project and contribute to the Node.js Loaders Working Group. We strive to be a good citizen of the JavaScript ecosystem.
And we keep doing that. Plug and play have tangible effects on ghost dependencies. Even if you're an NPM or PNPM user, you can partially thank us for decreasing the chances of ghost dependencies, because, again, we've been working with third party projects to solve their missing dependencies. Each time one of our users told us hey, Yarn is reporting a problem with this package, we look at it we decide whether it was a problem in Yarn or in the third party and we discuss with their maintainers. And it improved a lot of packages in a dramatically short time. We also advocate for new features to other package managers. And sometimes we even implement them ourselves. As an example, I myself implemented optional peer dependencies in NPM years ago. Additionally, we work with the Node project. For example, I designed and implemented CorePack for Node, which ships both Yarn and PMPM with Node.js. We also contribute to the Node.js Loaders Working Group. So that the Loader API, which is used by things like Jest in order to mock modules, is as powerful as it can be. So we're really trying to be a good citizen of the whole ecosystem and not only work to improve our projects, but everything that is JavaScript.
9. Planning, Fun, and Open Source
In order to achieve our goals, we carefully plan for the long term and make sure everyone agrees. Fixing things on the wrong layer can hurt maintainability. We focus on finding the right fix and deciding the paths to follow. We correct mistakes along the way, keeping the end goal in mind. Fun is crucial for maintainers and open-source sustainability. It's important to find pleasure and pride in what we do. This ensures the project's longevity and our commitment to it. We invest in creating a fun and toxic-free environment for everyone involved.
In order to do that, we often pick the destination and interpolate the journey. The ecosystem has many tenants, and each time you want to do something, improvements will often have to be planned very long term, because everyone needs to agree, and no one agrees at first. So you need to really discuss a long time in order for things to come to fruition. Additionally, if we were to fix things on the wrong layer, it would hurt maintainability because it would become difficult to remove them once they are not needed anymore.
So first, what we do is that we figure out what's the right fix, in an ideal world, regardless of how long it would take. And we decide, what are the paths that we want to follow in order to get there. If we make mistakes along the road, we correct the trajectory as needed, but still keeping the end of the journey in mind. So far, it's worked well. For instance, Plug and Play has been developed like this. We knew where we wanted to go, and we decided to interpolate medium steps, from where we came, to progressively improve the situation in game's dependencies.
And finally the last rule, but the most important one. If it's not fun, you're not doing it right. Actually, this one is for us as maintainers. If it's not fun, we are not doing it right. You may remember what I said about open-source sustainability. It's extremely important for maintainers to find fun in what they are doing. it creates enthusiasm, it creates energy. It's more fun when you're not the only person doing it. Maintaining a project can be difficult, stressful. We have to remember every day that we are doing this for us. If I'm working on Yarn, I'm sorry, but it's not for you. It's because I find pleasure in doing it. I find pride in doing it. That's something that is important to remember when you're developing an open-source. And it's also important to remember it as a user. Because it's also a guarantee that we are going to keep working on this project because we are going to keep finding it fun. We are keep making sure that we don't burn out while working on it. Again, we are to us to keep the environment as fun and toxic free as possible. And we are doing, we are investing in it and that's something that we really keep in mind. Even if it's the last slide, that's the most applicable to all open source in general.
Open-Source Contribution Experience
The results of the open-source contribution experience survey are in: 55% haven't tried contributing, 36% loved it, and 9% found it difficult. It's encouraging to see many people making contributions and enjoying the experience. However, the 56% who haven't tried yet is still a concern. As maintainers, we're working to decrease this number and make people feel more invested in the projects they use.
I hope you liked this talk. Now we are going to do a quick Q&A. I will be available to answer your questions. You asked the question on the experience right, of contributing to open-source. And the results are there. 55% say that they haven't tried any of contributing to open source. 36% loved it. And for 9% it was difficult. So, what do you think about it? I think it's super interesting because open-source is such a big part of being a developer today. It's a good thing to see that so many people actually try to make a contribution and that they loved it. 56% who haven't tried yet is still a bit too large. And so, that's part of the work that we are doing as maintainers of many projects, trying to decrease this number so that people can feel invested in the project they use. Those are great numbers. I am more bothered about this pretty less number, but for people, for him, it was difficult. It could be difficult at times, I guess. But really glad that most of the people liked it. Awesome.
Maintaining Naming Consistency in Large Projects
To maintain naming consistency in a large project with multiple teams, linting can be used to enforce naming conventions. Code reviews and studying existing code can also help identify patterns to follow. By reusing existing code and implementing similar features, development can be simplified.
So, let's take some questions by the audience. Darkovic asks that naming consistency is cool to have, but how to keep it consistent in a huge growing project with multiple teams. It can be difficult. Something that I noticed could work is to enforce it via linting. You have some lint rules that allow you to prevent some patterns and that can work because if you know that people often tend to use, for example, allow instead of enable, then you can just add a lint rule to prevent the use of allow and tell them, hey, no, please use this nomenclature instead. So at the scale of a large codebase, that can work. And of course there's the code review where people used to the codebase can take a look at the code as a contributor to a codebase that you don't always know. One thing that also works is to look at how the existing code works, how it is implemented, what are the name of the things. And you can often see patterns in there that you can then reuse inside your PR. And the same is true not only for naming, but also in general. When you're writing a feature, often it's very close from something that already exists. So by looking at what already exists and how is it implemented, it's not a matter but it can be simplified a lot.
Version Consistency and Yarn's Strictness
Yarn is opinionated and strives to maintain consistency and safe patterns in projects. It may be more strict than other package managers, but it helps identify issues early and ensures consistency. Yarn's goal is to provide a reliable and consistent experience for developers.
Yeah, that's nice. We never used ESLint in my any of the process before, but we did had naming convention documentation like yeah, you go, read it and name it according to that. But yeah, it could also be messy sometimes documentation. Interesting, yeah.
Thank you for answering that. I hope that answered the questions by you, Darkovik.
We have another question by Joseph Reed, and they are asking we have dozens of devs contributing to our monorepo. Is it recommended that everyone be using the same exact version of Yarn or should that matter? Yeah, for sure. Even though we try not to ship breaking changes between release except major, of course we are humans and we can make mistakes or we can fix bugs that you are relying on or we can add bugs into our feature or releases. So usually what I noticed is that if everyone has the exact same version of a package manager whatever it is, then they are guaranteed to have the exact same behavior. If there is different versions there is always something that will be slightly different from one developer to another and even though in general it might not be a problem it will be a problem at some point and it will cause a very difficult to debug situation that you don't want to happen.
So with Yarn until a year ago we had the Yarn path setting that allowed you to check inside your repository so that it is transparently used by anyone in your company and with Node we worked during the past year in order to ship Core Pack which allows you to do the same thing but without checking in Yarn inside your repository. You just have to set a package manager field inside your packet.json and provided people installed Yarn using Core Pack they will get the version that you described. Essentially it is a way of locking down the version of your package manager. Okay. Yeah that's it. Thank you for answering that.
We have another question by Cici Miller. I once heard it said a good package manager will be able to tell you the reason not to use their product. What would your answer be to that and how can we help that answer go away in the open source community? That's an interesting question. I would say that Yarn is perhaps a bit more opinionated than other package managers. We really try to tell you when something doesn't seem right by our standards. For example, when a dependency is accessed but is not listed we tell you that it's a problem. In other package managers it might work because it usually works until it doesn't. Yarn really masterfies the errors early which can be a friction for adoption because some packages are not intended to support this kind of strictness and there is some work to do. That's something that I mentioned in the talk we have been doing with the ecosystem in its entirety. We went to a third party project that had an issue with that and we discussed it with them, we work with them to fix those problems in all the cases that were a reference for us. There are still a few that remains but every now and then we have the situation where we need to add a little bit more configuration in order to instruct YARN that some dependencies should be there and it can proceed. So in general I think the main disadvantage of YARN is also its biggest advantage in that we are very strict and we really want your project to be consistent and to have safe patterns. Yeah, consistent is definitely very much important.
Finding Ideas and YARN 4 Experiments
We find new ideas for improvement through our team of contributors and the Open Source community. We rely on people contributing to the project to help us improve and work on the ideas we have in mind. Currently, we are working on YARN 4, experimenting with the offline mirror and bundle splitting to improve performance. We have a large backlog of interesting small work.
So ok, we have some generic questions. How do you manage to find new ideas for improvement? We have a team of contributors. So, usually any one of us has ideas about how to improve YARN. An Open Source project always has more ideas than people to work on them. So that's also why we rely so much on people contributing to the project because then it helps us improve those crazy ideas we have in mind that we really want to work on but don't have quite the time yet. But every now and then, for example, right now we are working on YARN 4, the next major version of YARN. So we take the time to make some more crazy experiments in order to see what we could improve. So, for instance, in YARN 4 we will be experimenting a few things with the offline mirror by trying to disable it by default instead of enabling it by default. We've been also experimenting with bundle splitting YARN. We are trying to decrease the runtime of a read of the YARN binary. We are doing a lot of very interesting small work and usually we have a large backlog. That's how we do it.
Comments