Efficient Package Management with pnpm

Introduction to Package Managers

In the world of Node.js and JavaScript, package managers play a crucial role in handling dependencies and facilitating smooth software development. Among the popular package managers, npm, Yarn, and pnpm have carved out distinct niches due to their unique features and capabilities. While npm is the official package manager of the Node.js ecosystem, Yarn and pnpm emerged as alternatives to address specific shortcomings of npm, particularly in its earlier versions.

These package managers have evolved over time, with each offering different approaches to dependency management. Yarn, developed by Facebook, introduced innovative features like Plug'n'Play, which garnered both praise and criticism. Meanwhile, pnpm, despite being less popular initially, has gained significant traction due to its efficient use of disk space and faster installation process.

The Concept of Hardlinks and Disk Space Efficiency

One of the standout features of pnpm is its use of hardlinks to optimize disk space usage. Hardlinks allow multiple files to point to the same physical location on the disk without consuming additional space. This means that even if several projects share the same dependencies, pnpm ensures that there is only one copy of each file on the disk, linked across different projects.

However, it's important to note that hardlinks have their limitations. They can only exist within the same disk, which poses challenges when working with multiple drives or file systems. pnpm addresses this by creating separate storage for each disk, ensuring that projects on different disks have access to their respective dependencies without duplicating files unnecessarily.

Migrating from YARN to pnpm

For developers considering a switch from YARN to pnpm, the process is relatively straightforward. For a single package repository, the transition involves using the pnpm import command to convert the YARN log file to a pnpm log YAML. This simple step, along with updating CI scripts, allows pnpm to serve as a drop-in replacement for YARN.

In a workspace setting, the migration might require additional adjustments, particularly in listing package globs. Unlike YARN, which uses a field in package.json for this purpose, pnpm relies on a separate file, pnpm-workspace.yaml. Despite these differences, the overall migration process is designed to be smooth, leveraging pnpm's efficient handling of dependencies.

Understanding pnpm's Unique Node Modules Structure

pnpm distinguishes itself from other package managers through its unique node modules structure. Traditional package managers like npm and Yarn Classic hoist all sub-dependencies to the root of the node_modules directory, which can lead to potential conflicts and issues.

In contrast, pnpm maintains an isolated structure. It places only the direct dependencies at the root, while sub-dependencies are linked through symlinks to a dedicated pnpm folder. This approach prevents accidental access to undeclared dependencies, making codebases more robust and less prone to breaking changes.

Monorepo Support and Advantages

Monorepo setups, where multiple projects are managed in a single repository, pose unique challenges for dependency management. In traditional hoisted node_modules layouts, dependencies from different projects can inadvertently become accessible to each other, leading to potential issues during deployment.

pnpm addresses this by creating a single .pnpm folder at the root of the monorepo, ensuring that each project has access only to its declared dependencies. This isolated structure is particularly beneficial in preventing dependency conflicts and maintaining the integrity of individual projects within a monorepo.

Speed and Performance of pnpm

Another compelling reason to consider pnpm is its speed. Unlike npm and Yarn, which install dependencies in stages, pnpm processes each dependency separately. This parallel processing approach allows for faster resolution, fetching, and installation of packages.

pnpm's speed is further enhanced by its deterministic node modules structure and the use of a content addressable storage system. This system stores files based on their hash, ensuring that identical files across different package versions are stored only once, significantly reducing installation time.

Flexibility and Use Cases

pnpm offers flexibility for various use cases. Its ability to bundle as an executable means it can be used even on systems without Node.js pre-installed. This capability makes pnpm a versatile tool, potentially replacing other version managers like nvm or volta.

Choosing the right package manager depends on specific project needs. While npm, Yarn, and pnpm each have their strengths, pnpm's disk space efficiency, speed, and robust handling of dependencies make it a compelling choice for many developers.

Conclusion

In the ever-evolving landscape of software development, understanding the nuances of different package managers is essential. pnpm, with its unique features and efficient approach to dependency management, offers significant advantages in terms of speed, disk space usage, and reliability. Whether for single projects or complex monorepo setups, pnpm provides a robust solution that addresses many of the challenges faced by developers today.

As the ecosystem continues to evolve, staying informed about these tools and their capabilities will empower developers to make informed decisions, ensuring smoother and more efficient development processes.

Watch full talk with demos and examples:

Rate this content
Bookmark

From Author:

You will learn about one of the most popular package managers for JavaScript and its advantages over npm and Yarn.

  • A brief history of JavaScript package managers
  • The isolated node_modules structure created pnpm
  • What makes pnpm so fast
  • What makes pnpm disk space efficient
  • Monorepo support
  • Managing Node.js versions with pnpm

This talk has been presented at DevOps.js Conf 2022, check out the latest edition of this JavaScript Conference.

Watch video on a separate page

FAQ

Pnpm is a fast and disk space efficient package manager for JavaScript that aims to address issues found in npm versions 3 and 4. It utilizes a unique node module structure to ensure that dependencies are isolated and only direct dependencies are accessible to a project, reducing the risk of breaking changes.

Pnpm saves disk space by using a content addressable storage system, wherein files are saved based on a hash code derived from their content. This approach means if multiple projects use the same package, they will share the same files in the storage, significantly reducing disk space usage.

Pnpm is faster because it handles package installation stages separately for each dependency. This allows some packages to be resolved or fetched while others are still being fetched or written to node modules, enhancing the speed of operations.

Yes, pnpm provides excellent support for monorepos. It maintains an isolated module structure, ensuring that projects within a monorepo only have access to their own dependencies, which prevents accidental access to unrelated project dependencies.

Migrating from YARN to pnpm can be straightforward, especially for single package repositories. Pnpm provides an 'import' command that converts a YARN lock file to a pnpm lock file, making the transition smoother. For workspaces, adjustments might be needed for package listing due to differences in configuration files between YARN and pnpm.

Pnpm can handle multiple drives or filesystems by creating separate storage for each disk. This approach maintains efficiency on a single disk using hardlinks, but will resort to copying files across different disks, slightly reducing the space efficiency benefits in such scenarios.

Pnpm uses both symlinks and hardlinks to manage files. Symlinks are used to reference the real location of packages, aiding Node.js in dependency resolution. Hardlinks are used to link files from the content addressable store, saving disk space without creating duplicate files.

Zoltan Kochan
Zoltan Kochan
31 min
24 Mar, 2022

Comments

Sign in or register to post your comment.
Video Summary and Transcription
pnpm is a fast and efficient package manager that gained popularity in 2021 and is used by big tech companies like Microsoft and TikTok. It has a unique isolated node module structure that prevents package conflicts and ensures each project only has access to its own dependencies. pnpm also offers superior monorepo support with its node module structure. It solves the disk space usage issue by using a content addressable storage, reducing disk space consumption. pnpm is incredibly fast due to its installation process and deterministic node module structure. It also allows file linking using hardlinks instead of symlinks.

1. Introduction to pnpm and its Popularity

Hello, my name is Zoltan Kochan and today I'm going to talk about pnpm, which is a fast disk space efficient package manager. Let me introduce myself. I work remotely for BIT, a company that helps developers implement component-based development. Before that, I worked for Just Answer for nine years and have been developing and maintaining npm-pm since 2016. npm, yarn, and pnpm are the most popular JavaScript package managers. npm had issues in the past, but alternatives like yarn and pnpm were created. pnpm is an indie project that fixes the issues of npm v3 and is now supported by bit. It had a spike in popularity in 2021 and is used by big tech companies like Microsoft and TikTok. pnpm is unique because it only hoists the necessary dependencies to the root of node modules.

2. Isolated Node Module Structure with PMPM

But what will happen if a new version of Express comes out and cookie is not in its dependencies anymore? Your package will break. PMPM prevents such mistakes. With PMPM your code has only access to packages that are declared as dependencies of your project. In this example, Express is a direct dependency and KUKI is a dependency of Express. The real location of Express is inside the .pmpm folder. So nodejs will search for KUKI in the dedicated nodemodules folder which is at .pmpm express at 4.17.3/.nodemodules. Each package is in its own isolated folder with its own dependencies.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Modern Web Debugging
JSNation 2023JSNation 2023
29 min
Modern Web Debugging
Top Content
This Talk discusses modern web debugging and the latest updates in Chrome DevTools. It highlights new features that help pinpoint issues quicker, improved file visibility and source mapping, and ignoring and configuring files. The Breakpoints panel in DevTools has been redesigned for easier access and management. The Talk also covers the challenges of debugging with source maps and the efforts to standardize the source map format. Lastly, it provides tips for improving productivity with DevTools and emphasizes the importance of reporting bugs and using source maps for debugging production code.
The Future of Performance Tooling
JSNation 2022JSNation 2022
21 min
The Future of Performance Tooling
Top Content
Today's Talk discusses the future of performance tooling, focusing on user-centric, actionable, and contextual approaches. The introduction highlights Adi Osmani's expertise in performance tools and his passion for DevTools features. The Talk explores the integration of user flows into DevTools and Lighthouse, enabling performance measurement and optimization. It also showcases the import/export feature for user flows and the collaboration potential with Lighthouse. The Talk further delves into the use of flows with other tools like web page test and Cypress, offering cross-browser testing capabilities. The actionable aspect emphasizes the importance of metrics like Interaction to Next Paint and Total Blocking Time, as well as the improvements in Lighthouse and performance debugging tools. Lastly, the Talk emphasizes the iterative nature of performance improvement and the user-centric, actionable, and contextual future of performance tooling.
Yarn 4 - Modern Package Management
JSNation 2022JSNation 2022
28 min
Yarn 4 - Modern Package Management
Top Content
Yarn is a package manager that focuses on stability, performance, and security. It offers unique features like plug and play installation, support for nonmodules, and the exec protocol. Yarn is committed to being a good citizen in the open-source community and contributes to fixing dependencies. It is part of the Node.js Loader's working group and advocates for Corepack. Yarn is still experimental but is improving its user experience and security features. Contributions are welcome, and switching to Yarn can improve performance in large projects.
Beyond Virtual Lists: How to Render 100K Items with 100s of Updates/sec in React
React Advanced 2021React Advanced 2021
27 min
Beyond Virtual Lists: How to Render 100K Items with 100s of Updates/sec in React
Top Content
The Talk discusses optimizing rendering of big tables using Flipper, a new version that is ten times faster with improved user interaction and richer data. It explores optimizing rendering with React, virtualization, filtering, sorting, and windowing techniques. The introduction of the Flipper Datasource packet simplifies handling updates, inserts, and removals. The performance of the Flipper data source package is excellent, even in a debug build of React, with minimal CPU usage. The Q&A session covers incremental sorting, dynamic row height, and the potential for two-dimensional virtualization in the future.
Debugging with Chrome DevTools
JSNation Live 2021JSNation Live 2021
11 min
Debugging with Chrome DevTools
Top Content
Here are some tips for better utilizing DevTools, including using the run command, customizing keyboard shortcuts, and emulating the focus effect. Learn how to inspect memory, use the network panel for more control over network requests, and take advantage of console utilities. Save frequently used code as snippets and use local overrides for easy editing. Optimize images by using a more optimized format like AVIF and track changes in the network panel to see the reduced data size.
High-Speed Web Applications: Beyond the Basics
JSNation 2022JSNation 2022
30 min
High-Speed Web Applications: Beyond the Basics
This talk covers the latest features in Chrome DevTools, including network tab analysis, performance tab optimization, and user flows. It discusses optimizing HTTP requests with fetch priority to improve loading time. The performance tab provides insights on frame drops, long tasks, and the importance of minimizing total blocking time. The talk also highlights the optimization of page rendering and introduces user flows in Chrome DevTools.

Workshops on related topic

React, TypeScript, and TDD
React Advanced 2021React Advanced 2021
174 min
React, TypeScript, and TDD
Top Content
Featured WorkshopFree
Paul Everitt
Paul Everitt
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
Mastering advanced concepts in TypeScript
React Summit US 2023React Summit US 2023
132 min
Mastering advanced concepts in TypeScript
Top Content
Featured WorkshopFree
Jiri Lojda
Jiri Lojda
TypeScript is not just types and interfaces. Join this workshop to master more advanced features of TypeScript that will make your code bullet-proof. We will cover conditional types and infer notation, template strings and how to map over union types and object/array properties. Each topic will be demonstrated on a sample application that was written with basic types or no types at all and we will together improve the code so you get more familiar with each feature and can bring this new knowledge directly into your projects.
You will learn:- - What are conditional types and infer notation- What are template strings- How to map over union types and object/array properties.
Integrating LangChain with JavaScript for Web Developers
React Summit 2024React Summit 2024
92 min
Integrating LangChain with JavaScript for Web Developers
Featured Workshop
Vivek Nayyar
Vivek Nayyar
Dive into the world of AI with our interactive workshop designed specifically for web developers. "Hands-On AI: Integrating LangChain with JavaScript for Web Developers" offers a unique opportunity to bridge the gap between AI and web development. Despite the prominence of Python in AI development, the vast potential of JavaScript remains largely untapped. This workshop aims to change that.Throughout this hands-on session, participants will learn how to leverage LangChain—a tool designed to make large language models more accessible and useful—to build dynamic AI agents directly within JavaScript environments. This approach opens up new possibilities for enhancing web applications with intelligent features, from automated customer support to content generation and beyond.We'll start with the basics of LangChain and AI models, ensuring a solid foundation even for those new to AI. From there, we'll dive into practical exercises that demonstrate how to integrate these technologies into real-world JavaScript projects. Participants will work through examples, facing and overcoming the challenges of making AI work seamlessly on the web.This workshop is more than just a learning experience; it's a chance to be at the forefront of an emerging field. By the end, attendees will not only have gained valuable skills but also created AI-enhanced features they can take back to their projects or workplaces.Whether you're a seasoned web developer curious about AI or looking to expand your skillset into new and exciting areas, "Hands-On AI: Integrating LangChain with JavaScript for Web Developers" is your gateway to the future of web development. Join us to unlock the potential of AI in your web projects, making them smarter, more interactive, and more engaging for users.
From Todo App to B2B SaaS with Next.js and Clerk
React Summit US 2023React Summit US 2023
153 min
From Todo App to B2B SaaS with Next.js and Clerk
Top Content
WorkshopFree
Dev Agrawal
Dev Agrawal
If you’re like me, you probably have a million side-project ideas, some that could even make you money as a micro SaaS, or could turn out to be the next billion dollar startup. But how do you know which ones? How do you go from an idea into a functioning product that can be put into the hands of paying customers without quitting your job and sinking all of your time and investment into it? How can your solo side-projects compete with applications built by enormous teams and large enterprise companies?
Building rich SaaS products comes with technical challenges like infrastructure, scaling, availability, security, and complicated subsystems like auth and payments. This is why it’s often the already established tech giants who can reasonably build and operate products like that. However, a new generation of devtools are enabling us developers to easily build complete solutions that take advantage of the best cloud infrastructure available, and offer an experience that allows you to rapidly iterate on your ideas for a low cost of $0. They take all the technical challenges of building and operating software products away from you so that you only have to spend your time building the features that your users want, giving you a reasonable chance to compete against the market by staying incredibly agile and responsive to the needs of users.
In this 3 hour workshop you will start with a simple task management application built with React and Next.js and turn it into a scalable and fully functioning SaaS product by integrating a scalable database (PlanetScale), multi-tenant authentication (Clerk), and subscription based payments (Stripe). You will also learn how the principles of agile software development and domain driven design can help you build products quickly and cost-efficiently, and compete with existing solutions.
Building Pinia From Scratch
Vue.js Live 2024Vue.js Live 2024
70 min
Building Pinia From Scratch
Workshop
Eduardo San Martin Morote
Eduardo San Martin Morote
Let's dive into how Pinia works under the hood by building our own `defineStore()`. During this workshop we will cover some advanced Vue concepts like dependency Injection and effect scopes. It will give you a better understanding of Vue.js Composition API and Pinia. Requirements: experience building applications with Vue and its Composition API.
Mastering 3D Web Development with TresJS ecosystem: A Vue.js Workshop
Vue.js Live 2024Vue.js Live 2024
119 min
Mastering 3D Web Development with TresJS ecosystem: A Vue.js Workshop
Workshop
Alvaro Saburido
Alvaro Saburido
Introducing "Mastering 3D Web Development with TresJS," a specialized workshop crafted for Vue.js developers eager to explore the realm of 3D graphics within their web applications. TresJS, a powerful custom renderer for Vue, is specifically designed to work seamlessly with Vue's reactive system. This workshop offers a deep dive into integrating sophisticated 3D visualizations and interactive experiences directly into Vue applications, leveraging the unique strengths of both Vue and TresJS ecosystems.
This workshop is designed for Vue.js developers looking to expand their skill set into the third dimension, UI/UX designers interested in incorporating 3D elements into web applications, and front-end developers curious about the potential of 3D graphics in enhancing user experiences. You'll need to be familiar with Vue.js to benefit from this workshop fully.
What You Will Learn- Introduction to TresJS: Discover the fundamentals of TresJS and how it integrates with the Vue ecosystem to bring 3D graphics to life.- Creating 3D Scenes with Vue: Learn to construct intricate 3D scenes utilizing Vue components, enhancing your user interfaces with dynamic and immersive visuals.- Interactivity and Animation: Master the techniques to make your 3D scenes interactive, responding to user inputs for a captivating user experience.- Integrating with Vue Features: Explore advanced integration of TresJS with Vue’s reactivity, composables, and the Vuex store to manage state in 3D web applications.- Performance and Best Practices: Gain insights into optimizing your 3D scenes for performance and best practices to maintain smooth, responsive web applications.