Package management in the realm of Node.js and JavaScript has seen significant evolution over the past few years. Among the various package managers available, pnpm stands out due to its efficient use of disk space and speed. This article delves into the mechanisms that make pnpm a compelling choice for developers, especially when compared to npm and Yarn.
One of the notable features of pnpm is its use of hardlinks and symlinks. Unlike traditional methods where each file is copied into the node_modules directory, pnpm creates hardlinks to a content-addressable storage. This means that files do not consume extra disk space as they are mere references to the actual data stored elsewhere on the disk. This approach is particularly beneficial when managing large projects with numerous dependencies, as it significantly reduces the disk space required.
Transitioning from YARN to pnpm is a process that many developers may find themselves considering. The good news is that this migration is relatively straightforward. For single package repositories, the transition involves using the pnpm import command to convert YARN lock files to pnpm lock YAML files. Developers then update their CI scripts by replacing YARN commands with pnpm equivalents. In workspace settings, adjustments are needed in how package globs are listed, with pnpm utilizing a separate file for this purpose. Overall, the migration is often seamless, allowing pnpm to serve as a drop-in replacement in many scenarios.
pnpm distinguishes itself with its unique node module structure. Traditional package managers like npm and Yarn hoist sub-dependencies to the root of node_modules, which can lead to dependency conflicts and unexpected behavior. pnpm, however, ensures that only the direct dependencies of a project are placed at the root, while sub-dependencies remain isolated. This isolation prevents projects from inadvertently accessing undeclared dependencies, thereby maintaining a cleaner and more predictable package structure.
The speed of pnpm is another aspect that sets it apart. Unlike npm and Yarn, which handle dependency installation in stages, pnpm performs these tasks concurrently. It resolves, fetches, and writes dependencies in parallel, optimizing the overall installation process. This efficient handling of dependencies not only speeds up installations but also minimizes the time spent resolving potential conflicts, making pnpm a faster option for developers.
Monorepo setups, where multiple projects are housed within a single repository, present unique challenges in package management. pnpm addresses these challenges with its superior node module structure. It creates a single .pnpm folder at the root of the monorepo, and each project links only its direct dependencies. This setup prevents cross-project dependency access, which can lead to issues when projects are deployed independently. By maintaining clear boundaries between project dependencies, pnpm ensures that each project functions correctly both within and outside the monorepo.
Despite its advantages, pnpm is not without its challenges. Some tools and packages in the ecosystem do not yet support the symlink-based structure used by pnpm, which can lead to compatibility issues. However, pnpm provides a workaround by allowing developers to switch to a classic hoisting mode if needed, ensuring broader compatibility while still benefitting from pnpm's core efficiencies.
One of the common questions surrounding pnpm is why other package managers have not adopted similar approaches. The answer lies in the trade-offs involved. While pnpm's methods offer significant performance and space-saving benefits, they also introduce complexity in the package manager's codebase. This complexity can make it harder for new contributors to engage with the project, posing a challenge for broader adoption.
In addition to its primary features, pnpm offers additional conveniences, such as being bundled into an executable that can be used even on systems without Node.js pre-installed. This feature allows developers to manage Node.js installations and versions directly through pnpm, providing an alternative to tools like nvm or volta.
Ultimately, the choice of package manager depends on the specific needs of a project. Developers should consider the features and trade-offs of npm, Yarn, and pnpm to determine the best fit for their workflows. Each of these package managers is mature, widely used, and supported by a robust community, ensuring reliability and ongoing development.
pnpm represents a significant advancement in package management, offering solutions to common issues faced by developers. Its efficient use of disk space, speed, and robust handling of dependencies make it an attractive option for modern software development. By understanding these benefits and how to implement pnpm effectively, developers can enhance their project management and streamline their workflows.
Pnpm is a fast and disk space efficient package manager for JavaScript that aims to address issues found in npm versions 3 and 4. It utilizes a unique node module structure to ensure that dependencies are isolated and only direct dependencies are accessible to a project, reducing the risk of breaking changes.
Pnpm saves disk space by using a content addressable storage system, wherein files are saved based on a hash code derived from their content. This approach means if multiple projects use the same package, they will share the same files in the storage, significantly reducing disk space usage.
Pnpm is faster because it handles package installation stages separately for each dependency. This allows some packages to be resolved or fetched while others are still being fetched or written to node modules, enhancing the speed of operations.
Yes, pnpm provides excellent support for monorepos. It maintains an isolated module structure, ensuring that projects within a monorepo only have access to their own dependencies, which prevents accidental access to unrelated project dependencies.
Migrating from YARN to pnpm can be straightforward, especially for single package repositories. Pnpm provides an 'import' command that converts a YARN lock file to a pnpm lock file, making the transition smoother. For workspaces, adjustments might be needed for package listing due to differences in configuration files between YARN and pnpm.
Pnpm can handle multiple drives or filesystems by creating separate storage for each disk. This approach maintains efficiency on a single disk using hardlinks, but will resort to copying files across different disks, slightly reducing the space efficiency benefits in such scenarios.
Pnpm uses both symlinks and hardlinks to manage files. Symlinks are used to reference the real location of packages, aiding Node.js in dependency resolution. Hardlinks are used to link files from the content addressable store, saving disk space without creating duplicate files.
pnpm is a fast and efficient package manager that gained popularity in 2021 and is used by big tech companies like Microsoft and TikTok. It has a unique isolated node module structure that prevents package conflicts and ensures each project only has access to its own dependencies. pnpm also offers superior monorepo support with its node module structure. It solves the disk space usage issue by using a content addressable storage, reducing disk space consumption. pnpm is incredibly fast due to its installation process and deterministic node module structure. It also allows file linking using hardlinks instead of symlinks.
Hello, my name is Zoltan Kochan and today I'm going to talk about pnpm, which is a fast disk space efficient package manager. Let me introduce myself. I work remotely for BIT, a company that helps developers implement component-based development. Before that, I worked for Just Answer for nine years and have been developing and maintaining npm-pm since 2016. npm, yarn, and pnpm are the most popular JavaScript package managers. npm had issues in the past, but alternatives like yarn and pnpm were created. pnpm is an indie project that fixes the issues of npm v3 and is now supported by bit. It had a spike in popularity in 2021 and is used by big tech companies like Microsoft and TikTok. pnpm is unique because it only hoists the necessary dependencies to the root of node modules.
2. Isolated Node Module Structure with PMPM
But what will happen if a new version of Express comes out and cookie is not in its dependencies anymore? Your package will break. PMPM prevents such mistakes. With PMPM your code has only access to packages that are declared as dependencies of your project. In this example, Express is a direct dependency and KUKI is a dependency of Express. The real location of Express is inside the .pmpm folder. So nodejs will search for KUKI in the dedicated nodemodules folder which is at .pmpm express at 4.17.3/.nodemodules. Each package is in its own isolated folder with its own dependencies.
3. Superior Monorepo Support with QNPM
QNPM has superior monorepo support due to its node module structure. In a monorepo, all dependencies of all projects are hoisted to the root, allowing access to dependencies of other projects. This can cause issues when installed outside the monorepo. pnpm's isolated module structure ensures projects only have access to their own dependencies. In a monorepo, pnpm creates a single .pnpm folder in the root, linking projects to their direct dependencies.
4. npm's Approach and pnpm's Solution
npm is working on a setting to support a pnpm styled mod modules directory. Some tools and packages in the ecosystem are broken due to this hoistedmodmodules architecture. pnpm provides a solution by allowing the creation of classic modmodules without using symlinks.
5. Efficient Disk Space Usage with pnpm
pnpm solves the disk space usage issue by using a content addressable storage. Files are saved by a hash code, allowing for efficient storage. Every file in node modules is just a hardlink to the content addressable storage, reducing disk space consumption. Dependencies in new projects consume a lot less disk space than with npm or Yarn.
6. Why pmpm is so fast
Why is pmpm so fast? npm and Yarn install dependencies in stages, while pmpm runs the installation stages separately for each dependency. This makes pmpm incredibly faster. Some of these speed optimizations are possible thanks to the unique deterministic node module structure created by pmpm and the usage of the content addressable store. However, these optimizations make the code of the package managers harder to understand, so it's a tradeoff.
7. Pnpm Features and File Linking
Pnpm is bundled to an executable, allowing you to use it even without pre-installed Node.js. You can also use Pnpm to install and switch between different versions of Node.js. When choosing a package manager, consider the features of Pnpm, Yarn, and Npm. Visit the Pnpm website and follow Pnpmjs and my Twitter accounts for more information. Files from the content addressable store in Pnpm are linked using hardlinks, not just symlinks. A symlink is a reference to another place on the disk, while a hardlink is another file with its own location.
8. Hardlinks, Migration, and Multiple Drives
You can have many hardlinks on the disk at different places. For Node.js, it is like separate files. These files are pointing to the same location on the physical disk. These files do not consume additional extra space on your disk. So let's take some questions from the audience. How difficult is to migrate the codebase from YARN to pnpm, for example? It's... If it's just a single package repository, then it's really easy. What is the full form of pnpm? What does it stand for? This name was actually invented before me. The first maintainer of pnpm was Rico Stacrus from the Philippines. It is just performance npm. I wanted to ask a question. How does pnpm work on computers which might have multiple drives or multiple file systems? How does everything work on that? The drawback with hardlinks is that a hardlink can be only in scope of one disk. So you cannot have your ContentAddressable store on disk C and your project on disk D, if you are using Windows for instance.
This Talk discusses modern web debugging and the latest updates in Chrome DevTools. It highlights new features that help pinpoint issues quicker, improved file visibility and source mapping, and ignoring and configuring files. The Breakpoints panel in DevTools has been redesigned for easier access and management. The Talk also covers the challenges of debugging with source maps and the efforts to standardize the source map format. Lastly, it provides tips for improving productivity with DevTools and emphasizes the importance of reporting bugs and using source maps for debugging production code.
Today's Talk discusses the future of performance tooling, focusing on user-centric, actionable, and contextual approaches. The introduction highlights Adi Osmani's expertise in performance tools and his passion for DevTools features. The Talk explores the integration of user flows into DevTools and Lighthouse, enabling performance measurement and optimization. It also showcases the import/export feature for user flows and the collaboration potential with Lighthouse. The Talk further delves into the use of flows with other tools like web page test and Cypress, offering cross-browser testing capabilities. The actionable aspect emphasizes the importance of metrics like Interaction to Next Paint and Total Blocking Time, as well as the improvements in Lighthouse and performance debugging tools. Lastly, the Talk emphasizes the iterative nature of performance improvement and the user-centric, actionable, and contextual future of performance tooling.
Yarn is a package manager that focuses on stability, performance, and security. It offers unique features like plug and play installation, support for nonmodules, and the exec protocol. Yarn is committed to being a good citizen in the open-source community and contributes to fixing dependencies. It is part of the Node.js Loader's working group and advocates for Corepack. Yarn is still experimental but is improving its user experience and security features. Contributions are welcome, and switching to Yarn can improve performance in large projects.
The Talk discusses optimizing rendering of big tables using Flipper, a new version that is ten times faster with improved user interaction and richer data. It explores optimizing rendering with React, virtualization, filtering, sorting, and windowing techniques. The introduction of the Flipper Datasource packet simplifies handling updates, inserts, and removals. The performance of the Flipper data source package is excellent, even in a debug build of React, with minimal CPU usage. The Q&A session covers incremental sorting, dynamic row height, and the potential for two-dimensional virtualization in the future.
Here are some tips for better utilizing DevTools, including using the run command, customizing keyboard shortcuts, and emulating the focus effect. Learn how to inspect memory, use the network panel for more control over network requests, and take advantage of console utilities. Save frequently used code as snippets and use local overrides for easy editing. Optimize images by using a more optimized format like AVIF and track changes in the network panel to see the reduced data size.
This talk covers the latest features in Chrome DevTools, including network tab analysis, performance tab optimization, and user flows. It discusses optimizing HTTP requests with fetch priority to improve loading time. The performance tab provides insights on frame drops, long tasks, and the importance of minimizing total blocking time. The talk also highlights the optimization of page rendering and introduces user flows in Chrome DevTools.
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.
The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.
React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
TypeScript is not just types and interfaces. Join this workshop to master more advanced features of TypeScript that will make your code bullet-proof. We will cover conditional types and infer notation, template strings and how to map over union types and object/array properties. Each topic will be demonstrated on a sample application that was written with basic types or no types at all and we will together improve the code so you get more familiar with each feature and can bring this new knowledge directly into your projects. You will learn:- - What are conditional types and infer notation- What are template strings- How to map over union types and object/array properties.
Dive into the world of AI with our interactive workshop designed specifically for web developers. "Hands-On AI: Integrating LangChain with JavaScript for Web Developers" offers a unique opportunity to bridge the gap between AI and web development. Despite the prominence of Python in AI development, the vast potential of JavaScript remains largely untapped. This workshop aims to change that.Throughout this hands-on session, participants will learn how to leverage LangChain—a tool designed to make large language models more accessible and useful—to build dynamic AI agents directly within JavaScript environments. This approach opens up new possibilities for enhancing web applications with intelligent features, from automated customer support to content generation and beyond.We'll start with the basics of LangChain and AI models, ensuring a solid foundation even for those new to AI. From there, we'll dive into practical exercises that demonstrate how to integrate these technologies into real-world JavaScript projects. Participants will work through examples, facing and overcoming the challenges of making AI work seamlessly on the web.This workshop is more than just a learning experience; it's a chance to be at the forefront of an emerging field. By the end, attendees will not only have gained valuable skills but also created AI-enhanced features they can take back to their projects or workplaces.Whether you're a seasoned web developer curious about AI or looking to expand your skillset into new and exciting areas, "Hands-On AI: Integrating LangChain with JavaScript for Web Developers" is your gateway to the future of web development. Join us to unlock the potential of AI in your web projects, making them smarter, more interactive, and more engaging for users.
If you’re like me, you probably have a million side-project ideas, some that could even make you money as a micro SaaS, or could turn out to be the next billion dollar startup. But how do you know which ones? How do you go from an idea into a functioning product that can be put into the hands of paying customers without quitting your job and sinking all of your time and investment into it? How can your solo side-projects compete with applications built by enormous teams and large enterprise companies? Building rich SaaS products comes with technical challenges like infrastructure, scaling, availability, security, and complicated subsystems like auth and payments. This is why it’s often the already established tech giants who can reasonably build and operate products like that. However, a new generation of devtools are enabling us developers to easily build complete solutions that take advantage of the best cloud infrastructure available, and offer an experience that allows you to rapidly iterate on your ideas for a low cost of $0. They take all the technical challenges of building and operating software products away from you so that you only have to spend your time building the features that your users want, giving you a reasonable chance to compete against the market by staying incredibly agile and responsive to the needs of users. In this 3 hour workshop you will start with a simple task management application built with React and Next.js and turn it into a scalable and fully functioning SaaS product by integrating a scalable database (PlanetScale), multi-tenant authentication (Clerk), and subscription based payments (Stripe). You will also learn how the principles of agile software development and domain driven design can help you build products quickly and cost-efficiently, and compete with existing solutions.
Let's dive into how Pinia works under the hood by building our own `defineStore()`. During this workshop we will cover some advanced Vue concepts like dependency Injection and effect scopes. It will give you a better understanding of Vue.js Composition API and Pinia. Requirements: experience building applications with Vue and its Composition API.
Introducing "Mastering 3D Web Development with TresJS," a specialized workshop crafted for Vue.js developers eager to explore the realm of 3D graphics within their web applications. TresJS, a powerful custom renderer for Vue, is specifically designed to work seamlessly with Vue's reactive system. This workshop offers a deep dive into integrating sophisticated 3D visualizations and interactive experiences directly into Vue applications, leveraging the unique strengths of both Vue and TresJS ecosystems. This workshop is designed for Vue.js developers looking to expand their skill set into the third dimension, UI/UX designers interested in incorporating 3D elements into web applications, and front-end developers curious about the potential of 3D graphics in enhancing user experiences. You'll need to be familiar with Vue.js to benefit from this workshop fully. What You Will Learn- Introduction to TresJS: Discover the fundamentals of TresJS and how it integrates with the Vue ecosystem to bring 3D graphics to life.- Creating 3D Scenes with Vue: Learn to construct intricate 3D scenes utilizing Vue components, enhancing your user interfaces with dynamic and immersive visuals.- Interactivity and Animation: Master the techniques to make your 3D scenes interactive, responding to user inputs for a captivating user experience.- Integrating with Vue Features: Explore advanced integration of TresJS with Vue’s reactivity, composables, and the Vuex store to manage state in 3D web applications.- Performance and Best Practices: Gain insights into optimizing your 3D scenes for performance and best practices to maintain smooth, responsive web applications.
Comments