Why I'm Working on Yarn

(This post is about Yarn, a new JS package manager that was announced today.)

I work with Node and npm packages almost every day, on Tilde's main app, Skylight, or on one of Ember's many packages.

Many have remarked upon how fast the npm registry has grown, and it's hard to imagine working on any of my packages without the npm ecosystem.

I've also worked on a couple of application-level package managers (Bundler for Ruby and Cargo for Rust), so it's no surprise that people have routinely asked me whether I'd consider writing a "bundler for node".

While it's something I considered idly from time to time, the truth is that for all of the complaints people have about the official client, it does a whole lot that people rely on, and the npm team has done a lot to improve it over the years. I genuinely respect their work, and believe that the hard work associated with maintaining a project the size and scope of the npm CLI client is vastly underappreciated.

So when I was first approached about helping with the Yarn project, I wasn't immediately sure I was interested. That changed quickly after talking with Sebastian McKenzie and James Kyle, two developers I have tremendous respect for, especially after working with them to integrate Babel into Ember.

There are a handful of "high order bits" that make me enthusiastic about the Yarn project.

First, it's no secret that major frameworks like Ember, Angular and React stress the limits of the official npm client. The team working on Yarn reached out to the major frameworks and made sure that Yarn would be a good fit for projects written using them.

In the month or so that I've worked on Yarn with a couple of dozen people from a bunch of companies, I was able to raise concerns and address them directly as an active contributor to the project. This makes me feel confident that, along with other contributors to Ember, I will be able to help shape Yarn into a project that effectively satisfies our needs.

Second (and relatedly), the Yarn project is set up as a community project, using a standard open source license, in its own GitHub organization, and set up to use the governance model that has worked effectively for Ember and Rust. This is something that I raised early with existing contributors to Yarn, and was very heartened by how enthusiastic everyone was about it.

This means that existing contributors and newer contributors work together to propose new features and other changes. In practice, I have seen this improve the decisions of the frequent contributors, and also provide a path for members of the community to write proposals, contribute code, and eventually become core contributors themselves. The exact details of Yarn governance are still being discussed on GitHub and we hope to finalize them soon.

Lastly, I believe in the core technical values of the project.

(It's not a coincidence that I write about the technical details last. I consider the technical details to be secondary to the way in which the project is set up to evolve. Technical mistakes can and will be corrected with a strong community, but not the other way around.)

I've written before that I consider predictability ("determinism") to be critical to projects like Yarn, especially in large applications.

From the get-go, the Yarn lockfile guarantees that repeatedly running yarn on the same repository results in the same packages.

This is true across time, across development machines, and when deploying applications to production. If I could have only one thing from Yarn, it would be this.

Second, Yarn attempts to have good performance, with a cold cache, but especially with a warm cache.

The Yarn that's shipping today already has good performance, but I think we can do even better.

Finally, Yarn makes security a core value.

Today, Yarn uses checksums to verify the integrity of every installed package before executing code. Interestingly, in addition to helping with security, checksums also help to avoid inadvertent errors caused by faulty caching or captive portals (something I have experienced repeatedly over the years, but which is devilishly tricky to track down without a full npm cache clean && rm -rf node_modules).

There are a number of additional steps we could take to further improve the security of Yarn, and making security a core value from the get-go makes me confident we will continue to work on them as we continue to develop Yarn.

I'm looking forward to working with the existing contributors to Yarn and many future contributors to make Yarn an awesome tool for the entire Node community.