Otter Thoughts
A Guide to the Software Industry
Menu

The Tyranny of Dependencies October 15, 2017

I once worked on a Node.js project with over 800 dependencies. Months of labor was wasted keeping up with API changes. Deployments were unexpectedly delayed by a week or more. A different project with hundreds of dependencies had its Node version pinned to an exact bugfix revision. We couldn’t upgrade even though there were known security holes and a serious SSL bug in that version. More dependencies, more problems.

Some dependencies are unavoidable. Never roll your own crypto code. Don’t write your own database or compression libraries. Where to draw the line can be hard to tell. A GIF loader? A JSON parser? It depends on your goals and what the options are. Does the potential dependency have a huge tree of its own dependencies? Is it made by some flaky group with a history of backwards-incompatible changes? Is it a glitchy piece of shit, at least moreso than your version would be? Does it impose requirements on your build chain? Does it have an acceptable license? Do you have the time and skill to implement your own? Would a home made version have sufficient features?

Will you end up having to write a custom implementation in the future anyway?

Don’t assume 3rd party libraries are more stable or higher quality. Just take a look at the monstrosity that is KDE these days. About a year ago I ran into a peculiar problem when updating my Gentoo desktop. I keep certain bloated software packages banned from my system through the package manager. One of them is Ruby. When trying to update a the kdewebkit module, portage threw an error because Ruby and a hundred or so of it’s dependencies got pulled in to the dependency graph. Some pig-headed developer had decided to use a Ruby-based automated testing framework for C++ based kdewebkit. Never mind the questionable utility, the automated tests weren’t even run during install. Effectively, the “simple” testing framework required a hundred packages, several gigabytes of space, and who knows how many lines of source code, all of which had absolutely no use on my system. Who’s going to maintain that? What happens when any of that breaks? What about when the Ruby community decides the kitchen sink is just not enough, as is their habit? (I masked that version of kdewebkit for the time being and emerged on. Some time later common sense prevailed and the Ruby dependency was dropped. Hail Gentoo.)

In 2016 a dev named Azer Koçulu inadvertently broke the internet when he removed 250 of his Node.js modukes from the npm repository after some jerkwad app startup’s lawyers started hassling him over one innocently named “kik”. One of these modules was “left-pad”, which add spaces to the left of some text. It’s a trivial task consuming only a eleven lines. No one should ever add an external dependency for that in Node. Turns out quite a few major projects were dependent on it. Companies everywhere large and small were left in the lurch, unable to deploy until they fixed it. The problem was so bad that the CEO of npm reinstated the left-pad module without it’s author’s permission.

You may notice that some of my projects brag about having few to no dependencies. I don’t have time to waste on keeping up with changes, reading bad documentation, or rooting out bugs in someone else’s code.  It’s faster to write easy things myself from scratch. Changes only happen when and how I make them happen. Users of my libraries only have to worry about me making a change, not dozens or hundreds of people in the entire dependency tree.

People today have a tendency to be shortsighted and impatient. Code written in that manner becomes unmanageable very quickly. It often only takes a few months before the continuing costs outweigh the perceived upfront benefits. Between myopic managers and deficient developers commercial projects tend to accumulate excessive numbers of dependencies. Fight against this whenever possible. The best option is to just not ask; roll your own code for simple things and pretend that left-pad library never existed. Remember, you will be the one under the gun battling a Babel version conflict on Friday night trying to deploy a patch for that critical production bug related to an unannounced API change in a dependency of one of your dependencies.

 

Categories: Uncategorized

Leave a Reply