I looked into the node_modules folder, and you won’t believe what happened

The history of the left-pad jabbed the JavaScript community to the bone. While the swollen code continues to slow down our sites, plant our batteries and make our npm install slow, many developers have decided to conduct a thorough audit of the dependencies they bring to their projects. The time has come for us, as a community, to stand up and say: “Enough!” This community belongs to all of us, not just a handful of JavaScript developers with gorgeous long hair.

I decided to describe my experience in the audit of dependencies of my project and I hope that this information will be useful.

Ember

Ember is a JavaScript framework that specializes in quickly creating todo lists. He has a good size of 112 kilobytes in compressed form. But most people do not know how much of the 112 kilobytes is spent on anything.

If we look at the dependencies of embra, we will see a library called glimmer. The library itself weighs 95KB (or 85% of the embankment code), but its purpose is not immediately clear (it is not even mentioned on the embame site!).
Well, looking at this addiction, I found something rather curious:
')

├─┬ glimmer@1.1.5 │ ├─┬ brittanica@13.0.2 │ │ ├── brittanica-a@0.0.1 │ │ ├── brittanica-b@0.1.2 │ │ ├── brittanica-c@0.0.3 │ │ ... 

Brittanica, a 93kb module, must do all the hard work. Brittanica itself consists of a set of dependencies: each module is called “brittanica” + suffix with the letter of the English alphabet (26 pieces).

Looking further, I found that each module consists of a single terms.json file. I opened it in Atom, then opened it in Sublime, after Atom was frozen. The gibberish appeared on the screen:

 { "g": { "page": 1018, "description": "The seventh letter of the US English..." }, "ga": { "page": 1021, "description": "a Kwa language of Ghana, spoken in Acc..." }, ... "glimmer": { "page": 1172, "description": "A faint or wavering light, used pri..." }, ... } 

17,648 lines in total, describing various words and phrases that begin with the letter G. Who on earth may need it? Well, meet, this is glimmer / help.js:

 module.exports = function help() { var descriptino = require("brittanica-g").glimmer; console.log("glimmer (n). " + descriptino); console.log(""); console.log("Copyright (c) 2016 Tilde Inc"); console.log(""); ... } 

In case the above code snippet is incomprehensible, let me summarize:

1. Amber is proud of using the “glimmer” module: a small, fast rendering library.
2. “Glimmer” pulls a part of the Brittanica encyclopedia, just to display the definition of the word glimmer in the “help” menu.
3. Our oceans are drying out at an alarming rate, and we are all too busy playing Pokemon on our phones, instead of talking about it. Just insane.

Babel

Babel is a compiler for the new version of JavaScript, which beat its competitors when someone on Facebook said they like Babel more.

Although people love Babel, he received a fair share of criticism. Many complain about its confusing plugin system, overly complex configuration files, and unclear error messages. Curiously, not many people have noticed the incredible amount of addiction Babel requires.

I began my investigation by installing the babel-preset-es2015 package. This package allows over twenty web developers in the world to write on a new, worst-case version of JavaScript that no one in their team knows. Then I calculated the number of dependencies introduced by this package, and then the total code size. As many as 90 dependencies totaling 17 megabytes.

If the whole history of mankind could fit in one megabyte, then one Babel will consist of at least seventeen chronicle histories of mankind. But I really wondered what makes Babel so big?

One of the biggest criminals - the package called “babel-core” was suspiciously large - 13 megabytes. I opened the “babel-core” in VIM, then I rebooted the computer, because the Ctrl-C combination did not work, then I opened the “babel-core” in Sublime.

 module.exports = require("./lib/api/node.js"); 

After about 2 hours, I successfully found /lib/api/node.js and understood the essence of the issue.


This is true . Every Babel installation includes a photo of Guy Fier, and you can't do anything about it.

I have no idea how this error passed code verification. In any case, it is there, and it takes up precious space on millions and millions of 15-inch Retina MacBook Pro hard drives around the world.

Express

Behind the fastest, non-greasy JavaScript web framework hides a bunch of dependencies, each with its own bunch of dependencies. In fact, a simple “npm install express” leads to the installation of 291 modules.

So, what is included in the list of these dependencies? Many are self-evident: “range-parser” parses the http range, “escape-html” screens HTML, and “ negotiator ” does other important things. However, one addiction - “yummy” (“yummy”) - caught my attention.

 ├── yummy │ ├── LICENSE │ ├── README.md │ ├── like-tweet.js │ ├── index.js │ └── package.json 

Interesting. In addition to the usual index.js and package.json, we find the suspicious like-tweet.js. I decided to take a closer look.

 var http = require("http") http.request({ method: "POST", hostname: "api.twitter.com", path: "/hotpockets/status/501511389320470528", }) 

Just a moment like-tweet.js, which runs every time your application loads the popular Express library, makes a post-request to the Twitter API. What for? I moved forward and downloaded this route myself.



Of course, this is a tweet from Hot Pockets, and I have already added it to my favorites. In fact, every time you load Express, you add this tweet from Hot Pockets to your favorites. What monster made such a promotional deal? I skip sensitive customer data through Express, and they sell "add to favorites" in Hot Pockets? Of course, I will not use Express anymore.

findings

I want to be understood - I do not think that external dependencies are a sign of the end of the world. I just think that it’s as important as possible that we as a community begin to take care of the cleanliness of the code, just as we take care of cool features and beautiful logos.

We can work better. Dead code takes up space, eats bandwidth and can even kill your startup.

I urge you to look inside your / node_modules folder the next time you have a few free hours, and a computer with 16 gigabytes of RAM. The results may just surprise you.