Lunch Break Refactoring: Introduction to Submod Scripts

I think that project refactoring is a topic that is close to each developer. Often, we are faced with problems when the IDE and regular expressions are no longer enough for us, and then means such as those described in this post come to the rescue. Codemod scripts are a very powerful tool. After its development, it will become clear that your refactoring will never be the same. Therefore, I translated this post for our harabrablog. I wish you a pleasant reading.

Maintaining the code base can turn into a headache for any developer, especially when it comes to JavaScript. In the conditions of constantly changing standards, syntax and critical changes of third-party packages, it is very difficult to maintain such code.

In recent years, JavaScript has changed beyond recognition. The development of this language has led to the fact that even the simplest task of declaring variables has been changed. In ES6, there were let and const , switch functions and many other innovations, each of which benefits developers.

With the creation and maintenance of operational code designed to withstand the test of time, the burden on developers is increasing. From this post, you will learn how to automate tasks for large-scale code refactoring using Codemod scripts and the jscodeshift tool, which will allow you, for example, to easily update your code to use new language features.

Codemod

Codemod is a tool developed by Facebook to refactor large code bases. It allows the developer to reorganize a large amount of code in a short period of time. For small refactoring tasks like renaming a class or variable, the developer uses the IDE, such changes usually affect only one file. The next tool for refactoring is global search and replace. Often it can work using complex regular expressions. But this method is not suitable for all cases.

Codemod is written in Python, it takes a number of parameters, including expressions for searching and replacing.

 codemod -m -d /code/myAwesomeSite/pages --extensions php,html \ '<font *color="?(.*?)"?>(.*?)</font>' \ '<span style="color: \1;">\2</span>' 

In the example above, we replace <font> with <span> using the built-in style to specify the color. The first two parameters are flags indicating the need to search for multiple matches (-m), and the directory to start processing ( -d /code/myAwesomeSite/pages ). We can also limit the extensions being processed ( -extensions php , html ). We then provide expressions for search and replace. If the replacement expression is not specified, you will be prompted to enter it at run time. The tool works fine, but it is very similar to existing search and replace tools using regular expressions.

jscodeshift

jscodeshift is next in the refactoring toolbox. It was also developed by Facebook and is intended for processing several files by the Codemod script. Being a Node.js module, jscodeshift provides a simple and convenient API, and "under the hood" uses Recast , which is an AST-to-AST ( Abstract Syntax Tree ) conversion tool.

Recast

Recast is a Node.js module that provides an interface for parsing and regenerating JavaScript code. It can analyze the code as a string and generate objects from it that correspond to the AST structure. This allows developers to check the code for templates such as function declarations.

 var recast = require("recast"); var code = [ "function add(a, b) {", " return a + b", "}" ].join("\n"); var ast = recast.parse(code); console.log(ast); //output { "program": { "type": "Program", "body": [ { "type": "FunctionDeclaration", "id": { "type": "Identifier", "name": "add", "loc": { "start": { "line": 1, "column": 9 }, "end": { "line": 1, "column": 12 }, "lines": {}, "indent": 0 } }, ........... 

As you can see from the example, we pass a line of code to a function that adds two numbers. When we parse a string and print out the resulting object, we can see the AST: we see FunctionDeclaration , the name of the function, etc. Since this is just a JavaScript object, we can change it as we like. You can then call the print function to return the updated line of code.

AST (Abstract Syntax Tree)

As mentioned earlier, Recast builds an AST from a line of code. AST is a tree view of the abstract syntax of source code. Each tree node is a construction in the source code. ASTExplorer is an online tool that will help you understand and understand your code tree.

With ASTExplorer, you can view AST simple sample code. We declare a constant named foo and assign it the string value 'bar' .

 const foo = 'bar'; 

This corresponds to the following AST:

In the body array, there is a VariableDeclaration branch that contains our constant. All VariableDeclarations have an id attribute that contains important information (name, etc.). If we were to create a Codemod script to rename all instances of foo, we would use this attribute and go through all instances to change the name.

Installation and use

Using the tools and techniques discussed above, we can take full advantage of jscodeshift. Since, as we know, it is a Node.js module, you can install it for a project or globally.

 npm install -g jscodeshift 

After installation, we can use the existing Codemod-scripts in conjunction with jscodeshift . You need to provide some parameters that tell jscodeshift what we want to achieve. The basic syntax is a jscodeshift call with the path to the file or files to be converted. An important parameter is the location of the conversion script (-t) : it can be a local file or the URL of the Codemod script file. By default, jscodeshift searches for the transform script in the transform.js file in the current directory.

Other useful options are a test run (-d), which will apply the conversion, but will not update the files, and the -v option, which will output all the information about the conversion process. Transformation scripts are Codemod scripts, simple JavaScript modules that export a function. This function takes the following parameters:

• fileInfo
• api
• options

The fileInfo parameter stores all information about the current file, including the path and source. The api parameter is an object that provides access to jscodeshift helper functions, such as findVariableDeclarators and renameTo . Finally, the parameter is the options that allow you to pass parameters from the command line to the Codemod. For example, if we want to add a version of the code to all files, then we can pass it through the command line parameters jscodeshift -t myTransforms fileA fileB --codeVersion = 1.2 . In this case, the options parameter will contain {codeVersion: '1.2'} .

Inside the function that we export, you need to return the converted code as a string. For example, if we have a line of code const foo = 'bar', and we want to convert it, replacing const foo with const bar , our code will look like this:

 export default function transformer(file, api) { const j = api.jscodeshift; return j(file.source) .find(j.Identifier) .forEach(path => { j(path).replaceWith( j.identifier('bar') ); }) .toSource(); } 

Here we combine a number of functions into a chain of calls and at the end call toSource() to generate a converted line of code.

When returning the code, you must follow some rules. Returning a string other than the input is considered a successful conversion. If the string is the same as the input, then the conversion is considered unsuccessful, and if nothing is returned, then it is not needed. jscodeshift uses these results when processing conversion statistics.

Existing Codemod Scripts

In most cases, developers do not need to write their own code - many typical refactoring actions have already been turned into Codemod scripts. For example, js-codemod no-vars , which convert all var instances to let or const depending on the use of the variable (in let - if the variable is changed later, to const - if the variable will never be changed).

js-codemod template-literals replaces string concatenation with pattern strings, for example:

 const sayHello = 'Hi my name is ' + name; //after transform const sayHello = `Hi my name is ${name}`; 

How to write codemod scripts

We can take the no-vars script mentioned above and break the code to see how the complex Codemod script works.

 const updatedAnything = root.find(j.VariableDeclaration).filter( dec => dec.value.kind === 'var' ).filter(declaration => { return declaration.value.declarations.every(declarator => { return !isTruelyVar(declaration, declarator); }); }).forEach(declaration => { const forLoopWithoutInit = isForLoopDeclarationWithoutInit(declaration); if ( declaration.value.declarations.some(declarator => { return (!declarator.init && !forLoopWithoutInit) || isMutated(declaration, declarator); }) ) { declaration.value.kind = 'let'; } else { declaration.value.kind = 'const'; } }).size() !== 0; return updatedAnything ? root.toSource() : null; 

This code is the core of the no-vars script. First, the filter is run on all VariableDeclaration variables, including var , let and const , and returns only var declarations, which are passed to the second filter, which calls the user function isTruelyVar . It is used to determine the nature of a var (for example, var inside a closure, or is declared twice, or a function declaration that can be raised) and will show whether it is safe to do the conversion of this var. Each var returned by the isTruelyVar filter is processed in a foreach loop.

Inside the loop, check if var is inside the loop, for example:

 for(var i = 0; i < 10; i++) { doSomething(); } 

To determine if var is inside a loop, you can check the parent type.

 const isForLoopDeclarationWithoutInit = declaration => { const parentType = declaration.parentPath.value.type; return parentType === 'ForOfStatement' || parentType === 'ForInStatement'; }; 

If var is inside the loop and does not change, then it can be replaced by const . Validation can be performed by filtering the var AssignmentExpression and UpdateExpression. AssignmentExpression UpdateExpression. AssignmentExpression will show where and when var was initialized, for example:

 var foo = 'bar'; UpdateExpression ,    var  , : var foo = 'bar'; foo = 'Foo Bar'; //Updated 

If var is inside a loop with a change, let used, since it may change after creating an instance. The last line in the script checks if something has been changed, and if the answer is positive, a new source code for the file is returned. Otherwise, a null returned, indicating that the data was not processed. The full code for this Codemod script can be found here .

The Facebook team also added several Codemod scripts to update the React syntax and process changes in the React API. For example, react-codemod sort-comp , which sorts the methods of the React life cycle to conform to the ESlint sort-comp rule .

The latest popular React Codemod script is React-PropTypes-to-prop-types , which helps to cope with the recent React change, which requires developers to install prop-types to continue using PropTypes in React version 16 components. This is a great example of using Codemod scripts . The method of using PropTypes not immortalized in stone.

All the following examples are true.

Import the React and access the PropTypes from the default import:

 import React from 'react'; class HelloWorld extends React.Component { static propTypes = { name: React.PropTypes.string, } ..... 

Import the React and do the named import for PropTypes:

 import React, { PropTypes, Component } from 'react'; class HelloWorld extends Component { static propTypes = { name: PropTypes.string, } ..... 

And another option for the stateless component:

 import React, { PropTypes } from 'react'; const HelloWorld = ({name}) => { ..... } HelloWorld.propTypes = { name: PropTypes.string }; 

Having three ways to implement the same solution makes it difficult to use a regular expression for search and replace. If our code had all three options, we could easily switch to the new PropTypes pattern by running:

 jscodeshift src/ -t transforms/proptypes.js 

In this example, we took the PropTypes Codemod script from the react-codemod and added it to the transforms directory in our project. The script will add import PropTypes from 'prop-types '; for each file and replace all React.PropTypes with PropTypes .

findings

Facebook began to implement code support, allowing developers to adapt to its ever-changing API and practical approaches to working with code. Javascript Fatigue has become a big problem, and, as we have seen, tools that help in the process of updating existing code contribute a lot to its solution.

In the server-side development world, programmers regularly create migration scripts to keep databases up-to-date and keep users up-to-date on their latest versions. The creators of JavaScript libraries could provide Codemod scripts as such migration scripts so that when new versions with critical changes are released, you can easily process your code for updating.

Having a Codemod script run automatically during installation or upgrade can speed up the process and increase consumer confidence. In addition, the inclusion of such a script in the release process would not only be beneficial for consumers, but also reduce the costs of the accompanying updates examples and guides.

In this post, we looked at the nature of the Cdemod scripts and jscodeshift and how quickly they can update complex code. Starting with Codemod and moving to tools such as ASTExplorer and jscodeshift , you can learn how to create Codemod scripts according to your needs. And the presence of a wide range of ready-made modules allows developers to actively promote the technology to the masses.

Translator’s note: there was an interesting talk on JSConfEU on this topic.