[bookmark] Tools for testing javascript projects

The author of the material, the translation of which we publish today, an employee of Welldone Software , says that if you briefly talk about tools for testing JavaScript projects, it is recommended to use Jest for modular and integration testing, and TestCafe for user interface tests. However, each specific project may need something special. The best way to find exactly what you need is to take a few tools that seem to fit and test them in action. Experiments will tell you what it is worth staying at.



We present to your attention a review of the most widely used testing tools for JS projects, which should be noted in 2018.

General Purpose Testing Tools

▍Jsdom

Jsdom is a JavaScript implementation of the WHATWG DOM and HTML standards. In other words, jsdom mimics the browser’s environment without performing anything other than the usual JS.
')
In such an environment, tests can be performed very quickly. The minus jsdom is that not everything can be simulated outside the real browser (for example, this approach will not take screenshots), so the use of this tool limits the testing capabilities.

It is worth mentioning that the JS community is working intensively on jsdom and is improving this tool. The capabilities of its current version are very close to the current browser.

▍Istanbul

The Istanbul tool allows you to analyze code coverage with unit tests. He displays detailed reports, focusing on which you can get an accurate idea of ​​what has not yet been tested in the project, and estimate the amount of work needed to improve the situation.

▍Karma

Karma allows you to run tests in the browser and in environments that resemble browsers, including jsdom.
Karma supports a testing server with a special web page, in the environment of which tests will be performed. This page can be opened in a variety of browsers. In addition, this means the possibility of remote testing using services like BrowserStack .

▍Chai

Chai is the most popular assertion making library.

NexUnexpected

Unexpected is a library for creating statements, the syntax of which is slightly different from Chai. In addition, this library is extensible, allowing you to create more advanced statements. In particular, we are talking about the use of libraries based on unexpected, for example, such as unexpected-react , details of which can be found here .

▍Sinon.JS

Sinon.JS is a powerful autonomous system that provides the ability to test JavaScript projects using so-called spyware (spy), stubs and imitations (mock). This system can work with any unit testing frameworks.

▍Testdouble.js

Testdouble.js is a less popular library that performs the same functions as Sinon, while its developers say that it solves similar problems better than Sinon. It is distinguished by a set of features, some features of the architecture and an approach to testing, which can make it useful in many situations. Details about testdouble can be read here , here and here .

▍Wallaby

Wallaby is another tool worth mentioning. It is not free, but many users believe that it is worth the money that they ask for it. Wallaby works in the IDE (all major IDEs are supported) and performs tests corresponding to code changes. Data on test results are displayed in real time in the same place where the code is located.


Wallaby

▍Cucumber

Cucumber helps developers with writing tests in BDD. Tests are divided between the acceptance criteria files prepared using the Gherkin syntax and the test files themselves that correspond to them. Tests can be written in various languages ​​that the framework supports, including JS.

Here is an example of a file like-article.feature :

 Feature: A reader can share an article to social networks As a reader I want to share articles So that I can notify my friends about an article I liked Scenario: An article was opened   Given I'm inside an article   When I share the article   Then the article should change to a "shared" state 

Here is an example of a file like-article.steps.js :

 module.exports = function() { this.Given(/^I'm inside an article$/, function(callback) {   //     }) this.When(/^I share the article$/, function(callback) {   //     })   this.Then(/^the article should change to a "shared" state$/, function(callback) {       //     }) } 

Many commands find this syntax more convenient than TDD.

Modular and integration testing frameworks

The first thing that is probably worth choosing when searching for suitable testing tools is the framework and libraries to support it. It is recommended to use what the selected framework provides as long as there is no need for unique tools. Here are some general guidelines:

• If you are looking for a certain starting point, or you need a fast framework for a large project, choose Jest.
  
• If you are interested in the flexibility and extensibility of the configuration - take a look at Mocha.
  
• If you want simplicity, try Ava.
  
• If you need control over low-level testing mechanisms - pay attention to the tape.
  

Here is a brief overview of the most famous tools in this category.

▍Mocha

Mocha is currently the most widely used library. Unlike the Jasmine library, which we will discuss later, this library uses third-party assertion-building tools and external tools for creating imitations and spyware functions (usually Enzyme and Chai). This means some additional difficulties in setting up Mocha, and the fact that the resulting functionality will be divided between different libraries, but this framework is more flexible and expandable compared to Jasmine.

For example, if you need special assertion logic , you can fork Chai and replace only Chai in your toolbox with your own library for creating assertions. This can be done in Jasmine, but in Mocha this change will be more obvious.

Here are some features of Mocha that you should pay attention to:

• Community. Community forces have developed many plugins and extensions for use in unique test scenarios.
  
• Extensibility Plugins, extensions, and libraries that can be used with Mocha, such as Sinon, include features not found in Jasmine.
  
• Global variables Mocha, by default, creates test structures in global form. This does not apply to claims, imitations, spy functions, which distinguishes Mocha from Jasmine. To some, such heterogeneity of global objects seems illogical.
  

▍Jest

Jest is a testing framework developed by Facebook. It is based on Jasmine. As of today, Facebook has redesigned most of its functionality and has created many new features based on it.

It is worth noting that after analyzing the huge amount of materials on Jest, we can conclude that many developers are impressed with the speed and convenience of this framework. Among the features of Jest are the following:

• Performance. The first thing to say is that the Jest framework is recognized as the fastest in application to large projects with many test files due to the implementation of an intelligent parallel testing mechanism. We can confirm this by our own experience, besides, here , here , here and here - materials supporting our conclusions.
  
• User interface. The jest interface is clear and easy to use.
  
• Having everything you need to get started. The jest comes with assertions, spyware, and imitation tools that are equivalent to separate libraries, like Sinon, that perform the same functions. If the standard features do not suit you and you need something special, you can use third-party libraries.
  
• Global variables As in the case of Jasmine, Jest, by default, creates global test variables, so they do not need to be explicitly connected. This feature can be perceived as a disadvantage, because such an approach harms the flexibility of the tests and their ability to manage them, but in most cases it just simplifies the work:
  
  

   // "describe"      //      : // import { describe } from 'jest' // import { describe } from 'jasmine' describe('calculator', function() { ... }) 

• Testing with pictures. There is a jest-snapshot library that is developed by Facebook. It can be used to organize testing using snapshots (snapshot testing) in almost any other framework, for example, through a suitable plug-in.
  
• Improved module imitation. Jest simplifies the imitation of "heavy" modules, which allows to improve the speed of testing. For example, using Jest, you can create a “stub” for a service that performs a network request, which gives a much higher speed compared to the actual execution of the request.
  
• Analysis of code coverage tests. Jest has powerful and fast embedded test analysis code that is based on Istanbul .
  
• Reliability. Although Jest is a relatively young library, during 2017, a lot of work was done on its stability, and now it can be considered reliable. Now, for example, all major IDEs support it.
  
• Development. During project development, tests are performed very quickly due to the fact that the system monitors file changes and does not perform unnecessary actions.
  

▍Jasmine

Jasmine is the testing framework on which Jest is based. If there is a Jest - who might need Jasmine? The thing is that Jasmine appeared earlier than Jest, there are a huge number of publications on it, many tools have been created for it.

In addition, the creators of Angular still advise using Jasmine, not Jest, although Jest is great for testing Angular projects, and many use it for this. Here are the main features of Jasmine:

• Having everything you need to get started. Jasmine includes everything you need to start testing.
  
• Global variables In Jasmine, all important testing tools are available in the global scope.
  
• Community. Jasmine has existed since 2009, during which time many publications have been published on this framework. In addition, many Jasmine based tools have been created.
  
• Angular support. All versions of Angular support Jasmine, and it is this framework that recommends the official Angular documentation .
  

▍Ava

Ava is a minimalist test library that supports parallel test execution. Here are its main characteristics:

• Having everything you need to get started. Ava comes with everything you need to start testing (in addition to tools for creating spyware and imitations that are easy to integrate into Ava). Tests on Ava can be performed in the Node.js environment. Here the following syntax is used to form the structure of the test and build statements:
  
  

   import test from 'ava' test('arrays are equal', t => { t.deepEqual([1, 2], [1, 2]) }) 

• Global variables As you can see from the above code fragment, the library does not create global variables, that is, the developer can better control what is happening.
  
• Simplicity. Ava is distinguished by a simple structure and a simple statement model. There is no overly complex API, but it supports many advanced features.
  
• Development. Ava is convenient to use in the development process, as the system allows you to quickly test the modified code ...
  
• Speed. Tests are performed in parallel as separate Node.js processes.
  
• Testing with pictures. This feature is supported as part of the framework.
  

▍Tape

Tape is the simplest testing framework considered here, with a small and clear API. This is a regular JS file that works in Node.js. Here are the main characteristics of the tape:

• Simplicity. The minimalist structure of tests and statements, the absence of a complex API. Everything is arranged even easier than in Ava.
  
• Global variables Global variables are not created, it gives the opportunity to better control the test.
  
• Lack of general condition. Tape does not welcome the use of functions like beforeEach , which reflects the desire for modularity tests and to ensure maximum control over the testing process.
  
• No need for command line interface. The tape library is capable of working wherever JS can run.
  

User Interface Testing Tools

To begin with, we note that here and here you can find excellent materials about service providers whose powers can be used to perform cross-platform and cross-browser testing.

Existing tools for testing user interfaces are very different from each other, so before you choose what you will test your interfaces, we recommend experimenting. If you briefly express the general recommendations for those who choose the environment for testing user interfaces, we can say the following:

• If you are looking for a simple and reliable universal tool, or some starting point in the search for a suitable framework - try TestCafe.
  
• If you want to keep up with the times and enjoy the support of the developer community - take a look at WebdriverIO.
  
• If you are not interested in cross-browser support, use Puppeteer.
  
• If your application does not have confusing user interaction scripts and complex interfaces, such as pages, full forms and navigation elements, use cross-browser tools without a user interface like Casper.
  

▍Selenium

Selenium allows you to automate the interaction with the browser, simulating user actions. This framework is not specifically designed for tests, it can control the browser for various purposes, providing a server that simulates user behavior in the browser using software mechanisms.

The possibilities of Selenium can be controlled in various ways using many programming languages, and the use of some tools allows you to work with Selenium without any programming.

We are, however, interested in the configuration in which the Selenium server is managed using Selenium WebDriver , which serves as a layer of interaction between Node.js and the server that controls the browser. The scheme of interaction of various parts of the system can be described as:

 Node.js <=> WebDriver <=> Selenium Server <=> FF/Chrome/IE/Safari 

WebDriver can be imported into the selected framework for testing. Tests can be written using browser management features:

 describe('login form', () => { before(() => {   return driver.navigate().to('http://path.to.test.app/') }) it('autocompletes the name field', () => {   driver     .findElement(By.css('.autocomplete'))     .sendKeys('John')     driver.wait(until.elementLocated(By.css('.suggestion')))     driver.findElement(By.css('.suggestion')).click()     return driver     .findElement(By.css('.autocomplete'))     .getAttribute('value')     .then(inputValue => {       expect(inputValue).to.equal('John Doe')     }) }) after(() => {   return driver.quit() }) }) 

By the way, the WebDriver itself may be quite enough to run the tests. There are recommendations , the authors of which advise using WebDriver in its original form.

However, one can not ignore the fact that there are libraries that extend the capabilities of WebDriver, represented either by its forks or wrappers for it.

The drawbacks of the wrapper libraries are that they complicate the system. The appearance of forks is bad, that the efforts spent on them could well be invested in the development of the WebDriver itself.

However, if we talk about Selenium, many people prefer not to use it directly. Take a look at some libraries that use the features of Selenium.

Appium

Appium provides an API similar to the Selenium API. This framework is designed to organize testing of mobile projects using the following tools:

• iOS 9.3+. Apple's XCUITest .
  
• Until iOS 9.3. Apple UIAutomation .
  
• Android 4.2+. UiAutomator / UiAutomator2 from Google.
  
• Android 2.3+. Google Instrumentation . (Instrumentation support is implemented via Selendroid ).
  
• Windows Phone. WinAppDriver from Microsoft.
  

So, if you use tools based on Selenium, you can use Appium to test mobile projects.

Protractor

Protractor is a wrapper library for Selenium that focuses on Angular. Here are the main features of this library:

• Angular support. Protractor is focused on Angular, although this library can be successfully used with other JS frameworks. Official Angular documentation recommends this particular library.
  
• Bug reports. There is a convenient mechanism for creating reports.
  
• Community and technology. TypeScript support is available in Protractor. The library is developing a huge team of Angular.
  

WebdriverIO

WebdriverIO offers its own approach to using the functionality of Selenium WebDriver. This library is for Node.js. Among the features of this project are the following:

• Syntax. WebDriverIO has very simple and readable syntaxes.
  
• Flexibility. WebDriverIO is a flexible and extensible library.
  
• Community. The library has a good support and enthusiastic community of developers, which has created plug-ins and extensions to increase the use of WebDriverIO.
  

Nightwatch

Nightwatch also has a different approach to working with Selenium WebDriver. This tool provides its own testing framework, which has a server, assertion tools, and various support tools. Here are the main features of Nightwatch:

• Framework Nightwatch can be used with other frameworks, as well as a standalone tool, which can be especially useful if you plan to perform functional tests with its means, without using other technologies.
  
• Syntax. Nightwatch has a very simple and readable syntax.
  
• Support and technology. There is no support for TypeScript. It should be noted that there is a feeling that this library enjoys somewhat less support than others.
  

▍TestCafe

TestCafe is a great alternative to tools based on Selenium. The code of this library was opened at the end of 2016. It should be noted that there is still a paid version of TestCafe, which offers testing tools that do not require programming. For example, these are tools for recording tests. Within the paid version is offered and user support. It should be noted that there are many outdated publications that came out before the opening of the library code and consider the closed code to be its shortcoming.

TestCafe is embedded in the pages in the form of a JS-script, it does not control the browser, as Selenium does. This allows TestCafe to run in any browsers, including mobile, and have full control over what happens in JavaScript.

The TestCafe library is written in JavaScript and focused on testing. It is currently developing rapidly, although it is already considered stable and having a sufficient number of possibilities. Consider its main features:

• Quick setup of the working environment. Perhaps we can say that the special configuration of the working environment for the use of TestCafe is not required. It is enough to open any browser and start testing.
  
• Support for various browsers, support for test automation platforms. TestCafe supports many browsers and devices and can be used on SourceLabs or BrowserStack platforms , which give access to various devices and browsers. In particular, we are talking about running tests in Chrome and Firefox without a user interface, which we will discuss later.
  
• Parallel testing. TestCafe can run tests in multiple instances of the browser at the same time. Such an approach can significantly reduce the testing time.
  
• Bug reports. TestCafe contains convenient tools for generating error reports.
  
• Ecosystem. TestCafe uses its own test structure . This can be very convenient, especially considering that user interface tests are usually performed separately from other tests, but not everyone likes this.
  
  

   import { Selector } from 'testcafe'; fixture `Getting Started`   .page `https://devexpress.imtqy.com/testcafe/example` //    test('My first test', async t => {   await t       .typeText('#developer-name', 'John Smith')       .click('#submit-button')       .expect(Selector('#article-header').innerText)       .eql('Thank you, John Smith!') }) 

▍Cypress

Cypress is a direct competitor to TestCafe. This framework works in almost the same way by incorporating tests into the code of the pages, but it tries to do it in a more modern, flexible and convenient way. Cypress is newer, this framework has recently moved from the closed beta stage to the public beta version (in October 2017), but many already use it. Here are the main features of Cypress:

• No cross browser support. Currently only Chrome is supported (and is not a version without a user interface). Development in the direction of cross-browser support is underway .
  
• Lack of advanced testing capabilities. Here, in comparison with TestCafe, there are not enough opportunities for parallel testing and some additional tools, but the appearance of all this is in the plans of developers.
  
• Documentation. Cypress has a well written and understandable documentation.
  
• Debugging Tools There are convenient tools for debugging and logging.
  
• Mocha. Mocha . , , Mocha, , .
  
  

   describe('My First Cypress Test', function() { it("Gets, types and asserts", function() {   cy.visit('https://example.cypress.io')   cy.contains('type').click()   //     URL,   '/commands/actions'   cy.url().should('include', '/commands/actions')   //   ,        .   cy.get('.action-email')     .type('fake@email.com')     .should('have.value', 'fake@email.com') }) }) 

▍Puppeteer

Puppeteer — Node.js, Google. Node.js API Chrome .

— Chrome v59+, --headless . , API , Puppeteer — , , JS-, Google .

, Firefox , 2017-, .

, Chrome Firefox . , TestCafe Karma.

Puppeteer :

• . Puppeteer — , , .
  
• Performance. Puppeteer , Chrome, , , PhantomJS, WebKit.
  
• . Chrome (, , Puppeteer) , Flash. , .
  

▍PhantomJS

PhantomJS Chromium , Chrome.

Puppeteer PhantomJS, , , PhantomJS, 2017, , - .

PhantomJS, Puppeteer? :

1. , , .
   
2. PhantomJS CasperJS. , PhantomJS.
   
3. PhantomJS WebKit, Chrome.
   
4. PhantomJS Flash, Chrome — .
   

▍Nightmare

Nightmare — , .

Nightmare Electron , PhantomJS, , Chromium . , Electron, , - JavaScript, HTML CSS.

, , Nightmare Chrome . Nightmare PhantomJS.

Nightmare:

 yield Nightmare() .goto('http://yahoo.com') .type('input[title="Search"]', 'github nightmare') .click('.searchsubmit') 

PhantomJS:

 phantom.create(function (ph) { ph.createPage(function (page) {   page.open('http://yahoo.com', function (status) {     page.evaluate(       function () {         var el = document.querySelector('input[title="Search"]')         el.value = 'github nightmare'       },       function (result) {         page.evaluate(           function () {             var el = document.querySelector('.searchsubmit')             var event = document.createEvent('MouseEvent')             event.initEvent('click', true, false)             el.dispatchEvent(event)           },           function (result) {             ph.exit()           }         ) // page.evaluate       }     ) // page.evaluate   }) // page.open }) // ph.createPage }) // phantom.create 

▍Casper

Casper — , PhantomJS SlimerJS ( , Phantom, Firefox Gecko). Casper -, , Phantom Slimer.

Slimer , . 2017- -, 1.0.0-beta.1, Firefox . Slimer.

, , , Casper PhantomJS Puppeteer Chrome, Firefox. , .

▍CodeceptJS

CodeceptJS , CucumberJS, API. , , .

CodeceptJS:

 Scenario('login with generated password', async (I) => { I.fillField('email', 'miles@davis.com'); I.click('Generate Password'); const password = await I.grabTextFrom('#password'); I.click('Login'); I.fillField('email', 'miles@davis.com'); I.fillField('password', password); I.click('Log in!'); I.see('Hello, Miles'); }); 

, : WebDriverIO , Protractor , Nightmare , Appium , Puppeteer . .

, , CodeceptJS.

Results

-. ? , , - , , , . , , : , , .

Dear readers! JS-?