Geekly Articles each Day
📜 ⬆️ ⬇️

Testing through the eyes of the developer: tools, myths, situations



Evgeny Safronov, Senior Developer, DataArt

“Testing can be used to prove the presence of errors in the program, and never to prove their absence!”
Edsger Dijkstra
')
Testing is an applied, standardized, engineering practice that is applicable in most branches of human life. Testing, like philosophy, metrics or practice, exists much longer than programming. For example, we forged a sword. To check whether it is sharp enough, it turned out, it is tested. In some epochs even on a live person, say, a slave.

Testing is a test of the functionality of a program, item or any industrial development. As in any business, there are subtleties and a philosophy here. It is probably closer to testers who look destructively at things we produce - from the very beginning they are thinking about how to break a product offered by developers. This is not very typical for users who are more predictable and usually find errors, accidentally trying to do something atypical with our program. Developers have a different approach to software, but we must remember: testers must break what we have created - this is their bread.

Why is testing programs so important?


Developers do not think through testing, we think creatively - and this can be a source of problems. When we are asked to write a program, we first of all think about concepts, data structures, their description and interaction. As a result, we present a solution - ready, albeit crowded with bugs. Usually we have little idea what will happen if the input data changes, if the user performs a large number of atypical operations.

At the same time, practice shows that the most costly mistakes occur as a result of small changes. Because if you have written a bad piece of code, a more experienced colleague will ask you to redo it. With experience, you yourself will notice the unfortunate fragments almost immediately. But when errors are minimal - a separate symbol, a point, a sign of a financial transaction, a rounding error - it is very difficult to find them, and at any stage.


Fig. 1. Authorization form and the number of options to fill it out.

Let's see why there is such difficulty in testing. In the figure we see a simple form of three fields. In the first two fields, you can enter from 1 to 255 letters, in the third from 1 to 20 characters. You can also leave the lines blank. Below we see the number of possible combinations, which is significantly higher than the number of elementary particles in the universe. I think this is convincing evidence that it is unrealistic to check all possible cases. Yes, and try to do it, probably inappropriate.

Types of project errors



Fig. 2. The scheme of the distribution of errors by type, according to the data of Steve McConnell’s “Perfect Code”.

Approximately 25% of the total amount accounted for structural errors. They arise at the design stage, when you create data structures and write implementations of manipulations with them, that is, you create some kind of “glue” that holds these structures together. This is a huge layer of fundamental mistakes.

Next are the data errors associated with the implementation of functionality, etc. It is interesting that the architecture is in last place. This can be explained by the fact that, creating the wrong architecture, it is very difficult in principle to produce the final product.


Fig. 3. Distribution of errors by development stages. Scheme of Steve McConnell.

At the design and design stage, there is a major layer of errors and defects. The well-known Pareto rule works here: 80% of defects are localized only in 20% of your code. As a rule, these are some kind of corner cases. If you write operations with mathematical, for example, financial logic, a lot of errors can be within the boundary values, when rounding numbers, etc. Most of your cases will work, but most of the defects will be localized in a small section of code.

Regardless of whether you use manual or unit testing, you cannot argue that by covering 50% of the code with tests, you managed to prevent 50% of possible defects. If you wrote 100 unit tests, you cannot say that the final quality of your product has improved, for example, twice. Because the developer who does not think through the prism of testing, checks the easiest cases. The required forms are filled with the standard name, surname and login - the developer goes further. He most likely will not check for cases when one of the fields remains empty, or when a name of 255 characters is actually inserted into it. He will not experiment with non-standard characters, combinations of lowercase and uppercase letters.

Testing is the most popular quality management technique. If you recall our example with a sword, then before using it in a real battle, you will definitely ask to experience it longer. It is even better if, at the production stage, you agree with the blacksmith to also test the metal from which he will forge the weapon. To improve the quality of the program, we, respectively, want to increase the amount of testing and at the same time use the best practices. In proportion to our efforts, the quality of our product will increase.

Anyone can test a product: developer, tester, manager, customer. Sometimes in small projects, the role of a tester can be performed by a manager or developer. But all the four roles here are very important - each participant in the process looks at the object from his own point of view. Developer - through the prism of how the code works. From the very beginning he has most of the information on how to break the interface. For a tester, finding bugs and breaking a product is a direct task. The customer looks at the tests in a completely different way: he is interested in how to write fewer tests and pay less money, still getting a quality product in the end. But the most interesting role of the manager, who can usually rely on cases from his own practice. However, as a rule, it serves as a bridge between developers, testers and customers, and its main task comes down to product presentation. But it is the manager who is responsible for the quality of the product at the outlet.

Testing black and white boxes


The classification of testing in the form of analysis of white, gray and black boxes is interesting to us because the developer looks at his own code as a white box. And the tester and the customer view the program as a black box - they do not fully understand how the program works, they only know which features are important and must work.

The bug on the website of the online store, because of which some product does not fall into the basket, for the developer - a minor problem. It is enough to commit and uncommit, change variables in one place - and everything will work fine. For the tester, and especially the customer, such a bug turns out to be critical, since it does not allow the client to buy the right product.

But the most interesting box is gray. Those who I call clever testers, who delve deeply into the code and how the product works, work with it. They are interested in how it is deployed and communicating with databases.

What tests do the developers write?





Fig. 4. The unit test pyramid.

The basis of the pyramid - unit testing - it is desirable that the unit tests in the project was a lot. This is followed by integration testing of our modules, Acceptance Tests and direct UI testing of specific features.

FIRST


FIRST is a methodology for describing requirements that tests must meet. First of all, modular, but in principle, these characteristics can be extrapolated to automatic tests. Its creator, the famous Uncle Bob, is the author of many programming practices.




Fig. 5. Scheme of the TDD process.

The two basic unit testing methodologies are similar TDD and BDD concepts. The BDD approach is based on TDD and aims to eliminate the small flaws that are present in TDD. The syntax of BDD tests is more business-oriented and understandable to the customer and generally less technically savvy person. TDD - how to do things right. BDD - how to do the right thing.

What does the TDD process look like?


First of all, you are writing a test for code that has not yet been written. Accordingly, see that it does not work. To make it work, the code still needs to be written. Then you still write the code, run the test and make sure that the test works. Then you modify your test (specify the requirements, add the check of boundary conditions, etc.). After that, your test stops working, and again you come to the need to modify your code. This process is looped through until your test clearly meets the final criteria and meets the tasks you are facing. Suppose if you solve a quadratic equation, your task is to write a function that finds the roots of a quadratic equation. Suppose you wrote a test, and it works well based on real numbers. But if the user wants to find a comprehensive solution, then it will not work, the test will fill up and require modification. Therefore, this scheme should be in the blood of the developers: you need to act on it until your feature meets all the requirements.

What does TDD look like?


suite('#factorial()', function() { test('equals 1 for sets of zero length', function() { assert.equal(1, factorial(0)); }); test('equals 1 for sets of length one', function() { assert.equal(1, factorial(1)); }); test('equals 2 for sets of length two', function() { assert.equal(2, factorial(2)); }); test('equals 6 for sets of length three', function() { assert.equal(6, factorial(3)); }); });

The example is written in JavaScript, but I think, in syntactical terms, it will also look something like for PHP or Java. It's pretty obvious. By opening the test, you can easily tell which points were not tested. You can add something quickly and easily. And thus check whether your function works correctly.

BDD is very similar to TDD, but differs in that the wording in the description is slightly better adapted for people less closely associated with the development: managers or business analysts.

 describe('#factorial()', function() { it('should return 1 when given 0', function() { factorial(0).should.equal(1); }); it('should return 1 when given 1', function() { factorial(1).should.equal(1); }); it('should return 2 when given 2', function() { factorial(2).should.equal(2); }); it('should return 6 when given 3', function() { factorial(3).should.equal(6); }); });

Features of unit testing





Fig. 6. Unit testing tools.

QUnit is a library from jQuery developers that allows you to write unit tests in TDD style, with the assert mechanism. You write qunit.test, the name of the test, and what you want to test. Then run it in a separate file that your code should see, and you can make sure that the code works.

Mocha is a testing framework that allows you to write tests in TDD and BDD format. As a rule, it is used in conjunction with other tools in order to fully implement the TDD approach in work. That is, it allows you to run and describe tests in the correct format, and for example, another library (most often Chai) is responsible for processing assertion check (asserts).

Sinon is a tool for creating Mocks, stubs and spies, which is very often used in modern successful projects. It contains a set of functions and modules that are great help and solve a large number of typical problems that arise from the developer during the creation of unit tests.

Jasmine is a popular BDD library that has actually become a standard in the most common Angular Javascript framework ecosystem.

Myths



Project Situations


Source: https://habr.com/ru/post/342098/

More articles:


All Articles