The problem of duplication and obsolescence of knowledge in mock objects or Integration tests is good
Many programmers, when choosing between integration and unit test, prefer the unit test (or, in other words, the unit test). Some consider integration tests to be antipattern, some simply follow fashion trends. But let's see what this leads to. To implement a unit test, mock objects are hung not only on external services and data warehouses, but also on classes implemented directly inside the program. At the same time, if the mockable class is used in several other classes, then the mock object will be contained in tests for several classes . And since it is customary to set the tested behavior inside the test (see given-when-then , arrange-act-assert , test builder ), the behavior of the moka is set again each time in each test, and the DRY principle is violated (although code duplication may not be) . In addition, the class behavior is declared in the mock object, but this declaration itself is not checked, so over time, the behavior declared in the mock may become outdated and begin to differ from the actual behavior of the class being mocked. This causes a number of difficulties:
1) Firstly, when changing the functional, it is difficult to remember at all that, in addition to the class and tests for it, it is necessary to change the mocks of this class. Let's consider the development cycle in the framework of TDD: "creating \ changing tests for functionality -> creating \ changing functionality -> refactoring". Mock-objects are a declaration of class behavior and are not related to any of these three categories (they are not tests for functionality, despite the fact that they are used in tests, and even more so they are not functionality itself). Thus, changing the mock objects of classes implemented within the program does not fit the concept of TDD .
2) Secondly, it is difficult to find all the mocking places of this class. I have not seen a single tool for this. Here you can either write your bike, or watch all the places of use of this class and select those where the mocks are created. But with manual search you can make a mistake, overlook something. Then you probably had a question: if the problem is as fundamental as the author describes, has it really occurred to anyone to implement the tools that simplify its solution? I have a hypothesis on this. A few years ago, I started writing a library that was supposed to build a mock object just like an IOC container collects a regular class, and automatically create and run tests on the behavior described in mocks. But then I abandoned this idea because I found a more elegant solution to the problem of the mocks: just do not create this problem . Probably for a similar reason, a specialized tool for searching mocks of a particular class is either not implemented or is little known.
')
3) Thirdly, there may be a lot of class moistening places, and changing them all is a routine exercise. If a programmer is forced to do a routine that cannot be automated, then this is a clear sign that something is wrong with the tools, architecture, or workflow.
I hope the essence of the problem is clear. Next, I will describe ways to solve this problem and tell you why, from my point of view, integration tests are preferable to unit tests.
As a solution to the problem, I propose to apply mocking only for external services and data warehouses, and in other cases to use real classes, i.e. write integration tests instead of modular ones. Some programmers are skeptical about integration tests, and they would not like this idea.
Let's look at what arguments are the opponents of integration tests.
Evidence:
Let's imagine that in some class that is used everywhere, there was an error. After that, the tests of the class itself reddened, as well as all integration tests in which this class was used. As a result, half of the tests in the project is red. How to understand what causes the redness of the tests? What dough to start with? But if, instead of a class, its mock object was used, only tests of this class would be reddened.
Denial:
Let's remember the workflow in the framework of TDD: “red” tests, signaling an error -> creating / changing functionality -> “green” tests. Accordingly, when the functionality changes, the programmer first modifies the tests so that they test the modified functionality. Since the code is still obsolete functionality, the tests do not pass. Then the programmer rules the functional code, and the tests pass. If the programmer worked with the classes, but not with their tests, then he did not act within the framework of TDD.
But even if the programmer changed the code, but did not change the tests and did not check their passing, the fall in the tests can be traced by the continuous integration server, which automatically runs the tests each time it goes to the version control system. The author of the changes will see a message about the fall of the tests, hot on the heels will remember which classes he rules, and first of all he will begin to deal with tests of these particular classes. If a programmer inadvertently introduced a bug into a certain class, and then corrected it, then not only the tests of this class will turn green, but also all the tests in which this class was used. But what if they do not turn green? Then it is a signal that changes in the class led to changes in the behavior of other classes where this class was used, and now either these classes have errors, or their tests have deviated from the logic of the application.
Another case is possible. If for some reason the class in which the mistake was made was not well covered with tests, then the unit tests on the mocks would not reveal the problem at all. The integration tests will at least signal the problem, although in order to identify the problem class, you will have to resort to the good old tracing.
To summarize: if you follow TDD, then redness of the tests of those classes that you have not changed is an advantage, because it signals problems. If you do not follow TDD, but use continuous integration, then redness of “extra” tests is not such a problem for you. If you do not follow TDD and do not perform a regular test run, then the problem of detecting “dropped test - problem class” compliance is topical for you. In this case, it is better to solve the problem of duplication of knowledge in mocks and the lack of tests for behavior declared in mocks, not by using integration tests instead of modular ones, but by other means (we will talk about them a little later).
Evidence:
Unit testing, unlike integration testing, forces programmers to inject dependencies through a constructor or properties. And if you use integration testing instead of modular, then the junior can instantiate dependencies directly in the class code. And I don’t have much time to write architectural notes and review codes. Yes and no one to charge. And do not want to.
Denial:
In fact, not only unit testing can force a programmer to inject dependencies. IOC-container handles this very well. In fact, if you inject dependencies, then you probably use IOC-container. You can, of course, write the most important class creation factory in which the entry point is located. But IOC-container solves many typical problems and simplifies life. For example, you can make a singleton a single line of code without singing into the pitfalls of a singleton implementation. So, if you inject dependencies, but do not use IOC-container, then I recommend starting it.
In general, if you are using unit testing, then you are almost certainly using IOC-container. If you are using IOC-container, then it prompts the programmer to inject dependencies. Of course, you can create an object without using IOC-container, but you can also create a class in the same way without supplying it with a unit test. So, I do not see the unit tests with significant advantages in terms of encouraging the execution of the principle of Inversion of control.
In addition, you can not force programmers to act in the way you want due to limitations in the architecture, but simply to explain the benefits of dependency injection and the use of an IOC container. Coercion by force , as well as any violence, can cause counter resistance.
Evidence:
The author of the article with the loud title “Integration tests - the lot of crooks” writes that he hates integration tests with all passion and considers them to be a virus that brings endless pain and suffering. He justifies his thoughts as follows:
Denial:
The author of that article gives the following definition of an integration test:
As you can see, in this definition there is not a word that integration tests can be written only on the main class in which the entry point is located, but the author of the above article implicitly relies on this condition in his reasoning.
According to TDD, tests are designed to check the functionality (feature), and not the ways of program execution. Follow TDD, and you will not encounter the problems that this author spoke of. Just write integration tests as you would write unit tests, but do not mock the classes implemented in your program, and you will not face the problem of an exponential increase in the number of tests.
With this, unfortunately, you can not argue - integration tests are almost always performed longer than modular ones. Moka creation is, of course, not free and takes some time, but the application logic, as a rule, runs longer. Hypothetically, it is quite possible that tests are performed unsatisfactorily for a long time, and you are not going to optimize the logic being tested in the near future. And optimization of tests can become quite logical decision. For example, the use of mocks.
The first way, as I said before, is to use mocks only for declaring the behavior of external services and data warehouses.
The second way is to automatically check the relevance of the behavior declared in moke. For example, you can automatically create and run the appropriate test. But then you need to consider that the mockable class may have its own dependencies, some of which may be external services. For speed, you can first test the unique behavior (specified in mocks) of the classes of the bottom layer, then the behavior of the classes that use the previous classes, and so on. Then, if some kind of identical behavior is declared in mocks in several places, then it can be checked only once.
For each unique case of mocking, you can manually write a test and somehow define the correspondence between the moke and the test for it, and instruct programmers to manually maintain this correspondence when the functionality changes.
You can simply instruct programmers to manually maintain the relevance of mock objects. But then you have to slightly change the workflow, moving away from the classic TDD, replace “Changing tests for functionality -> Changing functionality -> ...” to “Changing tests for functionality -> Changing declarations of this behavior (in mocks) -> Changing functionality - > ... ".
To eliminate the problem of code duplication during mocking, you can put all the mocks on one class in a separate storage. This will simplify the stage “Changing declarations of behavior in mocks”, but can reduce the readability of a unit test - then decide for yourself, based on your own priorities.
Martin Fowler noticed the formation of two different TDD schools - classical school and mokist:
Both of these schools have their advantages and disadvantages. Personally, I think that the shortcomings of the classic TDD are more acceptable and more solvable than the shortcomings of the Moody TDD. Well, someone may think the other way around - he can cope well with the consequences of using mokyd TDD and not consider problems that arise with classic TDD to be acceptable. Why not? All people are different, and everyone has the right to their own style. I just argued why I personally like the classic more, but the final choice is yours.
PS I do not urge you to completely abandon the unit tests. When using classic TDD, tests for those classes that do not apply to methods and properties of other classes will be modular.
1) Firstly, when changing the functional, it is difficult to remember at all that, in addition to the class and tests for it, it is necessary to change the mocks of this class. Let's consider the development cycle in the framework of TDD: "creating \ changing tests for functionality -> creating \ changing functionality -> refactoring". Mock-objects are a declaration of class behavior and are not related to any of these three categories (they are not tests for functionality, despite the fact that they are used in tests, and even more so they are not functionality itself). Thus, changing the mock objects of classes implemented within the program does not fit the concept of TDD .
2) Secondly, it is difficult to find all the mocking places of this class. I have not seen a single tool for this. Here you can either write your bike, or watch all the places of use of this class and select those where the mocks are created. But with manual search you can make a mistake, overlook something. Then you probably had a question: if the problem is as fundamental as the author describes, has it really occurred to anyone to implement the tools that simplify its solution? I have a hypothesis on this. A few years ago, I started writing a library that was supposed to build a mock object just like an IOC container collects a regular class, and automatically create and run tests on the behavior described in mocks. But then I abandoned this idea because I found a more elegant solution to the problem of the mocks: just do not create this problem . Probably for a similar reason, a specialized tool for searching mocks of a particular class is either not implemented or is little known.
')
3) Thirdly, there may be a lot of class moistening places, and changing them all is a routine exercise. If a programmer is forced to do a routine that cannot be automated, then this is a clear sign that something is wrong with the tools, architecture, or workflow.
I hope the essence of the problem is clear. Next, I will describe ways to solve this problem and tell you why, from my point of view, integration tests are preferable to unit tests.
As a solution to the problem, I propose to apply mocking only for external services and data warehouses, and in other cases to use real classes, i.e. write integration tests instead of modular ones. Some programmers are skeptical about integration tests, and they would not like this idea.
Let's look at what arguments are the opponents of integration tests.
Statement 1. Integration tests are less helpful in finding errors than unit tests.
Evidence:
Let's imagine that in some class that is used everywhere, there was an error. After that, the tests of the class itself reddened, as well as all integration tests in which this class was used. As a result, half of the tests in the project is red. How to understand what causes the redness of the tests? What dough to start with? But if, instead of a class, its mock object was used, only tests of this class would be reddened.
Denial:
Let's remember the workflow in the framework of TDD: “red” tests, signaling an error -> creating / changing functionality -> “green” tests. Accordingly, when the functionality changes, the programmer first modifies the tests so that they test the modified functionality. Since the code is still obsolete functionality, the tests do not pass. Then the programmer rules the functional code, and the tests pass. If the programmer worked with the classes, but not with their tests, then he did not act within the framework of TDD.
But even if the programmer changed the code, but did not change the tests and did not check their passing, the fall in the tests can be traced by the continuous integration server, which automatically runs the tests each time it goes to the version control system. The author of the changes will see a message about the fall of the tests, hot on the heels will remember which classes he rules, and first of all he will begin to deal with tests of these particular classes. If a programmer inadvertently introduced a bug into a certain class, and then corrected it, then not only the tests of this class will turn green, but also all the tests in which this class was used. But what if they do not turn green? Then it is a signal that changes in the class led to changes in the behavior of other classes where this class was used, and now either these classes have errors, or their tests have deviated from the logic of the application.
Another case is possible. If for some reason the class in which the mistake was made was not well covered with tests, then the unit tests on the mocks would not reveal the problem at all. The integration tests will at least signal the problem, although in order to identify the problem class, you will have to resort to the good old tracing.
To summarize: if you follow TDD, then redness of the tests of those classes that you have not changed is an advantage, because it signals problems. If you do not follow TDD, but use continuous integration, then redness of “extra” tests is not such a problem for you. If you do not follow TDD and do not perform a regular test run, then the problem of detecting “dropped test - problem class” compliance is topical for you. In this case, it is better to solve the problem of duplication of knowledge in mocks and the lack of tests for behavior declared in mocks, not by using integration tests instead of modular ones, but by other means (we will talk about them a little later).
Statement 2. Integration tests are less helpful in designing than modular ones.
Evidence:
Unit testing, unlike integration testing, forces programmers to inject dependencies through a constructor or properties. And if you use integration testing instead of modular, then the junior can instantiate dependencies directly in the class code. And I don’t have much time to write architectural notes and review codes. Yes and no one to charge. And do not want to.
Denial:
In fact, not only unit testing can force a programmer to inject dependencies. IOC-container handles this very well. In fact, if you inject dependencies, then you probably use IOC-container. You can, of course, write the most important class creation factory in which the entry point is located. But IOC-container solves many typical problems and simplifies life. For example, you can make a singleton a single line of code without singing into the pitfalls of a singleton implementation. So, if you inject dependencies, but do not use IOC-container, then I recommend starting it.
In general, if you are using unit testing, then you are almost certainly using IOC-container. If you are using IOC-container, then it prompts the programmer to inject dependencies. Of course, you can create an object without using IOC-container, but you can also create a class in the same way without supplying it with a unit test. So, I do not see the unit tests with significant advantages in terms of encouraging the execution of the principle of Inversion of control.
In addition, you can not force programmers to act in the way you want due to limitations in the architecture, but simply to explain the benefits of dependency injection and the use of an IOC container. Coercion by force , as well as any violence, can cause counter resistance.
Statement 3. To cover the same functionality with tests, integration tests will require much more than modular
Evidence:
The author of the article with the loud title “Integration tests - the lot of crooks” writes that he hates integration tests with all passion and considers them to be a virus that brings endless pain and suffering. He justifies his thoughts as follows:
You write integration tests because you are not able to write perfect unit tests. You know this problem: all your tests have passed, but the program still reveals a defect. You decide to write an integration test to make sure that the entire program execution path works as it should. And everything seems to be going fine until you think: “Let's use integration tests everywhere.” Bad idea! The number of possible ways to execute a program non-linearly depends on the size of the program. To cover a web application with 20 pages, you need at least 10,000 tests. Maybe a million. When writing 50 tests per week, you write only 2,500 tests per year, which is 2.5% of the required amount. And after that you wonder why you spend 70% of your time answering user calls ?! Integration tests are a waste of time. They must remain in the past.
Denial:
The author of that article gives the following definition of an integration test:
It has been shown that it has been shown that it can be used to
An integration test is such a test, the result of passing through which depends on the correctness of the implementation of more than one piece of non-trivial logic (method).
As you can see, in this definition there is not a word that integration tests can be written only on the main class in which the entry point is located, but the author of the above article implicitly relies on this condition in his reasoning.
According to TDD, tests are designed to check the functionality (feature), and not the ways of program execution. Follow TDD, and you will not encounter the problems that this author spoke of. Just write integration tests as you would write unit tests, but do not mock the classes implemented in your program, and you will not face the problem of an exponential increase in the number of tests.
Statement 4. Integration tests run longer than modular ones.
With this, unfortunately, you can not argue - integration tests are almost always performed longer than modular ones. Moka creation is, of course, not free and takes some time, but the application logic, as a rule, runs longer. Hypothetically, it is quite possible that tests are performed unsatisfactorily for a long time, and you are not going to optimize the logic being tested in the near future. And optimization of tests can become quite logical decision. For example, the use of mocks.
Ways to combat duplication and obsolescence of knowledge in moka
The first way, as I said before, is to use mocks only for declaring the behavior of external services and data warehouses.
The second way is to automatically check the relevance of the behavior declared in moke. For example, you can automatically create and run the appropriate test. But then you need to consider that the mockable class may have its own dependencies, some of which may be external services. For speed, you can first test the unique behavior (specified in mocks) of the classes of the bottom layer, then the behavior of the classes that use the previous classes, and so on. Then, if some kind of identical behavior is declared in mocks in several places, then it can be checked only once.
For each unique case of mocking, you can manually write a test and somehow define the correspondence between the moke and the test for it, and instruct programmers to manually maintain this correspondence when the functionality changes.
You can simply instruct programmers to manually maintain the relevance of mock objects. But then you have to slightly change the workflow, moving away from the classic TDD, replace “Changing tests for functionality -> Changing functionality -> ...” to “Changing tests for functionality -> Changing declarations of this behavior (in mocks) -> Changing functionality - > ... ".
To eliminate the problem of code duplication during mocking, you can put all the mocks on one class in a separate storage. This will simplify the stage “Changing declarations of behavior in mocks”, but can reduce the readability of a unit test - then decide for yourself, based on your own priorities.
Conclusion
Martin Fowler noticed the formation of two different TDD schools - classical school and mokist:
I’m looking for a second dichotomy: TDD. This is where you can use a mock (or other double).
If you are awkward, you can use it. I would like to use the TDDer for the mail service. The kind of double doesn't really matter that much.
A mockist tdd practitioner, however, he will always be interested. In this case for both the warehouse and the mail service.
Both of these schools have their advantages and disadvantages. Personally, I think that the shortcomings of the classic TDD are more acceptable and more solvable than the shortcomings of the Moody TDD. Well, someone may think the other way around - he can cope well with the consequences of using mokyd TDD and not consider problems that arise with classic TDD to be acceptable. Why not? All people are different, and everyone has the right to their own style. I just argued why I personally like the classic more, but the final choice is yours.
PS I do not urge you to completely abandon the unit tests. When using classic TDD, tests for those classes that do not apply to methods and properties of other classes will be modular.
Source: https://habr.com/ru/post/275249/
All Articles