Unit Test Definition
On our Internet sites you have recently made a serious noise about whether TDD is still alive, whether it is needed, and in what form. It all started with David Hanson ’s article “TDD is dead. Long living testing ” , followed by articles by many authors and lively discussions of this problem, including hangout with David, Kent Beck and Martin Fowler (by the way, the next hangout will be tomorrow, May 16).
But few know that a few days before that, all the same Martin Fowler tried to define the unit test ( bliki: UnitTest ), the translation of which is presented below. And after the translation, some of my thoughts are on this.
-
In the world of software development, they often speak of unit tests, and I am familiar with this concept throughout my programming career. However, like many other terms from the software world, this term is defined very poorly, and I often come across confusion when developers think that it has a stricter definition than it actually is.
')
And although I very often used unit tests, my final commitment to them arose when I started working with Kent Beck and using the xUnit family of testing tools. (It even sometimes seems to me that a more suitable term for this type of testing would be “xunit testing”.) Unit testing also became an important activity in Extreme Programming (XP - Extreme Programming), and quickly developed into development through testing (TDD - Test-Driven Development).
The role of unit tests in XP from the very beginning caused some concern. I clearly remember the discussion in the usenet group, in which an expert on testing scolded XP advocates for the misuse of the term “unit test”. We asked for his definition, to which he replied with something like: “at the very beginning of my training course on testing, I consider 24 different definitions of the unit test”.
Despite the differences, in some aspects our points of view converged. First, there is the notion that unit tests are low-level and concentrate only on a small part of the software system. Secondly, unit tests today are usually written by developers using their usual tools, to which is added some testing framework (*). Third, unit tests are expected to be significantly faster than other types of tests.
But there were differences of opinion. There are different points of view on what is considered a module . In an object-oriented design, a module is considered to be a class, in a procedural and functional approach a function can be considered a module. In fact, this is a situational concept: the team decides that it is reasonable to consider a module for understanding the system or testing it. And although I start with the notion that a class is a module, I often begin to consider a set of closely related classes as a single module. Less often, I can consider a subset of class methods as a module. In fact, it does not really matter how you define this concept.
A more important difference in approach is the question: should the module under test be separated from the interacting objects? Suppose you are testing an order class calculation method. The price calculation method calls some methods of the product and customer classes. If you follow the principle of isolation of interacting objects, you do not want to use real classes of products and customers here, because errors in the customer class will lead to a drop in the tests of the order class. Instead, you use the fakes ( Test Doubles ) of all interacting objects.
But not all developers use isolation. In fact, when xunit testing began in the 90s, we did not try to isolate the class being tested, unless communication with other objects was extremely inconvenient (such as interacting with a remote credit card verification system). We had no difficulty in understanding the real cause of the failure, even if neighboring tests fell during this. Therefore, from a practical point of view, we did not consider the lack of isolation a problem.
Although it was the lack of isolation in our definition of “modular test” that was the reason for its criticism. I think the definition of “unit test” is appropriate because it tests the behavior of a single module. We write a test, assuming that everything except this module works correctly.
When xunit testing began to gain popularity in the 2000s, the idea of ​​isolation returned with a new force, at least for some. We saw the rise of Mock Objects (Mock Objects) and frameworks to support mocking. As a result, there are two schools of xunit testers, which I call the classical school and the school of mokists (mockists). The adherents of the classical school do not bother with isolation, as do the Mokist. I know and respect the xunit-testers of both schools (although I myself belong to the classical school).
Even representatives of the classical school (including me) in the presence of complex interactions use fakes (test doubles). Counterfeits are invaluable for eliminating the uncertainty of behavior when working with remote services . Some representatives of the classical school believe that any interaction with external resources, such as databases or the file system, should use fakes. Part of this opinion is based on the risk of uncertain behavior, partly on problems with speed. And although I think this is a useful recommendation, I do not see it as an absolute rule. If a call to a resource is stable and fast enough for you, then there is no reason why it cannot be used from unit tests.
There are several common properties of unit tests: a small scope (small scope), they are written by developers, and they are quickly executed - which makes it possible to run them often during development. Indeed, this is one of the key properties of the Self-Testing Code . In this case, the programmer can run unit tests after any change in the code. I can run unit tests several times a minute, every time I need to compile code. This is useful, because if I accidentally break something, I want to immediately find out about it. If I broke something with my recent changes, it is much easier to immediately find this error, since I don’t have to look for it very far.
Translator's note
Kent Beck developed the idea of ​​running tests at compilation (and sometimes even without it) and proposed the idea of ​​continuous testing (Continuous Testing). Examples of such tools: Mighty-Moose and NCrunch for .NET, JUnit Max for Java.
When you run tests so often, you cannot run them all. Usually, you only need to run the tests that work with the code you are currently working on. In this case, you sacrifice the depth of testing in favor of the test run duration. I call this test suite the “compile suite” because I run them every time I compile, even in interpreted languages ​​like Ruby.
If you are using Continuous Integration, you must run the tests as one of its steps. This set of tests, which I call the “ commit suite”, should include all unit tests. It may also include some acceptance tests ( Broad-Stack Tests or End-to-End Tests ). As a developer, you must run this test suite several times a day, of course, before committing your changes to the version control system, as well as at any other time when you have such an opportunity - during a break or rally. The faster the set of fixation tests is performed, the more often you can run them (**).
Different people have different standards for the speed of execution of unit tests and their sets. So, for David Hanson ( David Heinemeier Hansson ) it is enough that the compile suite takes a few seconds to complete and the commit suite takes a few minutes. Harry Bernhardt ( Gary Bernhardt ) thinks it is too slow and insists that the compilation set be executed for about 300 ms, and Dan Bodart does not want to wait for the fix set to be executed for more than a few seconds.
I do not think that there is a single correct answer to this question. Personally, I did not see the difference when the compilation set is executed in a split second or a couple of seconds. I like the Kent Beck rule that the commit set should not be executed for longer than 10 minutes. The main idea here is that your test suite should be executed fast enough so as not to discourage you from running it often enough. And “quite often” means that when tests find a bug, you will have to dig a small amount of code and find it pretty quickly.
(*) I say "today", because it changed thanks to XP. In the debates of the beginning of the new century, supporters of XP were subjected to serious criticism, since the generally accepted point of view said that programmers should not test their own code. In some companies there were specialized “unit testers” whose only task was to write unit tests for the developers code. The reason for this view was the following: people have “conceptual blindness” when testing their code; programmers are bad testers, so it is useful to have some form of confrontation between programmers and testers. The point of view of XP supporters was that programmers can learn to be good testers, at least at the level of a separate “module”, and if we bring in an additional group for writing tests, the feedback provided by the tests will be incredibly slow. XUnit tools played a very important role in this, since they were designed specifically to minimize the overhead of writing tests.
(**) If you have useful tests whose execution duration exceeds the duration of the commit tests, you need to build a Deployment Pipeline and place these tests in the later stages of the pipeline.
-
In this article, Martin deliberately does not address the questions of writing code and tests; instead, he tries only to define the unit test and show the existence of different points of view on the very concept of a module, on the need for isolation and speed of execution.
I myself also quite often come across the opinion that the unit test should test the class in complete isolation from the rest of the world. For example, it is this approach that Robert Martin describes in his book Principles, Patterns, and Techniques for Agile Development, and I criticized him in the article “A Critical Look at the Principle of Inversion of Dependencies” .
In my understanding, there is absolutely nothing wrong with using specific classes in unit tests if their behavior is deterministic and fast. Separating extra dependencies can undermine class encapsulation and ultimately reduce the ease of understanding and maintaining the system. Any stable dependencies can and should be used directly, and only “volatile” dependencies whose behavior is not deterministic should be distinguished.
It turns out that I am a supporter of the classical unit testing school, and I believe that it is necessary to use real classes if they do not turn to non-deterministic external resources. The problem with the abundance of mocks in my understanding is that the obtained tests become too dependent on the test environment, which makes them fragile, and the abundance of interfaces and indirection worsens the “understandability” of the system adding flexibility that is not needed in 99% of cases .
Of course, there are other points of view on the use of mocks. So, Steve Freeman and Net Price in their book Growing Object-Oriented Software Guided by Tests have a different point of view. But at the same time, they very carefully follow the simple tests and do not allow the situation when each line of the test receives 5 lines of initialization of mocks.
It is absolutely normal to stick to either of the two camps: the classics or the Mokkists. The main thing is that your choice is conscious, and your tests simplify development and maintenance, and do not interfere with this.
But few know that a few days before that, all the same Martin Fowler tried to define the unit test ( bliki: UnitTest ), the translation of which is presented below. And after the translation, some of my thoughts are on this.
-
In the world of software development, they often speak of unit tests, and I am familiar with this concept throughout my programming career. However, like many other terms from the software world, this term is defined very poorly, and I often come across confusion when developers think that it has a stricter definition than it actually is.
')
And although I very often used unit tests, my final commitment to them arose when I started working with Kent Beck and using the xUnit family of testing tools. (It even sometimes seems to me that a more suitable term for this type of testing would be “xunit testing”.) Unit testing also became an important activity in Extreme Programming (XP - Extreme Programming), and quickly developed into development through testing (TDD - Test-Driven Development).
The role of unit tests in XP from the very beginning caused some concern. I clearly remember the discussion in the usenet group, in which an expert on testing scolded XP advocates for the misuse of the term “unit test”. We asked for his definition, to which he replied with something like: “at the very beginning of my training course on testing, I consider 24 different definitions of the unit test”.
Despite the differences, in some aspects our points of view converged. First, there is the notion that unit tests are low-level and concentrate only on a small part of the software system. Secondly, unit tests today are usually written by developers using their usual tools, to which is added some testing framework (*). Third, unit tests are expected to be significantly faster than other types of tests.
But there were differences of opinion. There are different points of view on what is considered a module . In an object-oriented design, a module is considered to be a class, in a procedural and functional approach a function can be considered a module. In fact, this is a situational concept: the team decides that it is reasonable to consider a module for understanding the system or testing it. And although I start with the notion that a class is a module, I often begin to consider a set of closely related classes as a single module. Less often, I can consider a subset of class methods as a module. In fact, it does not really matter how you define this concept.
Insulation
A more important difference in approach is the question: should the module under test be separated from the interacting objects? Suppose you are testing an order class calculation method. The price calculation method calls some methods of the product and customer classes. If you follow the principle of isolation of interacting objects, you do not want to use real classes of products and customers here, because errors in the customer class will lead to a drop in the tests of the order class. Instead, you use the fakes ( Test Doubles ) of all interacting objects.
But not all developers use isolation. In fact, when xunit testing began in the 90s, we did not try to isolate the class being tested, unless communication with other objects was extremely inconvenient (such as interacting with a remote credit card verification system). We had no difficulty in understanding the real cause of the failure, even if neighboring tests fell during this. Therefore, from a practical point of view, we did not consider the lack of isolation a problem.
Although it was the lack of isolation in our definition of “modular test” that was the reason for its criticism. I think the definition of “unit test” is appropriate because it tests the behavior of a single module. We write a test, assuming that everything except this module works correctly.
When xunit testing began to gain popularity in the 2000s, the idea of ​​isolation returned with a new force, at least for some. We saw the rise of Mock Objects (Mock Objects) and frameworks to support mocking. As a result, there are two schools of xunit testers, which I call the classical school and the school of mokists (mockists). The adherents of the classical school do not bother with isolation, as do the Mokist. I know and respect the xunit-testers of both schools (although I myself belong to the classical school).
Even representatives of the classical school (including me) in the presence of complex interactions use fakes (test doubles). Counterfeits are invaluable for eliminating the uncertainty of behavior when working with remote services . Some representatives of the classical school believe that any interaction with external resources, such as databases or the file system, should use fakes. Part of this opinion is based on the risk of uncertain behavior, partly on problems with speed. And although I think this is a useful recommendation, I do not see it as an absolute rule. If a call to a resource is stable and fast enough for you, then there is no reason why it cannot be used from unit tests.
Speed
There are several common properties of unit tests: a small scope (small scope), they are written by developers, and they are quickly executed - which makes it possible to run them often during development. Indeed, this is one of the key properties of the Self-Testing Code . In this case, the programmer can run unit tests after any change in the code. I can run unit tests several times a minute, every time I need to compile code. This is useful, because if I accidentally break something, I want to immediately find out about it. If I broke something with my recent changes, it is much easier to immediately find this error, since I don’t have to look for it very far.
Translator's note
Kent Beck developed the idea of ​​running tests at compilation (and sometimes even without it) and proposed the idea of ​​continuous testing (Continuous Testing). Examples of such tools: Mighty-Moose and NCrunch for .NET, JUnit Max for Java.
When you run tests so often, you cannot run them all. Usually, you only need to run the tests that work with the code you are currently working on. In this case, you sacrifice the depth of testing in favor of the test run duration. I call this test suite the “compile suite” because I run them every time I compile, even in interpreted languages ​​like Ruby.
If you are using Continuous Integration, you must run the tests as one of its steps. This set of tests, which I call the “ commit suite”, should include all unit tests. It may also include some acceptance tests ( Broad-Stack Tests or End-to-End Tests ). As a developer, you must run this test suite several times a day, of course, before committing your changes to the version control system, as well as at any other time when you have such an opportunity - during a break or rally. The faster the set of fixation tests is performed, the more often you can run them (**).
Different people have different standards for the speed of execution of unit tests and their sets. So, for David Hanson ( David Heinemeier Hansson ) it is enough that the compile suite takes a few seconds to complete and the commit suite takes a few minutes. Harry Bernhardt ( Gary Bernhardt ) thinks it is too slow and insists that the compilation set be executed for about 300 ms, and Dan Bodart does not want to wait for the fix set to be executed for more than a few seconds.
I do not think that there is a single correct answer to this question. Personally, I did not see the difference when the compilation set is executed in a split second or a couple of seconds. I like the Kent Beck rule that the commit set should not be executed for longer than 10 minutes. The main idea here is that your test suite should be executed fast enough so as not to discourage you from running it often enough. And “quite often” means that when tests find a bug, you will have to dig a small amount of code and find it pretty quickly.
Notes
(*) I say "today", because it changed thanks to XP. In the debates of the beginning of the new century, supporters of XP were subjected to serious criticism, since the generally accepted point of view said that programmers should not test their own code. In some companies there were specialized “unit testers” whose only task was to write unit tests for the developers code. The reason for this view was the following: people have “conceptual blindness” when testing their code; programmers are bad testers, so it is useful to have some form of confrontation between programmers and testers. The point of view of XP supporters was that programmers can learn to be good testers, at least at the level of a separate “module”, and if we bring in an additional group for writing tests, the feedback provided by the tests will be incredibly slow. XUnit tools played a very important role in this, since they were designed specifically to minimize the overhead of writing tests.
(**) If you have useful tests whose execution duration exceeds the duration of the commit tests, you need to build a Deployment Pipeline and place these tests in the later stages of the pipeline.
-
In this article, Martin deliberately does not address the questions of writing code and tests; instead, he tries only to define the unit test and show the existence of different points of view on the very concept of a module, on the need for isolation and speed of execution.
I myself also quite often come across the opinion that the unit test should test the class in complete isolation from the rest of the world. For example, it is this approach that Robert Martin describes in his book Principles, Patterns, and Techniques for Agile Development, and I criticized him in the article “A Critical Look at the Principle of Inversion of Dependencies” .
In my understanding, there is absolutely nothing wrong with using specific classes in unit tests if their behavior is deterministic and fast. Separating extra dependencies can undermine class encapsulation and ultimately reduce the ease of understanding and maintaining the system. Any stable dependencies can and should be used directly, and only “volatile” dependencies whose behavior is not deterministic should be distinguished.
It turns out that I am a supporter of the classical unit testing school, and I believe that it is necessary to use real classes if they do not turn to non-deterministic external resources. The problem with the abundance of mocks in my understanding is that the obtained tests become too dependent on the test environment, which makes them fragile, and the abundance of interfaces and indirection worsens the “understandability” of the system adding flexibility that is not needed in 99% of cases .
Of course, there are other points of view on the use of mocks. So, Steve Freeman and Net Price in their book Growing Object-Oriented Software Guided by Tests have a different point of view. But at the same time, they very carefully follow the simple tests and do not allow the situation when each line of the test receives 5 lines of initialization of mocks.
It is absolutely normal to stick to either of the two camps: the classics or the Mokkists. The main thing is that your choice is conscious, and your tests simplify development and maintenance, and do not interfere with this.
Source: https://habr.com/ru/post/222963/
All Articles