Tests for tests

One of the most frequent answers to the question “Why do I not write unit tests?” Is the question “Who will write tests for my tests? Where is the guarantee that there won't be an error in my tests either? ”, Which indicates a serious misunderstanding of the essence of unit tests.

The purpose of this note is to briefly and clearly fix this point so that there will be no more disagreement.

So unit test is a set of several examples showing what comes to the input of your program, and what happens at the output. For example, if the program calculates the length of the hypotenuse, then in the unit test one line is enough:

assertEquals(5, hypotenuse(3, 4)); 

You should come up with these test values ​​yourself: count as a bar on a piece of paper or on a calculator.

Of course, whether this test is enough is up to you. If it is very important in your specific task that the function works correctly on very small numbers, then a separate test should be added.

 assertEquals(0.00005, hypotenuse(0.00003, 0.00004)); 

it is also important to describe the behavior of the function in the case of negative or some other parameters. In general, you decide, but the basic idea is that in a unit test there should be several important examples.

')

rules

So, the unit test differs from the program in that:

• It is an order of magnitude simpler than the program under test.
• It has only a set of examples.
• It does not have the logic that is in the program (you invent the examples yourself)
• A unit test cannot and should not cover all possible cases. Rather, he should designate all important cases , that is, cases that should be separately discussed.
• A unit test should serve as program documentation — that is, after reading a unit test, a person must understand how the program works. Usually the names of test methods are used for this purpose.

The first point is the answer to the question “Who will write the tests for my tests?”. Since a unit test is an order of magnitude simpler than a program, a test for it would have to be written an order of magnitude simpler, but this is practically unrealistic, since it is already extremely simple.

The answer to the second question “Where is the guarantee that there will be no error in my tests?” Is this: there is no guarantee and cannot be. But the fact that the “correct” answers written in the test are obtained in a different way than the answers given by the program allows you to be more confident that both ways are correct.
UPD: kalantyr tells you that unit tests and code test each other as they are interrelated. That is, in a sense, the tests for the tests are the code itself.

Example

Let's try to give here a minimal unit test example that illustrates all of the above. Suppose we need to write a function that decides whether a given year is a leap year. Let me remind you that a leap year is a year that is divisible by 4, except for those years that are divided by 100, but not divided by 400.

 public class LeapYear { public static boolean isLeap(int year) { return year % 4 == 0 && year % 100 != 0 || year % 400 == 0; } } import org.junit.Test; import static junit.framework.Assert.*; public class LeapYearTest { @Test public void yearDividing4IsLeap() throws Exception { assertTrue(isLeap(2008)); assertTrue(isLeap(2012)); assertFalse(isLeap(2011)); } @Test public void exceptYearDividing100WhichIsNotLeap() throws Exception { assertFalse(isLeap(1900)); assertFalse(isLeap(2100)); } @Test public void exceptYearDividing400WhichIsLeap() throws Exception { assertTrue(isLeap(1600)); assertTrue(isLeap(2000)); } } 

As you can see, inside the test cases themselves are specific simple examples: 2000, 2008, 1011. I invented them myself, from the head. And the names of test cases explain them (summarize) in human language. Note that the one-to-one test method names match the description of the term “leap year” given above. So it should be: the test should be read as documentation. And this is how it looks, for example, in IDEA:

Errors

Because of the lack of understanding of the essence of unit tests, when writing them, major mistakes are often made that make unit tests useless, time consuming and lead to endless discussions about whether unit tests are needed at all.

The first common mistake is to check nothing at all. For example:

 @Test public void testLeapYear() throws Exception { int year = 2004; assertEquals(2004, year); } 

This unit test is meaningless, as it does not check our program. The maximum that he checks is that Java works correctly, but this is already paranoia. Let Oracle do this.
Despite the seeming absurdity, it is from such tests that everyone who takes JUnit in their hands for the first time begins.

The second typical mistake is to try to repeat in the test the same logic that is in the program. For example:

 @Test public void testLeapYear() throws Exception { for (int i=0; i<100000; i++) { assertEquals(i % 4 == 0 && i % 100 != 0 || i % 400 == 0, isLeap(i)); } } 

This unit test is bad just because a programmer can make the same mistakes as in the code itself.

Smells

Signs that a unit test is wrong:

• Conditions and cycles
  If you see conditions and cycles in a unit test, this is a clear sign that the test is wrong. Try to get rid of the cycles and write some simple examples.
  ^{Of course, cycles are sometimes needed in the test, for example, to compare two arrays. Rather, it was meant that there should be no cycles with the same logic as in the tested program.}
• The name of the test method does not say what should work and how.
  If you see in the title only the name of what is being tested - for example, testLeapYear, be on your guard. He probably does not test anything or tests too much. The correct name of the test method should sound something like this: “In such and such conditions, such and such a method should behave this way and that”.
• There is too much assert in the body of the test method.
  Such a test method checks too many aspects. If it breaks, it will be impossible to immediately determine by the name of the method what the error is - you will have to analyze the test class code. Try to break the test method into several methods and give them speaking names.
  ^{Of course, sometimes there can be a lot of assert in the test method, for example, in order to check that large HTML contains all the necessary values. Rather, this recommendation should be understood as “making the tests shorter and checking one possible path in one method to make it easier to localize the place where the test falls.”}
• What other signs do you know? ..

Open source

It is always useful to study the unit tests of well-known open-source projects. This topic is worth a separate article, but the first thing that came to mind was Apache Velocity and the Spring Framework . I didn’t like the test from the first UnicodeInputStreamTestCase project, since the names of the test methods do not really describe the program's behavior (for example, “testSimpleStream ()”). But I liked the test for Spring, for example, for example, for the CollectionUtils class there is the CollectionUtilsTests unit test, and for the Assert class there are AssertTests unit tests.

I hope this article will stop the eternal debate that unit tests are useless, time consuming, etc., and can serve as a starting point for further discussions about how to write, how much to write, what to write and so on. And someone, perhaps, will even have to break away from Habr and read serious books, such as " Test Driven Development (Kent Beck) ", " xUnit Test Patterns (Gerard Meszaros) " and my favorite " Clean Code (Robert C. Martin ) ".

To test or not to test - that is not a question ...