TSA style testing

When developers first discover the benefits of developing through testing, it’s like a transition to a new, better world, where there is far less stress and insecurity. This wonderful experience is really worth celebrating. But awareness of the benefits of testing is only the first step towards enlightenment. The most difficult thing is to understand what NOT to test.

If a beginner can not care about what is not worth testing on the first day, then on the second day he would be better to start to delve into it. People create habits, so if you start to form a bad habit of over-testing from the very beginning, it will be much more difficult for you to get rid of it later. And you should get rid of this habit.

“ But what's wrong with over-testing, Phil, don't you want your code to be safe?” If we catch another bug before getting into production, isn't it worth it? “Oh, no shit, it's not worth it, and don't call me Phil. Because of such reasoning, we got the TSA and how they merge billions into the feeling of eggs and the confiscation of Knipser.
')

Translator's note: TSA - Transportation Security Administration - the US Transportation Security Administration, who is fiercely hated for phenomenal meticulousness during a search (particularly at airports).

Tests are not free

Each line of code you write has a price. It takes time to write it, it takes time to update it, it takes time to read and understand it. Accordingly, the benefit from its writing should be greater than its value. In the case of over-testing, this is definitely not the case.

Think of it this way: what is the price of preventing a bug? If you need to write 1000 test lines to catch Bob’s accidental deletion of the “ validates_presence_of: name ” line, is it worth it? Naturally not (yes, yes, if you are working on a missile launch control system for Mars and the rocket will fly to the White House, if you forget to give her a name, you can test it - but this is not your case, so forget it).

The problem of over-testing is that it’s hard to come up with a catchy term for it. There is nothing as compact as test-first, red-green, or other sexy terms that help promote development through testing to its well-deserved place in the center of the scene. Testing only what is necessary requires a delicate approach, experience and considerable intuition.

Seven "NOT" testing

All the subtleties can be easily explained in 2 hours of dinner with enlightened interlocutors, but there is not enough space in the post. So let's throw some firewood into the fire of discussion - I’ll just give you a list of opinions without any subtleties about how you should test your typical Rails application:

1. Do not aim for 100% test coverage;
2. the ratio of code to tests above 1: 2 already smacks, above 1: 3 - it stinks;
3. if testing takes more than 1/3 of your time, then most likely you are doing it wrong; you definitely do it wrong if testing takes more than half of your time;
4. Do not test the standard associations, validations and Osprey ActiveRecord;
5. save integration tests for problems arising from the integration of individual elements (i.e. do not use integration testing for things for which unit tests can be used);
6. Do not use Cucumber unless you live in the magical kingdom of non-programmer-writer-tests (or send me a bottle of fairy dust if you are still there!);
7. Do not force yourself to write the test first for each controller, model or template (my usual ratio is 20% of tests first and 80% of tests after).

Considering all those hundreds of books on how to start applying development through testing, I would like to have one or two books on how to tame this beast. There are a large number of subtleties in determining what is worth testing and what is not, but they are all lost when everyone focuses on the same examples of how to test.

But back to the point. We must collectively decide that TSA-style testing (theatrical quality of test coverage) has completely discredited itself before we can proceed further. Very few applications are so critical that they really require testing everything.

In the wise words of Kent Beck, the man who contributed most to the popularization of development through testing:

I get paid for code that works, not tests, so my philosophy is to test as little as possible to achieve the required level of confidence (I suspect that my level of confidence is higher than industry standards, but maybe it’s just my ego). If I usually do not allow errors of a certain type (like passing invalid arguments to the constructor, for example), then I do not test it.