The Pros & Cons of Test-Driven Development

The conversation led IvanPonomarev

Test-driven development (TDD) is a practice known for quite some time. Development through short cycles “first of all we write a unit test, then a code, then we refactor, we repeat” in a number of companies it has been adopted as a standard. But does a team that has reached a good degree of maturity of the development process have to accept TDD? As with most other Extreme Programming practices, the controversy over TDD is still ongoing. Is the initial cost of training and implementation of TDD justified? Does TDD give a tangible win? Can this winnings be measured? Are there any cases where TDD harms the project? Are there any situations when it is simply impossible to solve the problem without TDD?
')
We talked about this with expert developers Andrei Solntsev asolntsev (the developer from Tallinn-based company Codeborne, who practices Extreme Programming and adheres to TDD) and Tagir Valeev lany (the developer at JetBrains, also develops the open-source StreamEx library and Java Byntcode Analyzer Java HuntBugs; he is convinced that byte-code Java HuntBugs; - useless practice). Interesting? Welcome under the cut!

JUG.ru Group:
- Good afternoon, Andrei, good afternoon, Tagir! Tallinn, Moscow and Novosibirsk on Skype. And so that your position becomes clear to readers, the first question is short: Do you work on TDD?

A. Solntsev:
- Yes, we work on test-driven development every day. That is, we write the test and code. And I think this is a very useful and correct practice. Ideally, almost everyone should work in such a way as to be effective.



T. Valeev:
- I do not work on test-driven development in the sense in which Andrew works. For the new functionality, I definitely write the code first, and then only tests for it. In the case of bug fixes, 50/50: if I already have a ready code and report came to me that it falls somewhere, I can write a test that reproduces this problem, and then fix it, or I can fix it first, and then write a test. But with unit tests, my code is still covered. I will cover all the new code that I write with unit tests.

JUG.ru Group:
- Test-driven development is known for a very long time, since the beginning of the two thousandth. However, for the most part, developers do not work on TDD, in my feeling. Why is this happening? I asked this question to Nikolai Alimenkov xpinjection . He gave two reasons. The first - just can not. Nobody taught them how to do it right. And the second - they mistakenly consider themselves cool architects: “Now I’ll quickly get some patterns out of the patterns, and it will immediately work.” What is your opinion? Why is TDD not used massively?

A. Solntsev:
- I agree with Nikolai. Indeed it is. Do not know how. And here the trouble is that it is not enough to read any manual or book to be able to: it does not work. You don't care when you try, you do something wrong. For example, when I first read about unit tests and started using them in my working draft, then after the fact I realized that I was doing this absolutely, fundamentally wrong. And all that I have done for the year, you can throw.

JUG.ru Group:
- Tagir, why don't you work on TDD?

T. Valeev:
- I just think that it is not necessary. That is, I understand the exceptional importance of unit tests, the importance of good test coverage, but writing tests ahead of the code is a waste of time and resources. The result of this is not getting better, and the time for this will be spent, most likely, more.

Quick fix vs. Code completion

A. Solntsev:
- Why do you think that is so? If you still write exactly the same number of tests, then why, when you write them first, does it take longer?

T. Valeev:
- At a minimum, you will not have code completion in the IDE. If you write a test for which there is no code, then you cannot write it effectively enough.

A. Solntsev:
- Well, this is easily solved. You do not write the test completely. You first write a call to a method that does not exist. Yes, here you do not have code completion, and this is good, because then you will think about how the method will be called. I wrote a call to the method - you can press Alt + Enter, an empty method will be generated in the IDE. Everything, further you will have code completion.

JUG.ru Group:
- Moreover, you will think up not just how it will be called, but what parameters it will have and how to call it in the most convenient way.

T. Valeev:
- I do not agree that in order to generate code, using quick-fixes is easier than using code completion. But perhaps this is the taste.

A. Solntsev:
- Not easier, but about the same in both and. Alt + Enter to press and in that, and in that case.

T. Valeev:
- No, I think using quick-fixs is harder. I will spend more time. And you need to press not only Alt + Enter. For example, I call such and such a method in a test and, say, I think that with such a string and with such a numerical parameter I should produce such a value. I wrote an assert. I still have no method and no class either. Thus, I must first say: "Create a class that is not yet." Choose a package for it. That is, I did not just press Alt + Enter, I also filled out the package in the dialogue, the scope of the class ...

A. Solntsev:
- So all of you will do all these things anyway, without a difference.

T. Valeev:
- But I will write them in a text editor, and not in dialogue. Well, with a class understood, here is the name of the method. In this case, I press Alt + Enter and get a dialog in which I am prompted to fill in the name of the parameters. Since my test contains a string and a number, the IDE does not tell me how these parameters should be called. That is, I will have to enter their names in the dialogue. Perhaps she also incorrectly guesses their types, especially if I assume that I have a complex type of some kind there. It's much easier for me to manually type it in the editor.

A. Solntsev:
- I will repeat once again, it is the same in time. That is, you will still have to end up, if your parameter name consists of ten characters, press ten keys. In any case TDD or non TDD, it will be exactly the same.

T. Valeev:
- There are two options. First: in the dialogue I will leave everything as it is, and then in the text editor I will simply edit what I had been acquired (or, in the dialogue, correct what I had been prepared). Second: I will simply enter in the empty text file immediately what I want, without correcting.

A. Solntsev:
- Absolutely no difference.

T. Valeev:
- In my opinion, there is a difference.

Usability vs. implementation efficiency

JUG.ru Group:
- But, colleagues, all the same for TDD says another fact. If you try to write a test first, you first have to do a setup. A test is not only a method call, it is also a setup. With the help of a constructor or a factory, are you going to create a test object? In the constructor or in the properties, will you pass some parameters? Where do you get these parameters? And this moment is very valuable, because if you immediately start thinking from the perspective of how to use it, then you get a more beautiful API.

A. Solntsev:
- Absolutely, I agree.

T. Valeev:
- This is a very good topic. And here it is just possible to argue, because any API should be considered both in terms of ease of use, and in terms of ease of implementation. And what's more, efficiency and implementation opportunities in general, because it may turn out that the API, which is convenient from the point of view of use, is inconvenient from the point of view of implementation because it simply lacks data. I, for example, write IDE. This is what I really do now. For example, I need a new method that the class will find for me. I want by the name “java.util.Collection” I find the class and find out what methods there are. I naturally think: “What do I need? I need a name with a String type. " Well, I am writing, say, the method findClass (String name), passing it the string “java.util.Collection” and checking that it should find something. Good, convenient test? Much easier. But when you start to implement, you will understand that “java.util.Collection” is not clear that, because you, for example, in different modules or in different projects can have different JDKs connected and this name can correspond to different classes, in which a different number of methods. When you implement it, you will not be able to not think about it, because you will immediately understand that there is a problem. You need a binding to the project or to some sort of resolve scope. Accordingly, in your convenient way of using there is simply not enough data to realize the result. So writing a test will not make a good API. In this case, we wrote a test, it was convenient for us to use it, but the API turned out to be bad.

A. Solntsev:
- So what? Not enough data, it is, but I do not see any problem here. You start writing with a test. I wrote the test as it would be most convenient to use it. As you realize, you see that there is not enough data, you go back a step and you understand: “Ok, so impossible, we need new data.” And you again complement the test and think about how the most convenient way to transfer this data to this method through the designer, via the injection service, as you please. Accordingly, you supplement the test to transfer this data, and you begin to implement further. I do not see the problem. Everything is just like that. Evolutionary design .

T. Valeev:
- The fact of the matter is that you don’t see the problem, but I don’t see, why write a test, then to see that this test is useless, that it won’t work? Why take these steps forward, steps back, when you can immediately write an implementation? And it will be done in the most convenient way and never have to rewrite the test. Immediately write the implementation, then the test. Not a single step back you do. It is effective.

A. Solntsev:
- Effective for whom? We return to the very basics: why do we need TDD at all? It allows you to think in advance how to use the API in the most convenient way. In your example you mentioned that you need to transfer, first, the name of the class, plus you need to somehow transfer additional parameters, say, a project, a Java version — you need to transfer some context. And how to convey it - there are different options. You can parameter the method. You can inject it into this class, into the service. You can inject through the constructor. You can make it just jerk some static variable or static method from somewhere. Or would load them from base, damn it. That is, there are different options. And when you start to do the setup in the test, as Ivan mentioned already, you start thinking through this moment: what is the most convenient way to transfer all these things there?

T. Valeev:
- By the way, you said a very interesting thing: "Static methods." That is, I have some kind of static context somewhere outside. And it may turn out that I’m really going to write this findClass method, it will have only one string parameter, and I’ll think: “Actually, you can work with this method too”. That is, implementation is possible if I can get somewhere from the global context, which project I have, JDK, and so on. But over time, it may be that maintaining this global context carries more overhead. It’s just that in a particular place it becomes already incomprehensible what the context is now. Here multithreading emerges: I have one context in one thread, another context in another stream. That is, starting from considerations to make it convenient to use, if I pay too much attention to this, it may turn out that the implementation will become unsupported in general.

A. Solntsev:
- You called it right. There are such problems. This is what happens in many projects. And I just want to emphasize that the test will reveal this much earlier, as soon as you understand in your test that you now need to somehow initialize the global context before running the test. Some kind of static variable to do. You will immediately see: “Oh, this is inconvenient. What if I launch two different tests and each has to be initiated in its own way, for example, in parallel? ”And instead of initiating a static variable, it is somehow better and more convenient to pass it, for example, as a parameter. This test will reveal instantly. In fact of the matter.

JUG.ru Group:
- Yes, from my practice: if the code becomes difficult for unit testing, then most often it happens because of some global contexts. This is where the signal sounds that we have not designed a very reasonable API, not even from the point of view of convenience, but from the point of view of general correctness. That is how problems arise. And if I need to run a test for different global contexts, but is it difficult for me to capture them? If, as a matter of fact, it turns out that the unit test generally depends on the software environment installed on the developer’s machine? So the problem is revealed that we have not thought about something. What are some things that we should pass as parameters, for example, they really depend on global contexts.

T. Valeev:
- But I don’t argue with the latest replicas! You do not forget that I am absolutely not against unit tests. I am hot for unit tests. That is, with the latest replicas in the context of the fact that we already have tests and code, and at some point we understand that something is wrong - I absolutely agree. Test and code should be. But we only have differences about the order in which they should appear. The position of Andrei, as I understand it, is that if you write the test first, then we move a little towards usability. My position is that if you do not write a test first, then we will be shifted towards ease of development.

A. Solntsev:
- We will not. We will not be shifted. We will not be in any way shifted towards ease of development. It is not true. Writing a test before does not interfere with the development in any way, no. This is a false assumption.

T. Valeev:
- Well, you have such an opinion, but I have another.

Cheaper, better - or immediately and then, and more?

JUG.ru Group:
- Before our conversation, I thought that he could go into a phase where everyone would insist on their subjective position. But is it possible here to have an objective view, is it possible to measure somehow the productivity of a team working in one way or another? I searched Google and found a link to research that was conducted by Microsoft and IBM. Two teams were forced to work in parallel on the same project, some worked on TDD, others did not work on TDD. And the conclusion was such that TDD labor costs are slightly higher, and TDD quality is significantly higher. That is, with slightly higher labor costs (this is an important point, they noted that the labor costs for developing TDD are higher), judging by the number of defects that had to be corrected later, with TDD, the quality is much higher. That is, perhaps the choice between TDD and non-TDD is the usual choice between price and quality.

A. Solntsev:
- I will comment on the price and quality. Slightly more cost was calculated at some first stage when development is underway. But do not forget that the project does not die on it. The project continues to live. It must be supported, then somehow developed. And a project that is of worse quality will require much more time and effort to support, more bugs will be, more difficult will be refactoring and so on. And so in the end, after a certain amount of time, it will be more expensive. Therefore, to say that when TDD will be higher costs - incorrect. In the long run, they are smaller. This is the same as buying a cheaper coat right here and now. TDD is an expensive coat. Right here and now you will buy a cheap coat without TDD: yes, now it seems that it is cheap, but in two years you will find that you have to buy a new coat every year.

JUG.ru Group:
- Unless you are so cool that you can write excellent code right away without TDD?

T. Valeev:
- I, of course, do not write wonderful code right away. I write at once more or less good code, then I write a unit test. And after that, I fix this more or less good code in accordance with the fallen unit tests. And then I send it also to code review and finalize it after that.

A. Solntsev:
- But, nevertheless, it turns out that there are still some repeated changes anyway?

T. Valeev:
- Multiple changes in any case will be. What's the difference? You write the test first, then you write the code. After that, you still have to run this bundle. You will find that the test fails, you will correct it. The same is with me, I just write the code first, then the test. And then I launch, further the same. To think that you write, all the same it is necessary.

Hold the task: "in the head" on paper or in tests?

A. Solntsev:
- I would like to say what I see as the main benefit of TDD. I see in this a tool that helps to think. That is, as an alternative to TDD, you can, for example, think through in advance in your head what you will do. And I believe that this is what many people do. They think: “Why do I need to write tests ahead, I can even think things through in my head”. Maybe Tagir just thinks so. And, maybe, it is good until a certain moment, not a lot of changes, and when a good fresh head. Indeed, in my head you can think over everything in some detail, but starting from a certain point the head can no longer cope. The second option is to draw on a piece of paper in advance, and it may be another option to discuss it with someone. That is, these are all options that allow you to think in advance about what you will write, but, in my opinion, of these, the unit test is the best, because it is closest to the code. All the same: if you think in your head and it seems to you that everything was thought out perfectly, then when you sit down and start writing code, it turns out nuances that you haven’t thought of. The same on a piece of paper: drew everything in detail, you start to write code, it turns out the nuances that you have not thought of.

T. Valeev:
- I just gave an example with findClass (...) to show that even if you write a test, you still find out the nuances that you did not think about.

A. Solntsev:
- You are right: yes, the nuances, of course, in any case become clear, undoubtedly. And when it turns out that you didn’t think about something - which option? Back and back further in my head to think again? The head will simply swell up and break, will not be able to think so much. Option to go back and then redraw everything on paper? This is an option, in principle, but in my opinion, it is much more efficient and faster to return to the unit test and finish it. It will be easier than redrawing everything on paper. Easier, faster, more efficient. I want to draw an analogy: a unit test, a piece of paper and thinking through in my head are about the same tools.

T. Valeev:
- If the task is too complicated to keep it entirely in mind, then I do not think that TDD will take it and will save everyone in a magical way. The task should be divided into subtasks. I solved a small subtask, tested it - well. This piece in your head is no longer detailed, it has become a working abstraction. Then you work on some new brick that uses the previous one.

A. Solntsev:
- This is also a method. I absolutely do not argue with that. Yes, it is necessary to break. No controversy. But the fact is that when you put so many such small bricks in your head per day, your head gets tired much faster. The question is exactly where to keep this workspace: in your head, on a piece of paper or in a test.

Asserts in the code - TDD or not?

JUG.ru Group:
- I myself do not do everything on TDD, although I strive for this. But in my practice there were some difficult tasks that I would not have “caught through” if I had not solved them right away using TDD. There are tasks whose solution, as I imagine, looks like the interaction of a large number of small “gears”. Algorithmically or structurally complex tasks. First, I have to be absolutely sure that each “gear” works correctly. Then I try to launch these “gears” into some kind of complex mechanism and achieve their correct interaction at a higher level. Perhaps this is a matter of inner conviction, but I’m sure that some tasks without TDD wouldn’t work for me. But this, perhaps, as anyone. Someone, perhaps, would have decided. Here you are, Tagir, how do you solve the most algorithmically complex problems?

T. Valeev:
- The main approach is, of course, splitting tasks into simpler ones. That is, you need to highlight some steps. At each step, you must have invariants. These are some statements, some assertions, which in this step must be true. It may be even easier to prototype it not with the help of unit tests, but by placing asserts in the code, that in this place I have such and such an invariant, in this place - such and such.

A. Solntsev:
- So it's the same. Asserts are the same as tests.

JUG.ru Group:
- If we think over invariants and turn them into asserts in the code, then this is the same kind of work that creating asserts in unit tests. Do you use asserts in the code?

T. Valeev:
- I use them precisely in the case of algorithmically complex problems. And, as a rule, I still delete them. But it depends on the situation, on the overall policy of the project. For example, in my library, StreamEx, I had straightforwardly fuzzy pieces, where I needed to be parallelized, and for all this to be correct. There were a million special cases where the order in which threads could come to a certain point. In addition to unit tests, I set up asserts to test not only the external API, but also some internal pieces. All this debugged far and wide. When it was necessary to make a release, I cleaned asserts. I did this for some reason, including because they can slow down if they are enabled at runtime.

JUG.ru Group:
- And they can also bring side effects.

T. Valeev:
- Well, these are bad asserts if they contribute. But the performance they can squander. If a person keeps “-ea” for the whole project, then why did he get an assertion that worked inside my code? He still does not benefit from it.

A. Solntsev:
- Yes, I agree that assertions are after all actually the same unit tests, just written elsewhere. Only with unit tests, plus the fact that they do not need to be removed, and they have no effect on performance. And since you write asserts, and then unit tests, then you are doing double work?

T. Valeev:
- In the process of prototyping, it is sometimes easier to write asserts, because when you prototype an algorithm, you can put an assertion, for example, put a loop inside, and then figure out how, for example, put the body of this loop into a separate method and open it for the unit test.

A. Solntsev:
- So we are actually talking about almost the same thing, just call it a little differently. But in my opinion, a unit test is more correct than an assert, if only because you think about it beforehand. And then do not have to redo it.

T. Valeev:
- If this is not about the external API, but about the internal structure of the algorithm, then how it is divided into methods is not very important, because no one sees this. The main thing is that it is correct and fast. And it may well be that all those places where you want to put asserts, if you break into these methods methods and put unit tests, the code will be too turned into noodles and it will be harder to perceive. I'm talking about algorithmically complex code. But here, perhaps, too, tastes. It seems to someone that a hundred single-line methods are better, and to someone that ten ten-lines are also good.

Writing interfaces before their implementation - TDD or not?

T. Valeev:
- And I also want to add regarding the design of the API. This would seem to be such a thing, for which TDD is really very useful, because if a new API is created, then we will immediately think about how to use it conveniently. And from my practice, if I really do not understand in my head how convenient it is, this way or that, when I have options, then my approach is probably the closest to classical TDD, but I still do not write a test ahead. I write only interfaces without implementation, and I write tests for these interfaces. . , . . , . , , , .

. :
– . .

. :
– , , .

. :
– . , Alt + Enter , , -, .

. :
– ! , . , , , , , , .

. :
- Yes. , .

, TDD ?

JUG.ru Group:
– , . , , « TDD? TDD – ?», - : «, ». , - — . , -, , , -, -. — , .

. :
- Yes!

. :
– , , . , , - - . , , .

, TDD ? TDD , , . , TDD – , TDD . , , . - , . , TDD, .

. :
– , , , . , , . , . : « , TDD ?» — . , , , TDD , . , , : «». , , , . , TDD – .

. :
– , , . , code review. , , — ?

. :
– , , . , : , , . .

. :
– , , .

TDD?

. :
– , , . . , , . : , , , . TDD, TDD . . .

JUG.ru Group:
– . . TDD, TDD, .

. :
– , . , . . — . , ? ? ?

. :
– , , Microsoft - , TDD : - . , . , . .

. :
- Yes of course. , , - - ? . .

JUG.ru Group:
– – . , . — — , , . : -, , , - . . . , — , , . - , TDD, TDD — . , .

, . .

. :
– , . .

. :
– . Mutually!

---

, , 10 «Radisson » ( ), .

• No Such Thing as Manual Testing and Other Confusions
• Appium: Automation for Apps
• How to teach robots to play games?
• Tips' Architecture Designs
• Page Objects - less is better
• Testing Distributed Systems
• Testing Android – Juno apps with: CI, Unit, Integration, and Functional (UI) tests. 100% Kotlin, 90% + RxJava, Spek, JUnit, DSL for UI tests
• Combining manual and automated testing: process and tools
• Shopping list: what you need to remember when running JMeter tests
• Static brain removal: what do code analyzers hide?