Technical duty and its habitat

This article is a free retelling of the report that I attended at the GOTO Berlin 2017 conference: A Crystal Ball to Prioritize Technical Debt .

Images from the report and the rights to them belong to the author @AdamTornhill .

Each developer basically understands what technical debt is. That in his project this debt is certainly present. If you're lucky, he will remember a few pieces of code that have long been asked to be rewritten.
')
But how to formalize the concept of technical duty, to explain it to others? And, all the more, explain it to the manager in such a way as to get approval for refactoring? How to find all the places in the project that need to be rewritten in an amicable way, and how to determine which of them should be rewritten first?

If these questions repeatedly have arisen, I ask under the cat.

Not all clumsily written code is by definition a technical duty. Of course, if there is such a code, then it is better to rewrite it sooner or later. But we all know that you can polish the code almost indefinitely. How to determine which code is a technical debt?

A rather good description of the technical debt was given by Martin Fowler:

It’s a quick and dirty design choice.

That is, the more effort we devote during development due to a piece of code, the more technical debt it is . It’s hard to disagree, but still not enough to clearly define which places should be rewritten.

In order to assess how much each specific file / class / function spends our efforts in developing, Adam introduces the notion of hot spots, Hotspots. And to search for these hotspots you need only one tool that almost every developer has - a version control system.



You can estimate the amount of effort to support a file with a code by looking at how often this file changes, and at the complexity of this file. With the assessment of the frequency of changes, everything is unambiguous and clear. Difficulty can be assessed in different ways, depending on your preferences. In the simplest case, this may be the file size or the number of lines of code. Other things being equal, maintaining a file with 100 lines of code is much simpler than a file with 1000 lines of code. If the file size in your case is not a criterion for assessing the complexity, you can use various utilities for static complexity estimation (for example, cyclomatic).

Then hotspots can be identified as follows:



Here is an example of finding hot spots in the Tomcat project:



The big blue circles are folders. Small files.
With all this, the presence of hotspot does not mean that this file is problematic (but most often it is). This means that you spend the most time in these files. And that when refactoring these files should be first in the list to make sure that the code is clean, easily maintained and extensible there.

Also, as an example, graphs of code analysis of several projects are given, as different as possible. Different languages, different times of life, different development companies. On the X axis we have files, on the Y axis - the frequency of their changes.



All projects have the same pattern. Most of the code is located in the “tail” of the graph. And accordingly, there is a very small part of the files that change very often. These files are also the first candidates for refactoring.

When searching for hotspots you can go deeper. And in the problem files to look for hotspots already at the level of individual functions:



The tool that was used to find such hotspots is codescene.io

Also useful practice is to regularly monitor the complexity of your code. The analysis of such data in the form of graphs will very clearly show at what points you have gone astray.

Results

I found this report useful primarily because of the clear definition of technical debt and its size.

It is clear that to engage in such a serious analysis of the code with the identification of hotspots, it must be very much bogged down in technical debts. But even at the basic level, in my daily work, I began to pay attention to the classes that I most often touch on, and I try to refactor such classes slowly.

Also, as a bonus (there was a question from the audience on the report), Adam told how to convey the need for refactoring to the management correctly. Most often, managers are not technical people, rather far from the code, otherwise there would be no problems. But what these people understand well is numbers and graphics. To properly convey information to them, you need to speak the same language with them. And just graphics with hotspots and temporal dependence of the complexity of the code can help here. On the graphs it can be shown that such a functional, which we added recently, greatly complicated the addition of new features. Therefore, if we want to further accelerate the pace of development, we need to spend some time refactoring.

Useful links:

• New Adam's book about code and architecture analysis
• An article in which the concept of hotspots is also described, but, as it seemed to me, is described side by side. I also wanted to focus on the search for problem areas and their connection with technical debt
• Video of the report itself