How we evaluated the quality of the documentation
Hi, Habr! My name is Lesha, I am a systems analyst at one of Alfa-Bank’s product teams. Now I am developing a new Internet bank for legal entities and individual entrepreneurs.
And when you are an analyst, especially in such a channel, without documentation and dense work with it - nowhere. And the documentation is the thing that always raises a lot of questions. Why is the web application not described? Why does the specification indicate how the service should work, but does it work at all wrong? Why is the specification generally only able to be understood by two people, one of whom wrote it?
')
In this case, it is impossible to ignore the documentation for obvious reasons. And in order to simplify our lives, we decided to conduct an assessment of the quality of documentation. How exactly we did it and what conclusions we came to - under the cut.
In order not to repeat in the text “New Internet Bank” several dozen times, I will write the NIB. Now we have over a dozen teams working on the development of NIB for entrepreneurs and legal entities. In addition, each of them either from scratch creates its own documentation for a new service or web application, or makes changes to the current one. Can the documentation in principle be of high quality with this approach?
And to determine the quality of the documentation, we have identified three main characteristics.
Summarizing - complete, current and understandable documentation.
To assess the quality of the documentation, we decided to interview those who work with it directly: NIB analysts. Respondents were asked to rate 10 statements according to the scheme “On a scale from 1 to 5 (I completely disagree - I fully agree)”.
The allegations reflected the characteristics of quality documentation and the opinion of survey compilers on NIB documents.
It is clear that simply the answer “From 1 to 5” could not reveal the necessary details, so a person could leave a comment on each item.
We did all this through the corporate Slack - we just sent a system analyst a proposal to take a survey. There were 15 analysts (9 from Moscow and 6 from St. Petersburg). After the survey was completed, we formed an average estimate for each of the 10 statements, which was then normalized.
It turned out that's what.
The survey showed that although analysts are inclined to believe that the implementation of NIB applications is fully documented, they do not give a clear agreement (0.2). As a concrete example, they pointed out that a number of databases and queues from existing solutions were not covered with documentation. The developer is able to tell the analyst that not everything is documented. But the thesis that developers conduct reviews of documentation also did not receive unequivocal support (0.33). That is, the risk of incomplete description of the implemented solutions is preserved.
The simplicity of the actuality is that there is no unambiguous agreement again (0.13), analysts are still inclined to consider the documentation as relevant. Comments made it possible for us to understand that more often there are problems with relevance on the front than on the middle. About us, however, did not write anything.
Whether the analysts themselves understand when to write and update the documentation, the agreement was already much more uniform (1.33), including its design (1.07). What was noted here as an inconvenience is the lack of uniform rules for maintaining documentation. Therefore, in order not to include the “Who is in the forest, who for firewood” mode, they have to work on the basis of examples of already existing documentation. Hence, a useful Wishlist is to create a standard for maintaining documents, to develop templates for their parts.
Documentation on NIB applications is relevant at the time of delivery for functional support (0.73). It is understandable, because one of the criteria for the delivery of the project on the functional support - this is relevant documentation. It is also sufficient to understand the implementation (0.67), although sometimes questions remain.
But what the respondents did not agree with (rather unanimously) is that the documentation on NIB applications is in principle only needed for functional support (-1.53). Analysts as consumers of documentation were mentioned most often. The rest of the team (developers) are much less common. Moreover, analysts believe that developers do not use documentation to understand what they need to implement, though not unanimously (-0.06). This, by the way, is also expected under conditions when the development of code and the writing of documentation proceed in parallel.
To improve the quality of documents, we decided to do this:
All this should help raise the quality of documents to a new level.
At least I hope so.
And when you are an analyst, especially in such a channel, without documentation and dense work with it - nowhere. And the documentation is the thing that always raises a lot of questions. Why is the web application not described? Why does the specification indicate how the service should work, but does it work at all wrong? Why is the specification generally only able to be understood by two people, one of whom wrote it?
')
In this case, it is impossible to ignore the documentation for obvious reasons. And in order to simplify our lives, we decided to conduct an assessment of the quality of documentation. How exactly we did it and what conclusions we came to - under the cut.
Quality of documentation
In order not to repeat in the text “New Internet Bank” several dozen times, I will write the NIB. Now we have over a dozen teams working on the development of NIB for entrepreneurs and legal entities. In addition, each of them either from scratch creates its own documentation for a new service or web application, or makes changes to the current one. Can the documentation in principle be of high quality with this approach?
And to determine the quality of the documentation, we have identified three main characteristics.
- It must be complete. It sounds pretty captain, but it is important to note. It should describe in detail all the elements of the implemented solution.
- It must be relevant. That is, correspond to the current implementation of the solution itself.
- It should be understandable. That the person using it understood, how exactly the solution is implemented.
Summarizing - complete, current and understandable documentation.
Poll
To assess the quality of the documentation, we decided to interview those who work with it directly: NIB analysts. Respondents were asked to rate 10 statements according to the scheme “On a scale from 1 to 5 (I completely disagree - I fully agree)”.
The allegations reflected the characteristics of quality documentation and the opinion of survey compilers on NIB documents.
- Documentation on NIB applications is relevant and fully consistent with their implementation.
- The implementation of NIB applications is fully documented.
- Documentation on NIB applications is needed only for functional support.
- Documentation on NIB applications is relevant at the time of their delivery for functional support.
- NIB application developers use documentation to understand what they need to implement.
- Documentation on NIB applications is enough to understand how they are implemented.
- I timely update the NIB project documentation in the event that they are finalized (by my team).
- NIB application developers review documentation.
- I have a clear understanding of how to arrange NIB project documentation.
- I understand when to write / update the NIB project documentation.
It is clear that simply the answer “From 1 to 5” could not reveal the necessary details, so a person could leave a comment on each item.
We did all this through the corporate Slack - we just sent a system analyst a proposal to take a survey. There were 15 analysts (9 from Moscow and 6 from St. Petersburg). After the survey was completed, we formed an average estimate for each of the 10 statements, which was then normalized.
It turned out that's what.
The survey showed that although analysts are inclined to believe that the implementation of NIB applications is fully documented, they do not give a clear agreement (0.2). As a concrete example, they pointed out that a number of databases and queues from existing solutions were not covered with documentation. The developer is able to tell the analyst that not everything is documented. But the thesis that developers conduct reviews of documentation also did not receive unequivocal support (0.33). That is, the risk of incomplete description of the implemented solutions is preserved.
The simplicity of the actuality is that there is no unambiguous agreement again (0.13), analysts are still inclined to consider the documentation as relevant. Comments made it possible for us to understand that more often there are problems with relevance on the front than on the middle. About us, however, did not write anything.
Whether the analysts themselves understand when to write and update the documentation, the agreement was already much more uniform (1.33), including its design (1.07). What was noted here as an inconvenience is the lack of uniform rules for maintaining documentation. Therefore, in order not to include the “Who is in the forest, who for firewood” mode, they have to work on the basis of examples of already existing documentation. Hence, a useful Wishlist is to create a standard for maintaining documents, to develop templates for their parts.
Documentation on NIB applications is relevant at the time of delivery for functional support (0.73). It is understandable, because one of the criteria for the delivery of the project on the functional support - this is relevant documentation. It is also sufficient to understand the implementation (0.67), although sometimes questions remain.
But what the respondents did not agree with (rather unanimously) is that the documentation on NIB applications is in principle only needed for functional support (-1.53). Analysts as consumers of documentation were mentioned most often. The rest of the team (developers) are much less common. Moreover, analysts believe that developers do not use documentation to understand what they need to implement, though not unanimously (-0.06). This, by the way, is also expected under conditions when the development of code and the writing of documentation proceed in parallel.
What is the result and why do we need these numbers
To improve the quality of documents, we decided to do this:
- Ask the developer to review written documents.
- If possible, to timely update the documentation, the front - in the first place.
- Create and adopt a standard for documenting NIB projects so that everyone can quickly understand what elements of the systems and exactly how to describe. Well, develop appropriate templates.
All this should help raise the quality of documents to a new level.
At least I hope so.
Source: https://habr.com/ru/post/448382/
All Articles