Cost of quality in software development

1. What is quality in software development?
2. How much does poor software cost us?
3. Who is responsible for the quality?

For me, an occasion to ask these questions was a meeting with a company in which the entire development department (about a hundred people) spent 3 months a year, was busy eliminating errors and defects, and the remaining 9 months they write software errors for Customers.

Below are the results of my theoretical and practical research in search of answers. I tried to present them simply, without the “brainstorming” inherent in this topic.

What is quality in software development?

The question what quality is in software development can be answered from different points of view. I will formulate my answers from the point of view of the Customer User and the Head of the company.
')
From the position of the User, high quality software is the complete absence of errors and the solution to the problem for which it was created.

The minimum level of quality is the solution of a business problem and the absence of software errors that block the possibility of its use ( Blocking errors ).

The average quality level is all that is higher + the absence of errors that significantly affect the experience of using the software ( Mandatory to eliminate ), while there are errors that do not affect the solution of the business problem: spelling, different fonts where there should be one and .d ( Desirable to eliminate ).

Errors that are found by users during the operation will be called Defects. The errors will be considered as what was found by the employees involved in the release of the software version, before transferring it to operation for the Users.

How much does low-quality software cost us?

Part 1: Defect Verification

When the user finds a defect, he contacts the support engineer. The JV engineer is trying to reproduce it and confirm that it is a manufacturing defect. Here are the first costs from which the cost of quality begins to take shape:

• User time costs - 1 unit.
• The time spent by the SP Engineer is 1 unit.

If the error is systemic, then everything is obvious, and if it is related to the work of the functional according to a certain logic?

Here, the SP engineer will need documentation that describes how the system’s reference response should be when it performs a specific user request. If this documentation is not available, then additional costs are required to ascertain whether this is a defect or a new requirement.

The lack of documentation is especially common when fast development is practiced. Everyone remembers “Working product is more important than exhaustive documentation” , but they forget about what is written below “That is, without denying the importance of what is on the right, we still more appreciate what is on the left” .

The joint venture engineer begins to put extra effort into finding out the truth. The owner of the Product (if any) or the Development Engineer who has done this function or knows this function module are connected to the investigation. One or both of them are trying to remember or identify the Customer for whom this is done.

The user of the system is not always the Customer who formulated the requirements for the operation of the system. The fact that it is not normal for the User for the Customer may be the norm.

Together with the customer, they discuss the situation and try to determine whether the system is working properly or abnormally. If the customer has amnesia, and this happens if you ordered the work 3 months ago (a long backlog with work half a year ahead), and they did this feature / function just now, they try to find some document or letter on the basis of which the development was made.

If they are lucky to find the answer to their question, then everything ends with the following costs:

• Product Owner - 1 unit.
• Development Engineer - 1 unit.
• Customer - 1 unit.
• The time spent by the SP Engineer is 1 unit.

If you are unlucky, they will be bargained over whether it should be considered a defect that will be fixed at their own expense, or a new requirement that the Customer will pay. Assume that we could not bargain, the Customer has pressed, we consider this a defect and eliminate at our own expense.

• Reputation Impairment - 1. (due to the fact that Defect and that the Customer had to be involved in the investigation)
• While the cost of eliminating the defect is not considered here, since they will be added a little later.

Subtotal:

1. User time costs - 1 unit.
2. The time spent by the SP Engineer is 2 units.
3. The time cost of the Product Owner is 1 unit.
4. The time of the Engineer-Developer - 1 unit.
5. Customer time costs - 1 unit.
6. Reputation Impairment - 1.

Cost of quality - 7 units

You can optimize costs in this part by spending in advance on the preparation of user documentation by Tech. Writer-Analyst, then it will look like this:

1. User time costs - 1 unit.
2. The time spent by the SP Engineer is 1 unit.
3. Time spent by Tech. Writer-Analytics - 1 unit.
4. Reputation Impairment - 0.5 units. (Because Defect)

Cost of quality - 3.5 units

In order to understand the moment when TechTeacher Writer-Analyst is needed, you need to start keeping records of situations when additional persons were involved in the investigation besides the SP Engineer and how much time they spent on it, how many such controversial situations turned into Defects and how much their removal cost. When the amount of these costs becomes more than the salary of Tech. Writer, it's time to introduce it.

Part number 2: Defects Elimination

Each “Blocking error” generates a release of a separate patching update. This update passes through standard testing procedures in order not to generate new errors. It is transmitted to the SP Engineer for distribution to Clients who use the software version in which this error was found and to close the defect “ticket”. Expenses:

• Development Engineer - 1 unit.
• Testing Engineer - 1 unit.
• SP Engineer - 1 unit.

The “mandatory bug fix” is eliminated in the next planned release (version), so the costs for fixing are lower because the cost of the release build and regression testing are on new features and are not included in the cost of fixing the bug. There are also no additional costs for the work of the Engineer of the joint venture other than closing the ticket. Expenses:

• Development Engineer - 0.5 units.
• Testing Engineer - 0.5 units.

Costs in both cases are paid from the company's profits.
Cost of quality - 4 units.

Part number 3: Preventing Defects (Quality based on control)

So that Users do not find Defects for this, have their own Testing Engineers. For them, the criterion of efficiency of 100% errors is 0% of defects.

Information about the defect found is transmitted to the Testing Engineer to analyze the reasons for missing a defect during testing and to make changes to the test methods and tools. Expenses:

• Engineer Tester - 1 unit.

Efficient work here negates the following time costs.
Part 1:

1. User - 1 unit.
2. SP Engineer - 1 unit.
3. Reputation Impairment - 0.5 units.

Part 2:

1. Development Engineer - 1.5 units.
2. Testing Engineer - 1.5 units.
3. SP Engineer - 1 unit.

Cost - 6.5 units. Attachments - 1 unit. Savings - 5.5 units.

Part number 4: Troubleshooting Errors

Each error found on its own, does not cost the company anything, since the costs of elimination fall into the final cost for the Client.

But the elimination of errors increases the cost of the product and the timing of the work, if the development makes a lot of heavy mistakes, and also have to perform more than 2 iterations of testing and eliminating errors. The Client does not like all this, and especially if he has something to compare with.

If the development is carried out for its own needs, then a large number of errors worsens the economy of returning investments to the Development and reduces the possibility of releasing more useful functions (features) per unit of time, which already negatively affects the economy of the entire business of the company.

Costs here:

1. Reputation Impairment - 1.
2. Development Engineer - 1 unit.
3. Testing Engineer - 1 unit.
4. Lost income - 2 units. (p.2 + p.3)

Cost of quality - 5 units.

Part number 5: Avoiding Errors (Quality based on the development of the production process)

The quality of the product needs not only and not so much control as to prevent the causes that directly affect the appearance of errors.

The mistakes that have been made are passed on to the Development Manager (Tim-Lida) for analysis in order to identify the causes, employees who make them more often, and more often “ill” products. This work results in a series of events on code review, additional training for development engineers on topics where there is a lack of qualifications, etc.

• Manager Development Costs - 1 unit.
• Development Engineer Costs - 1 unit.

Efficient work here negates the following Part 4 costs:

1. Development Engineer - 1 unit.
2. Testing Engineer - 1 unit.
3. Lost income - 2 units.
4. Reputation Impairment - 1.

Cost - 5 units. Attachments - 2 units. Savings - 3 units.

General calculation of the cost of quality

Usually the situation is as follows: We are developing, making mistakes, sometimes eliminating them requires more than two iterations and despite this work, users still find defects in the software that we have to fix by issuing additional updates.

Cost type	Part 1	Part 2	Part 4	Total
User time spent	one	-	-	one
Time spent by JV Engineer	2	one	-	3
Product Owner Costs	one	-	-	one
Development Engineer Time Costs	one	1.5	one	3.5
Customer time	one	-	-	one
The time spent by the Testing Engineer	-	1.5	one	2.5
Reputation degradation	one	-	one	2
Lost income	-	-	2	2
Total	7	four	five	sixteen

If we prepare documentation on the fact of development, having invested 1 unit of Technical Writer's resource, this will save 3.5 units and the cost of quality will decrease from 16 to 12.5. (Cost structure: Documenting + Defect Verification + Defect Elimination + Error Correction)

Having ensured non-admission of defects, having enclosed in it additional expenses of time of the Engineer-tester in the amount of 1 unit, the cost of quality will decrease from 12.5 to 7 units. (Cost Structure: Documenting + Non-Defecting + Troubleshooting)

Avoiding errors will require an investment of 2 units, one each from the Development Engineer and the Development Manager, and reduce the cost of quality from 7 to 4 units. (Cost Structure: Documenting + Non-Admitting Defects + Avoiding Errors)

If we invest in the fact that the Technical Writer - Analyst will begin to write user documentation about how the system should solve the business problem of the User before development, it may not be necessary:

1. Not only to argue with the customer whether it is a defect or not, but also to communicate with him about defects in general.
2. Analyze the reasons for which errors in the implementation of software targets have slipped to the Users through the Testing phase.
3. Analyze the reasons for which the Developer has done so that the program does not work as it should and gave it to Testing.
4. Spent on additional measures to prevent errors and defects, when the project documentation is missing and something needs to be invented to compensate for its absence.

And it needs only 1 unit of costs. ( Quality based on the development of the product design process )

Who is responsible for the quality?

Western companies have such a Quality Assurance Engineer position, which is responsible for the entire software production process and focuses on the quality assurance aspect.

We have Software Quality Assurance ( Software Quality Assurance ) exaggerated before being perceived as Testing, and Quality Assurance Engineer as a Testing Engineer. But the Testing Engineer is the maximum that can do is find all the errors and return them to the Software Engineer (Developer Engineer), but it does not affect their appearance.

The guys in the development can not make mistakes, but they are not robots, this is a creative process when you need to come up with something that was not there before and there is no place to look at how this can be done.

This is also sometimes complicated by the fact that, as input, a vague statement comes to them, the reason for which lies either in the fact that the Customer does not understand what he wants and someone missed this requirement before Development, or the Customer knows what he wants, but it worked “broken telephone ”and part of the requirements did not reach Development.

Team Lead (Development Manager) can help the Development Engineer learn how to do his job well and master the missing competencies and technologies, but he definitely should not be the Internal Tester of the development team and check each one in order not to skip the defective code to the next stage. Its task is to learn and build processes within the team so that there is no defective code.

There is also the Product Owner , who as the owner is suitable for the role of the Product Responsible for Quality. For this reason, they are sometimes divided into two positions by the Marketing Product Manager ( Product Manager ) and Software Delivery Manager (Software Release Manager). The first is responsible for the development of the product from the point of view of the market (working with customers and requirements), and the second for the development of the production process of product release.

If you look closely at everyone in the list, you can see that none of them personally produce “Quality”. Quality is not a program component that can be added to it. At the same time, they all participate in the creation of the Product, which solves the user's business problem and does not contain errors. Quality is teamwork .

It is impossible to divide responsibility into stages of the Software Development Life Cycle . As soon as this division begins, the inefficient work in the middle immediately makes the work done at the beginning meaningless and loads with additional work both those who stand at the end and the entire production chain starting an error at its input to eliminate it.

Conclusion

This is a classic approach to working with quality from industry, which is based on three methods:

1. Quality through the verification of the finished product (testing: trapping errors and defects)
2. Quality through the development of the production process (prevention of errors during production)
3. Quality through the development of the product design process (the prevention of errors laid into the skeleton of the product when designing)

And the main message: it is necessary to pay more attention to working with causes, and not working with consequences .

It is great for the development of products whose work can be designed in and out. And it does not work when there is no time to design the work of the product and the definition of how it should work occurs through the iterative release of working versions (empirically).

At the beginning of a fast iterative development, there is a single quality criterion - this is the fulfillment by a product of a business task for which it is created. With each iteration, the Customer adds new quality criteria (expectations from the product), which improve his experience in using the product to solve a business problem.

With the use of fast Agile frameworks, the number of errors and defects per unit of time increases in proportion to the number of releases. The more we do and release, the more mistakes we make. The more mistakes we make, the greater the percentage of the command load associated with correcting the mistakes made and reworking what has been done.

This is a fee for completing the aircraft already in flight, having no idea how it should look like in its finished form and respond to environmental conditions.

What to do with it? - I like the answers given in the article “The Future of Agile Development”.

What else to read on:

1. The future of agile development
2. What is software quality?
3. Project Quality Management
4. Career in IT: QA engineer
5. Guru of quality management and their concepts: E. Deming, J. Juran, F. Crosby, K. Ishikawa, A. Feigenbaum, T. Taguchi

Take a few minutes to answer three questions, so you can help diagnose the average temperature of attitude and attention to software quality in the IT industry in our country on August 1, 2017. +1 you in karma for this good deed.