Project management software development. Problems and solutions

Methodology Funnel In 2001, when there was still no Habr and a significant share of his modern readers, when the waterfall was omnipotent, and Edgeel was just beginning to speak, I did a little research on the subject of development methodologies and their differences from each other. The result was an article that was published on friendly websites. Some respected educational institutions even referred to the article when preparing courses on the basics of software project management. Since the friendly websites were not about IT, then the article eventually disappeared from them. In order to prevent its complete disappearance from the Runet, I will allow myself to publish it on Habré and invite everyone to take a short excursion into the past. Yes, many things now seem naive, but a number of conclusions are still more than relevant.

Attention, article from 2001 !

The article attempts to classify software development projects, reviewed and briefly analyzed the main methodologies for software production, made a brief analysis of the applicability of the main methodologies to the types of software development projects.

Introduction

Today, there is a fairly large number of standard processes and methodologies, applying which the firm receives one or another software production model. The most famous of these are the CMM (Capability Maturity Model) and the ISO 9000 series of standards. As a rule, organizations implement these processes exclusively in order to receive a certificate of conformity of their process to one of the above standards. However, paradoxically, it sounds, often attempts to introduce one of the above processes adversely affect the stability of the company.

In the past 3 years, the terms “lightweight process”, “adaptive process”, “unified rational process” and “extreme programming” have become fashionable in the west. What it is? And why does the effect of introducing ISO9000 sometimes turn out to be negative? What process to choose and what criteria to follow when choosing a process? The rest of the article is an attempt to find answers to these and other questions.

Project classification

The model and parameters of the software production process largely depend on the type of project. Each team has its customers with whom these or other relationships have developed.

Own customer

The situation in which the development team for a long time serves a single customer. As a rule, this option is characterized by excellent relations between the customer and the developers. Often these teams are located on the customer’s premises. The main characteristics of this type of project:

availability of the customer at any time;
sustainable understanding of the needs and requirements of the customer, formed over the years of cooperation;
fairly low quality requirements for software;
constant support of their products, their refinement and improvement;
stable financial relationship between the customer and the developers;
long-term plans for cooperation.

Product on order

The situation in which the development team finds a third-party customer and agrees with him about the development of a software product designed to solve certain customer problems. The main characteristics of this type of project:

availability of the customer can be described as "periodic";
the idea of the needs and requirements of the customer is formed on the basis of the survey;
pre-specified software requirements, according to which the finished product is accepted;
the developer supports the delivered product for a predetermined period of time;
financial relations are made in the form of a one-time contract;
Plans for further cooperation largely depend on the quality and sustainability of the product that the developer has delivered to the customer.

Replicable product

A situation in which the development team either has no specific customers at all (a “boxed product”), or a rather significant number of customers for the same product. The main characteristics of this type of project:

there is no specific customer who forms the product requirements;
An understanding of the needs and requirements of current and potential users is formed on the basis of market analysis and contacts with typical representatives of users. Feedback from users of the product is carried out mainly by e-mail;
The quality of the software is controlled only by the internal quality department + feedback from users. Beta testing is widely used;
the developer supports the delivered product for a predetermined period of time;
financial relationships exist in the form of a fixed price for a product;
product development plans are based on market analysis.

Outsourcing

The most young model of software production. Appeared on the basis of high qualification and low cost of labor of Russian programmers compared with their Western counterparts. The essence of this model is that a subcontract agreement is concluded between a western software company and a Russian company. The main characteristics of this type of project:

the developer does not contact with the end user of their products. All information is obtained in the form of technical specifications from the west side;
the idea of the needs and requirements of the customer is formed only on the basis of information provided by the western side;
the western side provides its requirements for the software production process and its quality;
product support is usually provided by the western side;
Financial relationships exist in the form of individual contracts between members of the development team and a western customer. The option is to enter into an agreement between a western customer and a legal entity acting on behalf of the development team.

Problems

Organization of the process

The most typical model of software production in Russia can be described as follows: “Each developer chooses one or another method or technique to create programs in accordance with their own habits and preferences. The almost complete lack of clear responsibility for the performance of certain functions. The quality of software is a random variable and directly depends on the abilities of individual employees of the company. Practically everything depends on the initiative and business qualities of several individuals. ” This formulation is almost completely consistent with level 1 CMM called "initial". According to some sources, 2 years ago, the share of software manufacturers using this model was over 70%.

Here are the conclusions of Martin Fowler [4], comparing the process of software production with classical types of construction and production:

during software production, the direct programming (construction) phase is much cheaper than all other phases (design, testing, etc.);
in the process of software production all the main efforts are directed to the design. The design process requires that creative and gifted people participate in it;
creative processes cannot be easily planned. Thus, the predictability of such processes may be an unattainable goal;
we must be very careful about using traditional metaphors in software production. This is a very special kind of activity that requires a special process.

Requirements and iteration

The classic software production process, which was used throughout the world until the mid-nineties, and which is practically a symbol of the structured programming era, consists of the following steps: survey, problem statement, design, programming, testing, and implementation. This process is called "Waterfall". It implies that the requirements for the software product, collected during the survey and formalized during the formulation of the problem, are fixed and do not change during the entire production cycle. However, modern business is very dynamic and changing requirements in it is a common thing.

Fowler writes: “In the process of producing software, everything depends on the requirements. If you cannot achieve sustainability requirements, you cannot create a predictable plan. ” How to be? On the one hand, the requirements must be sustainable, and on the other they will inevitably change during the course of the project.

The key to everything is an iterative production process.

Solutions

All ways to solve the above problems are reduced to changing the software production process so that the process is predictable, stable and in time ensures the fulfillment of the main goal - ready-made software.

Linear and iterative software development life cycles

The linear life cycle of software development (also known as “Waterfall”) involves the sequential completion of 6 stages: survey, problem statement, design, programming, testing and implementation. Such a life cycle implies the immutability of the requirements for software, from the moment the task is set to the moment of signing the act of introducing the developed product, no matter how long the software development stage is.

This approach is applicable to classical disciplines, such as building houses or machine tools. This approach is applicable even in the software industry, if the product requirements do not actually change during the entire development cycle. Another condition is the well-established software development technology, without the use of any completely unexplored technological innovations. An example of projects of this kind is the writing of a program, the control rocket flying to the moon (it is unlikely that humanity learned something radically new about the moon) or a calculator, the requirements for which have long been known and have not changed for a long time. But what about if you want to create, for example, an electronic store for a customer selling computers and components?

Modern business is very dynamic, and the requirements for an electronic store will change more than a dozen times during the development process. And each time it is necessary to rewrite a significant part of the code and bring a lot of documents in accordance with the new circumstances. And this, as a rule, is an unplanned loss of time, resources and efforts, which, as a result, results in the failure to meet deadlines and the production of software that does not satisfy (or does not fully satisfy) customer requirements. Where is the way out? There is one way out, in this case, using an iterative software creation cycle. Such a cycle is a set of iterations, within each of which the project goes through all 6 of the above steps. An iterative cycle will allow to identify serious problems at the earliest stages of development, manage risks, start testing before, etc. Such a process becomes more dynamic and manageable.

Existing Methodologies

There are quite a few typical software production processes in the world. ISO9001, ISO12207, ISO15504, CMM (Capability Maturity Model), MSF (Microsoft Solution Framework), RUP (Rational Unified Process), SCRUM, XP (eXtremal Programming), Crystal Clear, ASD (Adaptive Software Development), Lean Development - all this the types of software production processes, process families and methodologies. Under the methodology refers to a set of methods, practices, metrics and rules used in the production of software. According to Alistair Coburn [1], the methodology is needed in order to:

facilitate the process of introducing new people into the production process;
ensure interchangeability of people;
assign responsibility;
impress the sponsor / customer;
demonstrate a visible transparent process;
create a training base for their employees.

As defined by Jim Highsmith [16] "The real purpose of the methodologies is to increase productivity, providing solutions for customers"

Methodologies can be divided into 3 categories - heavy, light and medium. Simplified, each of which is designed to work in conditions of, respectively, large, small and medium-sized projects. However, as we shall see later, this is not quite the case.

According to Alistair Coburn [1]:

Methodology size is the number of controls that include intermediate releases, standards, activities, milestones, quality metrics, etc .;
methodology density is the level of detail and integrity required for elements of the methodology;
project weight is the size of the methodology multiplied by its density;
project size is the number of people involved in the project.

All the above definitions are conceptual, since there are no quantitative metrics for these values.

The first category of methodologies was born before everyone else and is an integral part of software quality models. Difficult methodologies differ in that they cover all aspects of a company producing software, from requirements management and process planning to regulating relationships with subcontractors and describing requirements for supporting processes. All methodologies in this category are intolerant of change and treat people as a normal resource. Examples: CMM, ISO9000, SPICE.

The second category of methodologies came into being as a set of methods and practices used by small development teams in successful projects. Here, great importance is attached to the relationship between people within the team, taking into account issues of tolerance and other psychological aspects. All processes in this category provide for an iterative software development life cycle. If you try to formulate the basic meaning of light methodologies, you get the following: "Ensuring the maximum speed and quality of software development with a minimum of restrictions and conventions." In particular, in all light methodologies, only the necessary minimum of documents is provided, since Tribute is given to the principle “Documentation is not an understanding.” The essential difference of these methodologies from methodologies of the first type is related to planning. The essence of this difference is best expressed by Jim Highsmith: “In traditional planning, deviations from the plan are errors that must be eliminated. However, in the adaptive process, deviations lead us to the right decision ”[4]. Examples: SCRUM, XP (eXtremal Programming), Crystal Clear.

The so-called “universal” processes fall into the third category of methodologies. The most prominent and well-known representative of this category is RUP. The main characteristic of such processes is scalability - i.e. The process can be configured to work in a small team on a small project, or in a large team on a large and serious project.

Process Applicability Analysis

Alistair Coburn, the author of Crystal, is one of the most famous modern experts who have made their methodologies the object of their research. The result of his research in this area is the book “Agile Software Development” [1], in which the author analyzes the software production process from different points of view. Alistair Coburn is the author of the concept “Software development is a collaborative game of invention and interaction.” He highlights 2 goals for this game:

first goal: to develop and implement working software;
second goal: to provide all the conditions for the success of the next game.

Thus, a minimax task is formulated: to produce software (often it does not require creating any documents at all) and to ensure the development of the next versions of this software (but for this it is necessary to create a number of documents that are necessary, for example, if one of the major developers will leave the team). From the point of view of this concept, the C3 project, in the course of work on which the XP methodology was created, was an unsuccessful game, since after all the main developers left the team, the software product became impossible to develop. The reason for this was that none of the remaining ones did not possess the necessary knowledge for this, and the detailed project documentation was not compiled.

At the same time, light processes are not applicable to work on large projects or on products, the degree of criticality of the correct operation of which is very high.

Figures 1-3 illustrate patterns in the use of different methodologies by development teams of various numbers.

Figure 1 shows the performance dynamics of a team of several people using different methodologies. The small team successfully works using lightweight methodologies. When we increase the methodology, the team has less and less time and effort to develop.

Figure 2 reflects the performance dynamics of a large team using different weight methodologies. Light methodology does not provide proper controllability of the process, so at the beginning of the schedule team performance is low. The team achieves the highest performance using a methodology that is sufficient to set up a controlled and dynamic process, but with further weighting of the methodology, there is a gradual decline in productivity.

Figure 3 reflects the ceiling of a team’s capabilities when using different weight methodologies.

Classification

Coburn has developed a classification [1] of project types, based on two parameters - the size of the project and its degree of criticality. The following degrees of criticality differ:

loss of comfort (Loss of comfort). The loss of comfort means that in the event of an error in the system, people's lives will be a little complicated, they will have to do more manual work, or call each other to fill the interrupted communication channel. Corporate infrastructure support programs usually lie in this area;
loss of a significant amount of money (Discretionary money). It means the loss of a certain amount of money or some other material equivalent, but only within the framework of discomfort. Usually, warehouse systems lie in this area;
Irreplaceable money loss (essential money). Means the loss of money or its equivalent in amounts comparable to the bankruptcy of the enterprise. The national bank management system lies in this area;
loss of life (Loss of life). It means that in case of an error of such a system there is a danger to human life. The control systems of the nuclear power plant, the spacecraft and the aircraft lie in this area.

Introducing some amendments to the proposed method of classification, you can also take into account the degree of distribution of the project (especially important for outsourcing projects), the qualifications of the team and other nuances.