Experience of transition from Waterfall to RUP methodology for the implementation of large IT projects

How did the need arise to move away from the classic Cascade Model of the development life cycle

In 2009, I was offered to choose and implement one of the “dead” projects. Everyone got the prefix "Peril" for what they had tried to do before, but nothing happened.

In large projects, the reason for failure often lies not in the professional level of the team, but in the readiness of the Customer to reach the end and get a return on their own invested efforts. The situation should develop so that the objectives of the project coincide with the short and medium-term objectives of the Customer.
')
A great example: only when they were under sanctions and the price of oil at $ 50 per barrel (as against 110 earlier), the country's leadership switched from reasoning and unhurried gestures to active actions to develop a high-tech economy.

So one of my customers matured and I undertook to make a project for it to develop a new functional module of the Corporate Information System (ERP-system), which was to add 400 new users to the system and ensure verification of 40,000 mortgage loans per year.

After 2 months, we already had the first version of the Technical Specification, which consisted of 200 pages of the main TK and 200 pages of applications to it with descriptions of various forms and business algorithms. I must say that before that I had seen the main TK for the entire corporate IP and at the time of implementation it consisted of only 600 pages. Therefore, even during the development of the specification for the module, the idea increasingly arose that with such a volume of requirements it is likely that we will not take off, and if we take off, it is not very fast.

After the guys from our dedicated development center at EPAM voiced the laboriousness of implementation at 6500 hours and with a duration of 1.5 years of development, it became clear that we wouldn’t definitely take off.

The business process and the procedures that needed to be automated were alive and constantly adapted to changes within the company and to the outside, both under market and the requirements of the Central Bank of Russia.

After a year and a half, those “pants” that we sew for them will be either small or not suitable at all, and this is money thrown away to the wind, time and development resources. It is enough to try to go through it once to understand that this is definitely not an option and career suicide within the company. After that, nothing serious will not be trusted.

Hence the conclusion that Waterfall is suitable for small projects and those where automated processes are not subject to constant changes. Something else was needed, suggesting early delivery and at the same time, so that it would be clear to all who are involved in the development process. Ideally Waterfall, but not Waterfall. It turned out to be the Rational Unified Process.

Two words about context

The company develops and develops its own corporate IP based on the Pivotal CRM platform. The reason for the independent development of KIS and additional functional modules to it is the lack of any alternatives in the domestic market and the expensive cost of acquiring western counterparts in the 2000s.

In addition to the company's employees, employees of more than 200 Partner companies across the country (B2B) work in CIS. As mentioned above, the development is outsourced to EPAM Systems. On the side of the company left business intelligence and project management.

What is wrong with the methodology of Waterfall (features and disadvantages of the cascade model)

This methodology is Mother for all subsequent and is universal. According to this methodology, the first programs were developed.

The basics that keep this Methodology and laid in it:

1. From beginning to end, the designed end result.
   
   
   (Architectural Result Project)

   
   
   
   
2. Instructions for assembling the final result.
   
   
   (Project Implementation Plan)

   
   
   
   
3. Consistent project implementation.
   
   
   Analysis, design, assembly, testing and transfer to operation

   
   
   
   
4. Resistance to changes in the Project Results and Assembly Instructions in the process of project implementation.
   
   
   Changes in the project

   
   
   
   
5. The result of the project is delivered as a single delivery at the end of the project.
   
   
   Dynamics of the project result assembly

   
   

In order to implement a long-lasting Waterfall project with a time span of more than 1-2 years, it is necessary to make assumptions about the business environment, customer needs and companies that will be at the time of project completion (assumptions related to the business objectives of the Project). You also need to assume and lay the estimated cost of resources for each year of the project’s life (assumptions related to the project implementation plan).

The greater the project content, the complexity and the implementation time, the more assumptions are put into the project, which may be both correct and incorrect. The most critical are the assumptions related to the business objectives of the Project, if they turn out to be wrong, then all the resources invested in the project, money and time are depreciated.

Attempts to make significant changes to the Project of the End Result in the course of implementation require a redesign cycle of the End Result and the Assembly Instructions. Significant changes often do not allow the continuation of work already started in the original plan and lead to a rework of what has already been done.

For this reason, the project team and stakeholders resist significant changes in the project only if circumstances and risks that have been played / revealed do not cast doubt on the whole project of the need to continue it. When changes are made, they most often lead to additional costs and lengthening the time. The increased duration again generates new requirements and new changes. It is like a vicious circle from which it is impossible to get a winner. From here there is a constant departure from the original dates and budget.

Therefore, this Methodology is suitable for construction projects of buildings and structures, such as a nuclear power plant, but is not suitable for developing Products for rapidly changing environments in which they will be applied. In which the lifetime of the Product in its original form without making changes is less than 5 years.

The idea of ​​iterative development is designed to eliminate the risks associated with the assumptions, to provide an opportunity to quickly receive feedback, reduce the scale of losses from incorrect business assumptions and make changes to the Draft Final Result based on the feedback received at the end of the next iteration.

Benefits of the Rational Unified Process

The RUP methodology was purposefully developed to develop large software systems.

Innovation number 1 , laid in its foundation, is “Business Modeling” and its associated Use Case.

First, a business usage scenario is developed, where the future system is a “black box” that satisfies all the needs of the User / Business. On its basis, system usage scenarios are developed that describe the functions of the system that will support the execution of the business scenario.

The main task in the development of corporate IP is to ensure the continuity of the supported business process. When the fabric of a business process breaks, then all the following stages in the production chain, and sometimes the entire business of tens of billions of rubles, arise.

An example of a stop trading Moscow Exchange

In the stock, currency and derivatives markets of the Moscow Stock Exchange, an abnormal situation ... Last time the exchange suspended trading on September 1, but this affected only the “Main Market” section. This failure was the fourth in the last four months. - Lenta.ru

Therefore, the design of a corporate IP is based on designing a new business process based on its use, to replace one process with another, and hundreds of people from a certain day started working differently than they used to. This is the main and main difference between the development of corporate IP from other types of software.

Innovation # 2 : Based on the selected system use cases of the system, architectural decisions are made and components are selected that will support business scenarios and decide whether they will be used together to support several business scenarios or remain tailored to a specific scenario. (Object Oriented Design and Programming)

The presence of individual components allows them to reuse and replace one with another. The more components are used together, the higher the probability of breaking the neighboring business scenario when making changes to the component. Also, the presence of common components limits the speed and scope of subsequent work on the development of the system. If one team is committed to changing some kind of common component, the other cannot start working on changing it until the first one finishes the work and debugs its business use case.

Novation # 3 : The presence of the selected use cases of the system allows to divide them into primary and secondary. The secondary ones are based on the results of the implementation of the primary scenarios, which by and large are self-sufficient. From here it is possible to select iterations and decompose the implementation of all scenarios on them.

In Waterfall, there are no iterations, as it is sometimes incredibly difficult to divide the functional requirements into self-sufficient blocks and implement them by iterations. I'm not saying that this can not be done, it is possible, but most often the result will not be a working prototype of an IT solution, but some part of a future program. This does not make business easier; it will get a working prototype in a year, the presence of iterations here helps, but not much.

This is how the concept RUP looks like in a simplified form:


Each business scenario is deployed in system usage scenarios. Each system scenario - in the requirements for interfaces, algorithms, data and non-functional (performance, availability time, response speed ...). Based on the selected requirements, the future system architecture and its functional modules (service subsystems) are determined. Subsystems are broken down into components. Components contain executable code.

The main idea of ​​RUP is to issue in the first iterations a prototype of a future system with a portion of ready-to-use / use business scenarios. Since the delivery of the part of the Result of the project (increment) is carried out at the end of each iteration, this allows you to start to benefit from the use of intermediate versions of the Result and return investments to the project during its implementation.

How we applied RUP

Step # 1 - I agreed with the manager on the use of a technical task template different from that adopted by the company. The basis of the new TK formed the business and system usage scenarios. This decision was made earlier than the decision on work on RUP and for a number of reasons which can be read in my article “What should be the TK for a corporate IP?” (See the link at the end of this article in the “What else to read” section).

Step 2 - Designed the target business process and prepared the first version of the TOR. After we received estimates of the complexity and duration of implementation, we decided that we would try to use RUP for this project. Smashed the target business process into 5 business procedures / 5 stages of the project. The first version of the TOR has become a concept / roadmap for moving towards a targeted process.

Step number 3 - Decided on the strategies of automation: move from the input or output point of the entire business process.

The first strategy “from the entrance point” creates tension and forces to automate all other stages in the chain. In the subsequent stages of the business process, the primary data that they need to work is no longer available.

The second strategy “from the exit point” does not have this voltage, but it also does not have the data and history that is created and accumulated in the previous stages. Need to regularly transfer the necessary information to the CIS.

Let's go on the first strategy. The problem of providing the subsequent steps with the necessary information was decided by uploading data to Excel. We got an unexpected effect in the form of growing motivation of business users to fully switch from Excel to an IMS with appropriate support and willingness to devote time to working with a business analyst to work out requirements.

Step # 4 - Prepared a statement of work for the first part of the system module (second iteration). While the first part was being developed, the TZ was prepared for the second part (the third iteration).

The first iteration was the preparation of a Conceptual Technical Assignment, where the business process was fully documented using EIS, and the first version of the architecture of the future module was developed.

Step # 5 - After the first part of the module has been launched and its pilot operation has started, we started preparing an addition to the TK for the first part and the TZ for the third part (fourth iteration). In addition, there are those requirements that were missed at the beginning and those that were unsuccessful in terms of use.

Result - For everything about everything, it really took us 1.5 years. A working prototype (the first part of the system module) was obtained 6 months after the project launch date. The rest came every 2 months.



From the architectural point of view, the components of the new functional module were sharpened solely for it and did not use the components of the main IC. It allowed:

1. To stabilize the operation of the entire system in the event of errors caused by regularly introduced changes in the module, and the module from errors in changes made to the overall functionality of the IC.
2. To work on the development of the existing EIS functionality in parallel with the development of a new module.

Although RUP is directly associated and associated with UML notations, it is not necessary to use them to describe business and system use cases, other notations will do. The main thing is that the used notations should be clear to both Business Users, who will agree on the statement, and to System Analysts and Developers who will continue to work with the Terms of Reference by translating it into design and technical documentation. To describe the business scenario, we used eEPC and ARIS, as well as the requirements of IDEF0 / 3 notations to set the framework for system scenarios: process input, process output, and executor.

The article “What should be the TK for a corporate IP?” Will give an idea of ​​what a business scenario and a system use scenario look like. In the original RUP manual, it is not recommended to make sketches of user interfaces when drafting system scenarios, but we considered them necessary as they facilitate understanding of business and system scenarios by visualizing their elements. This was practically the first prototype of the system, although not clickable.

After this project, the written TZ has become a new template (standard). The project also influenced the entire technological-production process of software development in the company making it rhythmic.

A bit of theory to understand the technical side of the RUP methodology and how to apply it.

Each iteration in RUP is a classic Waterfall, containing all 5 stages of requirements gathering and analysis, design, development, testing and delivery. But RUP also introduces such a concept as phases for a project: Requirements Collection and Analysis, Design, Construction, Implementation.



Now more about this.

First iteration, Phase Analysis

Dedicated to collecting the requirements of future users to the system. Users tell each their own piece of work (user’s scenario of using the system), and the Analyst tries to collect the whole workflow from individual operations and procedures. The purpose of this iteration is to reach a business scenario of using the system / future business process based on IP. There may be more than one of them, so you shouldn’t shut yourself to having one.

The customer company is a bank. He decides that he wants to start providing his clients with the service of remote cash withdrawal using ATMs installed in clients' offices and metro stations.

To provide this service, you need to develop and refine the software available to the bank. During the collection of requirements and their analysis, three major procedures were identified in the service business scenario:

1. Cash load at ATM Bank staff.
2. Cash withdrawal to the Client upon his request from the bank account.
3. Accounting for the number of uploaded and issued bills.

In this iteration, there are still no works related to the design of the future system architecture, program development, testing and delivery. RUP does not set conditions that they are necessarily necessary, but at your own discretion you can fulfill them.

Second iteration, Phase Analysis

After the business scenarios are defined, the collected requirements are laid out in steps of the business scenario and system scenarios. At this stage, those steps of business scenarios that were not identified in the first iteration are also identified.

During the second iteration, two additional procedures were identified for the main business scenario:

1. Monitoring and notification sent by an ATM to the bank, about the end of cash in it.
2. Monitoring and notification of the operability of the ATM or the presence of problems / errors in the work.

To ensure the availability of cash services 24/7.

The result of this iteration is:

1. 90% complete business scenarios.
2. Intended but not detailed system usage scenarios for each business scenario.
3. Collected requirements are structured by business and system scenarios.

Third iteration, Design Phase

The identified business scenarios are set implementation priorities and the most important of them become the subject of study at this iteration. In the selected business scenarios, the primary steps are the steps that are the basis for all subsequent ones and without which the implementation of the rest does not make sense. For these stages, system usage scenarios are detailed.

The finished result is transmitted to System Analysts and the Architect for modeling what will happen inside the system. Classes are distinguished, cooperation between them, the sequence of interaction. A basic understanding of the architecture of the future system is being formed; subsystems, system components and their distribution among IT solution nodes are highlighted.

Already in this iteration, development work can be started and the result of the iteration will be a ready-made prototype that supports the implementation of the primary steps of the key business scenario. It can already be shown to users to collect feedback and clarify requirements.

This prototype is not transferred to pilot or commercial operation, as the underlying architecture is the first trial ball and will be adjusted in subsequent iterations.



Fourth iteration, Design Phase

It starts parallel to the third iteration, when the analysis work is completed and design work has begun.



At this iteration, the development of the primary steps of the remaining business scenarios takes place. The reason why they are being worked out, rather than the next steps for key business scenarios, is the definition of the future architecture of the entire program.

Each business scenario may require for its implementation some specific components of the future system and for this reason the architectural solutions for each business scenario must be coordinated with each other.

The result of this iteration will be:

1. Architectural solutions for each business scenario.
2. Agreed design of a common system architecture 1.0
3. The second version of the prototype with ready-made first steps of secondary business scenarios in addition to the first steps of a key business scenario.

Fifth iteration, Construction Phase

It starts parallel to the fourth iteration, when the analysis work is completed and the design work has begun. It is assumed that by the time of the start of the iteration, feedback has already been received on the results of testing by users of the prototype made during the third iteration. Here is the study of the remaining steps of the key business scenarios and the associated system use cases.

It is assumed that by the time of the commencement of the Design phase, the fourth iteration will be implemented (developed) and testing of the second version of the prototype has begun.
There is a clarification of the agreed project of the overall system architecture after which the introduction of some conceptual changes to it is practically reduced to zero, but it is still possible, since the completely secondary business scenarios remain unfulfilled.

The result of the iteration are:

1. Fully implemented key business scenario.
2. Agreed design of a common system architecture 2.0
3. The third version of the system, which implements the key business scenario and the first steps for secondary business scenarios.

Here it is already possible to transfer the system for trial operation to business users, since the system is no longer a prototype and is more similar to the one that will be transferred to business users as its final version.

Sixth iteration, Construction Phase

It starts parallel to the fifth iteration, when the analysis work has been completed and design work has begun.

The remaining steps of the secondary business scenarios and the associated system use scenarios are prescribed. It is assumed that by the beginning of this iteration, the second version of the prototype, tested during the fourth iteration, was tested and feedback from users was received on working with the primary steps and stages of secondary business scenarios.

The result of the iteration are:

1. Fully implemented secondary business scenarios.
2. The final draft of the overall system architecture 3.0
3. The fourth version of the system that implements all business scenarios.

The full-scale trial operation of the system by business users and its fitting for industrial use begin.

Seventh iteration, implementation phase

Here there are some critical changes in the implementation of business scenarios that were identified during trial operation and do not allow full use of the system in industrial operation.

Eighth Iteration, Implementation Phase

At this iteration, work is carried out to improve the user-friendliness of the system and optimize the fabricated solutions. Users are primarily interested in the speed and throughput of their sections. System is being transferred to support. There are formal procedures for the completion and closure of the project.

Subtotal

The RUP methodology does not set conditions for the number of iterations and their distribution over the phases. The concept of phases and iterations is introduced to reduce risks, resource requirements, set the project implementation rhythm and provide the opportunity to make the required changes when developing a large information system.


The result of each iteration clarifies the plan for the next one and gives more understanding about whether we will need additional resources for the next iteration, whether additional iterations will be needed to complete the phase. A minimum team of several people can start working on a project and its expansion will be justified by the plan of the next iteration.

The results of the completion phase clarify the plan and the economy of the project. Here decisions are made to continue the project, whether it is necessary to narrow or expand the content of the project (volume of business requirements) in order to remain within the business framework: the desired delivery time for the project result, its cost (budget) and profitability (return on investment).

Conclusion

I understand that it is not easy to change the foundations adopted in the company how to implement IT projects, especially if the company has a Project Office with a methodology based on PMBoK and GOST. If after reading the article you understand that the near future is still Waterfall, then I recommend trying to implement the following minimum:

1. Register an automated business process based on the use of IP. This is more than 50% finished user documentation and acceptance tests.
   
   
2. Put in the project time that at least 1 time during the year will have to perform a full range of work on the redesign, preparation and coordination of changes to the project documentation.
   
   
3. Multiply the complexity of development work by 1.5. Since a third of the work done will have to throw out and there will be new requirements. The duration of the development phase is also multiplied by 1.5.
   
   
4. Divide the project budget into two parts: the creation and development.
   
   Approximately 30% of the total project budget for creation is to be allocated for development. It is necessary then that when you finish the project there will be an intermediate state - the project is implemented, but not accepted for maintenance .
   
   , , . , , () , - .
   
   , .

:

• Waterfall – ( ),
• RUP – ( ),
• Scrum – ( / ),
• Kanban (Continuous Integration), (Acceptance Test-Driven Development), (Continuous Deployment) – ( ).

RUP

1. « ?»
   
   , , . , .
   
   . RUP.
   
   
2. « »
   
   , RUP.
   
   — - / -. , .
   
   - UML, -. IDEFx Structured Analysis and Design Technique ( ) . ISO PMI. .
   
   
3. - . RUP Rational IBM. - Business Studio . 12 .