How to manage the giants: the rules of team building and building processes in web development
In Russia, more than 5,000 companies are engaged in customized web development (according to the Tagline analytical agency), but in my opinion, our market is still in its infancy. Many digital companies, web studios and integrators are in reality not ready for the quality development and support of truly large projects. AGIMA companies often turn to us, dissatisfied with the quality of work performed by their current contractor, whom they have chosen at the tender. For many years of working with major Russian companies, we have accumulated vast experience in organizing the processes of developing and developing large Internet projects, and I want to share them. In this article I will tell you how to properly organize the infrastructure, build communications between the teams and not forget about the important components of the agency service when working with "giants."
What do I consider a large project? If the project produces a total of more than 2500 man-hours per month by the entire project team, not counting the work of the manager. The scope of work is, as a rule, 2-3 major tasks of 800-1000 man-hours in parallel and 5-6 small daily tasks within the framework of certain subscriptions for technical support. These can be websites of large insurance companies, banks, large online stores, corporate portals, HR portals and media.
')
What awaits you in the article:
Manual on the requirements for the organization of the team and processes in the design, development and support of large projects. At the core is 10 years experience in the market of customized web development. During this time I was able to launch more than 100 projects. For the last two years I have been managing the development and support of AlfaStrakhovanie’s projects.
For whom the article is intended:
The article will be of interest to project managers and everyone who in one way or another relates to the organization of web application development processes and interaction with the client.
Disclaimer:
This article is not a panacea, but only a purely personal opinion of the author (Evgeny Lobanov, executive director of AGIMA).
Congratulations, you have a big project!
No matter how you got it. Often, large projects grow out of simple tasks: either you have satisfied the client on a small task and he decides to give you support and development entirely, or you grow with the client - this is the best option. But perhaps you just dumped and, without even waiting for it, won the tender for the support and development of a large project.
For example, one of our very first “giant” projects grew out of small technical improvements to the requirements of seo's. At some point, the client said: “We want to launch a new version of the site, almost everything is ready, but our current contractor cannot finish it. We liked how you quickly make improvements. Let's you finish it and run? ". We were faced with the fact that we acquired the contract, but we didn’t have the infrastructure and processes for such projects. What to do?
TEAM
We started with the most important thing - we formed a new team separately for the project, because if you have no hands, then you cannot “digest” a large amount of tasks. The process of selecting a team has become one of the most difficult and long stages. We filtered a bunch of resumes and conducted a lot of interviews. We started the search with the project manager and team leader, since these are key roles on any project, and they have already participated in the further selection of the team, this allowed us to approach the search for other participants more effectively.
First line
A large project is always a lot of communications (in our case there were more than 3 managers on the client side responsible for certain types of tasks), so we had to create a single entry point. At that time, the project was managed by one manager, and he could not cope with the stream of requests for assessments, tasks, questions and bugs that had collapsed on him. When we tried to drag all the responsible persons into our task manager, we immediately faced a huge number of duplicates and disgusting work on the tasks (some of them could contradict each other, as they were from different business divisions, but according to general pages: etc.). Then we decided to try a fairly simple thing - we entered the role of the project administrator. This is an impersonal e-mail address of Technical Support, which receives all requests from client managers in a single project; several dedicated employees can process it.
The volume of communications is so large that it fully occupies the working resources of at least one team member. To assist the project manager in managing the flow of requests, we hired a project administrator. It is the first line that monitors the speed of communications, so the response time and response to any request should be no more than 15 minutes. The first line manages the client's expectations, therefore, should promptly “ping” the project manager, team leader, or the client if there is no feedback from him for a long time. Also in the duties of the first line is the preparation of monthly reports on technical support.
Each task must be entered into the task manager. No task - no task. All tasks are updated and routed by the project administrator. So we unloaded our project manager for other, more important tactically, duties.
Project Manager
A long time ago we decided that the project manager, and only he, bears full responsibility for the project. This is the person who is engaged in planning, drawing up timings for each task, determining priorities and taking into account all risks. It is he who makes the key decisions (the use of a particular implementation and methodology, the decision to launch on a productive environment, engaging the expertise of employees), and also connects when solving problems that arise.
We recorded transparent KPIs, including PM should monitor the efficiency of the first line and the implementation of the SLA, as well as be responsible for the profitability of the project and the clear formulation of tasks. If the initial task changes, it must justify and defend changes in the cost and timing of each additional subtask. In case of disputes, words must be confirmed by facts, therefore all discussions must be documented or at least in writing, for example, in the mail.
All work is closely monitored by senior management: the account director and technical director, who weekly conduct a cut on all tasks from the work plan for the next month.
Timlid
This is the last link on the project, which decides on the launch of the task in production. In our company there are a lot of projects with complex integration tasks, so the main burden rests on the Temlides' shoulders - to scale to large projects, we connect 2 Timidovs simultaneously. We began to do this not immediately, but only with an awareness of the criticality of leave or illness, when the team was alone.
But, unfortunately, it is very difficult to find a ready timblid, therefore it is more correct to bring them up from good programmers. Not every programmer can be a team leader - the head of the development team must have managerial skills.
Product owner
At some point, we realized that we needed a person who manages the flow of thoughts of the client, directs the development strategy of the project in the right direction and conducts internal consultations of the product team. Product Owner needs to know more about the specifics of the business and the product, and be able to provide all the details. We took such a person from the client’s market so that he and the whole team selected the optimal solution of business problems and, together with the project manager and client, managed the team load.
Spare Bench: Connected Specialists
In addition to key specialists (designers, layout designers, programmers, testers), additional resources are often needed for large projects.
On our major projects most often needed:
There are tasks where one cannot do without an architect — this will allow minimizing risks in the further development and maintenance of the developed functionality. Often, the timely connection of the architect can save you from further refactoring, which is almost impossible to sell to the client.
Any new task at the design stage, we also pass through the architect and product owner.
Web analytics are also connected in the design. The data collected at the analytics stage is the foundation for the further interface not only in its design, but also in design.
We apply web analytics to those problems where there are controversial technical and interface differences. They can occur not only in communication with the client, but also within the team itself. In such cases, we resort to split tests and, based on the results, we apply more successful solutions in the final product.
This is the person who thinks the logic of the work and interaction of interfaces. In addition to prototypes, we always develop explanatory documentation (specifications) for prototypes. Due to this, additional costs are reduced at the design and layout stages, as well as easier supervision of the project by the designer and the project manager.
To refactor a functional with complex integrations, we involve the designer in tandem with the team leader. This allows you to optimize the current integration scheme, identify all shortcomings and make recommendations for their elimination.
Thus, a document called “Service Analysis” is developed, where everything is described in detail.
On each interface, we conduct testing and research, the results of which we share with the client. If there are overt problems, we redo them at our own expense.
The toolkit of our UX specialists:
Out-of-state resources
Load on large projects can both dramatically increase or sharply subside. There is always the probability of a floating volume of tasks and we have provided the ability to quickly scale. For this, remote teams are involved, since there is no point in inflating the internal staff, if in a month it needs to be significantly reduced.
Such teams will always be able to hedge in case of an urgent need to increase the amount of resources. At the same time, we documented with the client that the term for planning a new major task in the production plan can reach five working days.
PROCESSES
Methodology
When working on large projects, it is extremely important to choose the right methodology.
We came to the conclusion that in large projects one cannot often confine one work scheme and implement the whole project only with waterfalls or flexible methodologies. We use mixed methodologies, applying sprints and kanban boards as part of the iterations of the waterfall process. And the main thing that we understand is that, regardless of the methodology, it is important that the whole team takes part in the process of solving problems.
Workflows
The second global step, we agreed with the client a clear workflow. For example, we conduct releases on Tuesdays, assessments on Wednesdays, task planning in the production plan for the week and retrospective with us on Thursday. The schedule is strictly controlled by the project administrator at the task tracker level and the project manager at the request level from the client.
Speaking of proper resource planning, we fixed in all calculations in estimates and planning that the working day of the contractor is 6 hours, not 8. In addition, it is necessary to lay in advance time for accidents and bugs, and often search and analysis of the bug take 90 % of the time, and the correction - only 10%. To speed up the bug fixing, we have divided the resources: technical support and production are handled by various technical teams, the points of intersection are only management and team leaders.
Testing and QA
Large projects are always complex architecture, high load and large financial responsibility. Therefore, all critical functionality must be covered with a grid of functional tests. This helps prevent potential problems and optimizes time to fix bugs.
The problem of connected architecture has long been haunted by us and has added negativity to our relationship with the client. How to be confident in the quality of the product? First, in the process of developing a project, we began to use standardized checklists . Secondly, we started writing user cases at the design stage of applications, it reduced a lot of risks and costs at the development stage, for example, multiple task transfers between a programmer and a tester. If the task is returned to the developer from the retest, this is a signal of the problem, and if this happens a second or more times, you need to connect and find out what the reason is.
Automated testing within CI (git flow)
No matter how much we test, the errors still were, are and will be. We wondered - how to minimize the time for analyzing bugs and respond quickly to accidents?
Effectively this task was able to solve the monitoring of accessibility and functional continuity. We have developed a special event widget that aggregates the data in the context of each of the critical list of urls for uninterrupted operation (data is taken from our own monitoring system zabbix and two external independent monitoring services) and data on functional accessibility (from our build server TeamCity) . If accessibility problems or functional errors occur, a warning system is triggered: all responsible persons receive an email, sms and a call to a mobile phone. Thus, the response time to any idle time is no more than 15 minutes.
We made it a rule to run functional tests on the list of critical url:
Guidelines
Errors are not only technical, but also visual. They can be minimized by prepared guidelines for designers and layout designers (examples 1 and 2 ), which provide for all control and error handling states.
We have guidelines for programmers ( example ). This documentation, the description of classes and methods, integration mechanisms, etc. We aggregate all this useful information and store it in our knowledge base (we use confluence ) - project managers are responsible for updating the information.
It is important to take into account a competent architecture that will require a minimum of "crutches".
at the stage of finalization and development of the functional. When making a code, we adhere to the standards PSR-1 and PSR-2 . This allows you to reduce the time when switching and transferring tasks between developers and timlidov. Those who work in the IDE, for example in PHPStorm, can configure automatic verification of compliance with standards . We have implemented automated code checking on the back-end server side using phpSniffer.
Also, when developing the architecture, we always take into account the requirements for information security: monitoring the integrity of the code, compliance with the FZ-152, lack of xss and sql injection vulnerabilities and compliance with the requirements of TOP-10 OWASP.
In reality, selling guidelines, documentation and autotests is very hard. Often studios do not include these works in estimates, which leads to a terrible word.
“Refactoring”, but, as a rule, already with a different team of contractors. Therefore, it is necessary to immediately take seriously the development and development of projects. Prepare in advance for the support of large customers - this, first of all, develops our market and allows you to produce really high-quality projects.
PS In the next article in this series, we will look in detail at a toolkit for project managers that helps structure and optimize the planning and organization of processes on a project.
What do I consider a large project? If the project produces a total of more than 2500 man-hours per month by the entire project team, not counting the work of the manager. The scope of work is, as a rule, 2-3 major tasks of 800-1000 man-hours in parallel and 5-6 small daily tasks within the framework of certain subscriptions for technical support. These can be websites of large insurance companies, banks, large online stores, corporate portals, HR portals and media.
')
What awaits you in the article:
Manual on the requirements for the organization of the team and processes in the design, development and support of large projects. At the core is 10 years experience in the market of customized web development. During this time I was able to launch more than 100 projects. For the last two years I have been managing the development and support of AlfaStrakhovanie’s projects.
For whom the article is intended:
The article will be of interest to project managers and everyone who in one way or another relates to the organization of web application development processes and interaction with the client.
Disclaimer:
This article is not a panacea, but only a purely personal opinion of the author (Evgeny Lobanov, executive director of AGIMA).
Congratulations, you have a big project!
No matter how you got it. Often, large projects grow out of simple tasks: either you have satisfied the client on a small task and he decides to give you support and development entirely, or you grow with the client - this is the best option. But perhaps you just dumped and, without even waiting for it, won the tender for the support and development of a large project.
For example, one of our very first “giant” projects grew out of small technical improvements to the requirements of seo's. At some point, the client said: “We want to launch a new version of the site, almost everything is ready, but our current contractor cannot finish it. We liked how you quickly make improvements. Let's you finish it and run? ". We were faced with the fact that we acquired the contract, but we didn’t have the infrastructure and processes for such projects. What to do?
TEAM
We started with the most important thing - we formed a new team separately for the project, because if you have no hands, then you cannot “digest” a large amount of tasks. The process of selecting a team has become one of the most difficult and long stages. We filtered a bunch of resumes and conducted a lot of interviews. We started the search with the project manager and team leader, since these are key roles on any project, and they have already participated in the further selection of the team, this allowed us to approach the search for other participants more effectively.
First line
A large project is always a lot of communications (in our case there were more than 3 managers on the client side responsible for certain types of tasks), so we had to create a single entry point. At that time, the project was managed by one manager, and he could not cope with the stream of requests for assessments, tasks, questions and bugs that had collapsed on him. When we tried to drag all the responsible persons into our task manager, we immediately faced a huge number of duplicates and disgusting work on the tasks (some of them could contradict each other, as they were from different business divisions, but according to general pages: etc.). Then we decided to try a fairly simple thing - we entered the role of the project administrator. This is an impersonal e-mail address of Technical Support, which receives all requests from client managers in a single project; several dedicated employees can process it.
The volume of communications is so large that it fully occupies the working resources of at least one team member. To assist the project manager in managing the flow of requests, we hired a project administrator. It is the first line that monitors the speed of communications, so the response time and response to any request should be no more than 15 minutes. The first line manages the client's expectations, therefore, should promptly “ping” the project manager, team leader, or the client if there is no feedback from him for a long time. Also in the duties of the first line is the preparation of monthly reports on technical support.
Each task must be entered into the task manager. No task - no task. All tasks are updated and routed by the project administrator. So we unloaded our project manager for other, more important tactically, duties.
Project Manager
A long time ago we decided that the project manager, and only he, bears full responsibility for the project. This is the person who is engaged in planning, drawing up timings for each task, determining priorities and taking into account all risks. It is he who makes the key decisions (the use of a particular implementation and methodology, the decision to launch on a productive environment, engaging the expertise of employees), and also connects when solving problems that arise.
We recorded transparent KPIs, including PM should monitor the efficiency of the first line and the implementation of the SLA, as well as be responsible for the profitability of the project and the clear formulation of tasks. If the initial task changes, it must justify and defend changes in the cost and timing of each additional subtask. In case of disputes, words must be confirmed by facts, therefore all discussions must be documented or at least in writing, for example, in the mail.
All work is closely monitored by senior management: the account director and technical director, who weekly conduct a cut on all tasks from the work plan for the next month.
Timlid
This is the last link on the project, which decides on the launch of the task in production. In our company there are a lot of projects with complex integration tasks, so the main burden rests on the Temlides' shoulders - to scale to large projects, we connect 2 Timidovs simultaneously. We began to do this not immediately, but only with an awareness of the criticality of leave or illness, when the team was alone.
But, unfortunately, it is very difficult to find a ready timblid, therefore it is more correct to bring them up from good programmers. Not every programmer can be a team leader - the head of the development team must have managerial skills.
Product owner
At some point, we realized that we needed a person who manages the flow of thoughts of the client, directs the development strategy of the project in the right direction and conducts internal consultations of the product team. Product Owner needs to know more about the specifics of the business and the product, and be able to provide all the details. We took such a person from the client’s market so that he and the whole team selected the optimal solution of business problems and, together with the project manager and client, managed the team load.
Spare Bench: Connected Specialists
In addition to key specialists (designers, layout designers, programmers, testers), additional resources are often needed for large projects.
On our major projects most often needed:
- Architect
There are tasks where one cannot do without an architect — this will allow minimizing risks in the further development and maintenance of the developed functionality. Often, the timely connection of the architect can save you from further refactoring, which is almost impossible to sell to the client.
Any new task at the design stage, we also pass through the architect and product owner.
- Analyst
Web analytics are also connected in the design. The data collected at the analytics stage is the foundation for the further interface not only in its design, but also in design.
We apply web analytics to those problems where there are controversial technical and interface differences. They can occur not only in communication with the client, but also within the team itself. In such cases, we resort to split tests and, based on the results, we apply more successful solutions in the final product.
- Designer
This is the person who thinks the logic of the work and interaction of interfaces. In addition to prototypes, we always develop explanatory documentation (specifications) for prototypes. Due to this, additional costs are reduced at the design and layout stages, as well as easier supervision of the project by the designer and the project manager.
To refactor a functional with complex integrations, we involve the designer in tandem with the team leader. This allows you to optimize the current integration scheme, identify all shortcomings and make recommendations for their elimination.
Thus, a document called “Service Analysis” is developed, where everything is described in detail.
- Ux
On each interface, we conduct testing and research, the results of which we share with the client. If there are overt problems, we redo them at our own expense.
The toolkit of our UX specialists:
- usability testing;
- poll;
- direct and reverse card sorting;
- diary study;
- feedback analysis;
- semantic and sentimental analysis of texts.
Out-of-state resources
Load on large projects can both dramatically increase or sharply subside. There is always the probability of a floating volume of tasks and we have provided the ability to quickly scale. For this, remote teams are involved, since there is no point in inflating the internal staff, if in a month it needs to be significantly reduced.
Such teams will always be able to hedge in case of an urgent need to increase the amount of resources. At the same time, we documented with the client that the term for planning a new major task in the production plan can reach five working days.
PROCESSES
Methodology
When working on large projects, it is extremely important to choose the right methodology.
We came to the conclusion that in large projects one cannot often confine one work scheme and implement the whole project only with waterfalls or flexible methodologies. We use mixed methodologies, applying sprints and kanban boards as part of the iterations of the waterfall process. And the main thing that we understand is that, regardless of the methodology, it is important that the whole team takes part in the process of solving problems.
Workflows
The second global step, we agreed with the client a clear workflow. For example, we conduct releases on Tuesdays, assessments on Wednesdays, task planning in the production plan for the week and retrospective with us on Thursday. The schedule is strictly controlled by the project administrator at the task tracker level and the project manager at the request level from the client.
Speaking of proper resource planning, we fixed in all calculations in estimates and planning that the working day of the contractor is 6 hours, not 8. In addition, it is necessary to lay in advance time for accidents and bugs, and often search and analysis of the bug take 90 % of the time, and the correction - only 10%. To speed up the bug fixing, we have divided the resources: technical support and production are handled by various technical teams, the points of intersection are only management and team leaders.
Testing and QA
Large projects are always complex architecture, high load and large financial responsibility. Therefore, all critical functionality must be covered with a grid of functional tests. This helps prevent potential problems and optimizes time to fix bugs.
The problem of connected architecture has long been haunted by us and has added negativity to our relationship with the client. How to be confident in the quality of the product? First, in the process of developing a project, we began to use standardized checklists . Secondly, we started writing user cases at the design stage of applications, it reduced a lot of risks and costs at the development stage, for example, multiple task transfers between a programmer and a tester. If the task is returned to the developer from the retest, this is a signal of the problem, and if this happens a second or more times, you need to connect and find out what the reason is.
Automated testing within CI (git flow)
No matter how much we test, the errors still were, are and will be. We wondered - how to minimize the time for analyzing bugs and respond quickly to accidents?
Effectively this task was able to solve the monitoring of accessibility and functional continuity. We have developed a special event widget that aggregates the data in the context of each of the critical list of urls for uninterrupted operation (data is taken from our own monitoring system zabbix and two external independent monitoring services) and data on functional accessibility (from our build server TeamCity) . If accessibility problems or functional errors occur, a warning system is triggered: all responsible persons receive an email, sms and a call to a mobile phone. Thus, the response time to any idle time is no more than 15 minutes.
We made it a rule to run functional tests on the list of critical url:
- Daily run in automatic mode on schedule (9:00 and 18:00).
- Run on the dev-site of a specialist before pushing (development is carried out in feature-branches, each has its own site strictly allocated). When kommite work off hooks that run code review and unit tests. Upon successful completion of tests, the development branch automatically merges with a stable develop-branch and updates it. Next comes the automated functional testing of custom scripts. Before this, the script also prepares the environment: forcibly clears the cache, minifies the necessary files and launches the installers if the changes affect the changes in the database. According to the results of testing, a report is made available to all interested parties (developers, managers, testers) and a notification is sent to the mail with the status of passing the tests.
- Run on Stage (almost full take of a productive server) and in combat, after the release of the assembly.
Guidelines
Errors are not only technical, but also visual. They can be minimized by prepared guidelines for designers and layout designers (examples 1 and 2 ), which provide for all control and error handling states.
We have guidelines for programmers ( example ). This documentation, the description of classes and methods, integration mechanisms, etc. We aggregate all this useful information and store it in our knowledge base (we use confluence ) - project managers are responsible for updating the information.
It is important to take into account a competent architecture that will require a minimum of "crutches".
at the stage of finalization and development of the functional. When making a code, we adhere to the standards PSR-1 and PSR-2 . This allows you to reduce the time when switching and transferring tasks between developers and timlidov. Those who work in the IDE, for example in PHPStorm, can configure automatic verification of compliance with standards . We have implemented automated code checking on the back-end server side using phpSniffer.
Also, when developing the architecture, we always take into account the requirements for information security: monitoring the integrity of the code, compliance with the FZ-152, lack of xss and sql injection vulnerabilities and compliance with the requirements of TOP-10 OWASP.
In reality, selling guidelines, documentation and autotests is very hard. Often studios do not include these works in estimates, which leads to a terrible word.
“Refactoring”, but, as a rule, already with a different team of contractors. Therefore, it is necessary to immediately take seriously the development and development of projects. Prepare in advance for the support of large customers - this, first of all, develops our market and allows you to produce really high-quality projects.
PS In the next article in this series, we will look in detail at a toolkit for project managers that helps structure and optimize the planning and organization of processes on a project.
Source: https://habr.com/ru/post/300234/
All Articles