Analyst Days: Irina Surova - Using traces in practice
We publish an article written on the basis of the previous report of Irina Surova from the last Analyst Days conference. This year, Irina acts not only as a speaker, but also as a member of the program committee of the future Moscow conference.
Video report:
')
Presentation of the report:
www.slideshare.net/VLDCORP/ss-21934265
Good afternoon, we will now talk about tracing. I’ll say right away that this is a project in which the traces turned out. And it was a project, one might say, a very long test cat, because in the beginning we could look at the production of this product, at its release, in which there were no traces at all, then they appeared and now they are improving. Those. such laboratory studies are obtained: when we take the zero level, then we have some changes, there is an opportunity to see what happened and change something further.
.
I do system analysis in IT. I myself have been working on IT for a very long time, in system analysis for about 5 years. Passed from testers and, moreover, being a tester, the idea that connections between requirements and test cases are needed did not leave me. Since then, I really love this topic. And now I have such an opportunity, and I decided to tell you about it.
In general, on the slide you can see what I do, what I am proud of, with what questions I can contact.
• Tracing in educational literature - silver bullet
• Tracing in practice is difficult and difficult.
• Traces in research - challenges and hopes (see ollygotel.com/requirements-traceability )
• I am a perfectionist and practitioner
• There are experimental data - I want to share
Briefly about the definition:
[Gotel and Finkelstein 1994]
Or speaking in Russian:
If we look into educational literature (Wigers, etc.), everyone says that tracing needs to be conducted, without them impact analysis is impossible, accurate planning and current understanding of where the project is located is impossible. But in reality, most often it is very difficult and difficult. Such an experience, when it is difficult and difficult, is also on my project. And at the same time, apart from the experience of people like Wigers, i.e. people from the industry who write textbooks, and my own experience, here is information about how the computer science community is related to the traceability topic. Here I gave a link to the site of the lady who deals with this as an independent researcher in New York. She enters the community of people who develop the Traceability Body of Knowledge. They have a whole roadmap, how they plan to develop this discipline and her site has a lot of interesting, but very scientific. Accordingly, in my report there will be slides that I will turn over, because there is a lot of text there. Then you can watch them. Basically, I’ll talk about most of this. I want this presentation to help in some specific cases later. It is clear that now everyone is tired and wants pictures. So now there will be pictures, and then, if you want, look at the text.
I have experimental data based on 3 releases: release 1, which passed without tracing, release 2, which passed with traces, and now we have planned release 3, which will take place with improved traces.
• 1 of Kaspersky Lab products
• 3 releases (past, present, future)
• Impact of business requirements
• Influence of architecture
• Influence of the environment (tools, practices, etc.)
So, this is the context of our project. This is one of the Kaspersky Lab products. Release about once a year. The product is quite complicated. I came about mid-release 1. We will discuss a little how requirements management was built and, in principle, creating requirements in all releases, and also see what affects it: collecting business requirements, project structure, company culture. The fact is that in our company several products are being developed at the same time, and accordingly, it turns out that the environment can greatly influence this all.
Picture №1. When I came to this project, these were the systems (white squares in the diagram) that we used to manage the requirements. This is Excel, it contained a list of business requirements at the level “I want this feature to be” (quite large). For example, "device management." Plus a paragraph like "flash drives to manage on laptops." This is a business requirement. And such - the whole list in Excel.
In addition, because the product was built on the basis of another product, besides such business requirements there were also requirements in the form “take a feature from there and insert it into a new product” or “I want this, but please, at that angle and so that the user rights management system was”. This is the first level of description - the level of business requirements.
The second level is the system requirements for the functional area, i.e. clarifying business requirements that are contained in another Excel file. Those. there is a feature “device management” - excellent, we are doing an Excel file “device management” and requirements are being kept in it. In some places there are pictures, diagrams, etc. But the main set is a file in which all requirements are maintained. Those. we can supply, say, a link from a business requirement to an Excel file, but I would not say that it is a trace. In addition, it turns out, we see the artifacts with which the system analysts work, and I also highlighted the project plan and mail here.
Accordingly, when we talk about business requirements and system requirements, we always remember what they are for. RM needs them to plan a project, and the team needs them to develop. In any case, the project plan and other artifacts. Because On this project, analysts worked with Excel and mail, I do not cite the rest of the systems (for example, there are systems for maintaining versions of code files, but this is outside the scope of the presentation).
What problems did I, as an analyst, face? The project manager asks: "Is this business requirement part of the scop or not?" But I don’t know, because business requirements are conducted by business analysts and they don’t allow them to dig into their files. I can't be sure. The only thing that I can mark somewhere in my house and follow them carefully. And since There are 4 analysts, then there is a situation where they have saved something wrong or wrong, and inaccuracies accumulate.
When the questions “and we even implemented this functionality?” Begin, we again do not know. We issued the requirements in the form of an Excel file, but we don’t know how they were tested and developed, if the developer read all the lines. Until the end of the release, there is still the fear that “suddenly something will happen tomorrow?”, Another business requirement was forgotten, something was not implemented. And nothing is clear, all the relationships are “on the fingers”.
Because release only once a year and for the year the product manager always has new ideas, then we are trying to add something to this release. And it turns out, when we want to understand how much the scop will change, if a new business requirement is added, we again do not know. Because the project plan is a kind of “sausage” that is not related to this: it is very difficult to count. It takes a lot of team-lead time for analytics and development, as well as a test manager, to make any decision in order to understand the current state of the project. And when the development was finished and testing started, the situation becomes even worse, because even though there are bugs, everything is tested from what has been done, it is not known. And the absence of bugs says either that everything is well developed, or everything is bad, or we just haven't tested it properly.
At the time of release, we have a rather unclear scheme with which, I think, many are familiar. Someone happens similar, but not so terrible. Others have no requirements at all, and, in fact, have nothing to follow (this is typical for small Agile projects, but their practice does not scale well for large projects).
We received a list of calls (people) - these are the questions that were asked of me and I could not answer them.
• It is difficult to determine whether this business requirement is included in the release rate or not (C1)
• It is difficult to determine how adding a business requirement will affect the scop release (C2)
• Difficult to determine release status (what remains to do, what has already been done) (C3)
• Difficult to reconcile requirements (SRS for functional area - large, tabular format is not convenient for working with changes and comments (C4 - not applicable to traces)
• It is difficult to accurately plan team work, because The functional area is too large to plan (C5).
• It is difficult to accurately plan the amount of development work in a particular area, since number of requirements unknown (C6).
• It is difficult to quickly determine the quality of development of a specific functional, since the requirements and test cases are not related (C7).
• It is difficult to precisely plan the implementation of the functional, since it is unclear what the adjacent teams have already put and what is not (C8).
In the end, we somehow survived, passed the release, were happy when it ended. But life does not end there. I came to the previous release as an analysts timlid, and I had to reach for the release (“everything for the front, everything for victory!”) - on my word of honor, on professionalism, on excellent testers, on a good team (and also on snot ” and "on the claws" ). But this does not end there - we need to take on the next release.
We had a little time to think about what needs to be taken into account in order to make it easier, and than we can answer the same challenges: business requirements are made or not, are they in or out, tested or not, what's going on and what we have to do before the release.
Then we began to consider what needs to be considered, what affects our project - not even the release, but its preparation. We considered that we have 3 groups of sources of requirements for the release:
What is the meaning of it? It helps to understand where features come from, especially in a high-tech (semi-scientific) boxed product such as an antivirus. In fact, the quadrant is one of the analytical studies that makes an overview of industry trends. And how does this happen? The company receives a questionnaire with a lot of questions in the style of "is there such a feature in your product", "how many clients do you have", etc. As a result, each company fills in all the sections in the questionnaire for its product, and after processing by an analytical agency, a report is obtained containing a display of the companies supplying the products in a specific segment of the industry where the position of the companies is determined by 2 axes: fullness of vision and ability to implement. square divided into 4 categories: niche players, applicants, visionaries and leaders. And the source of the new functionality in the product will be such questions as “Do you have data encryption? If so, which one? ”Etc.
In fact, in boxed products, the source of requirements for the product is not only questionnaires, but the overall trend of the industry: you need to look at the competitors, at their planned developments. However, reviews of analytical agencies are an important source of requirements.
As a result, the degree of detail of the features required to implement, in principle, coincides with what was requested in the first release in the form “I want a feature”. Those. Certain sections of the questionnaire were successfully covered with product functionality. And now we need to understand exactly how this functionality should be implemented. (Tracing is required)
The following is a graph of software tests of the VB100 group on the level of threat detection. Our company sells, in fact, its expertise, so our product must win in the tests. In external tests for detection, performance, etc. Those. we give it for testing, they chase it and get some kind of result. The organization and support of this process is carried out by our research divisions, which, unlike analytical agencies that track industry trends, exceed trends, “look beyond the horizon”, invent new things that will allow us to become better, and if we don’t become better, then would enter the top three. Those. a constant requirement is to ensure entry into the top 3. It turns out that it is important for these people to know how the technologies that they invented were implemented, how the analysts wrote the requirements, with the help of which tests it was tested. Thus, it is important for them to have traces from the initial business requirements to both the requirements in the specifications and the test cases.
Another (or rather two) source of requirements is the requirements of vendor certifications and regulators (for example, FSTEC in Russia, Common Criteria abroad, legislation of different countries). If you want to specify on the package "Works under Windows 8", your product needs to meet the requirements of Microsoft. If you want to deliver your product to government agencies, you must first certify it with the FSTEC, and sometimes with the FSB. And for each of these cases, we also need traces to track compliance with the initial certification requirements in the product.
Thus, we see at least 3 groups of consumers of trace requirements. It turned out that for our products it is critically important to have traces. We have customers who need to keep track of whether certain features are implemented in the product. One of the first lessons: no consumer - no need to do this, find those who really need it.
Next we discuss the product architecture. 2 products are drawn here: for individuals (for example, Kaspersky Internet Security 2013) and one that stands in the corporate network. It turns out that the network product is similar to the first, but also communicates with the administrative part. At the same time, both products are recruited from the component kit (or PDK - product development kit). These are "cubes" from which we collect our product. For example, a little earlier I was talking about innovative anti-virus technologies: they are “packaged” into a cube, and then we insert it into products. The result is a complex internal structure from different cubes wrapped in product logic, plus if it is a corporate product, then it will work under the control of another product. In such a situation, we are still obliged to provide the user with reliable protection and quick response to the call: only the virus has appeared, we must immediately provide the treatment and deliver it to the user quickly. To do this, we have infrastructure services that live in the cloud.
Thus, each product consists of components, sometimes it can work under the administration of the administrative part and it always communicates with cloud services. And in order (in the cloud) to have relevant data that will help users protect themselves, research units that carry out virus analysis work (by the way, virus analysts are much more important than system analysts at Kaspersky Lab) should be able to transfer their results from internal systems (Kaspersky network in the diagram) on the front-end to the cloud.
And now the funniest. We see a set of components on the product chart, each of which is done by completely different people; other teams do this. And in order to release a single product, they all have to do their job on time. Those. we must first tell them in time what we want from them (pass on the demands to them). And then monitor whether we are able to implement or not have time to put them in part (this is possible only through traces). For example, while developing an infrastructure service, they can still wait (the user does not have the product yet, no one is accessing the service), but if it concerns components, their absence will delay the product developers who cannot write the code on time.
What is a boxed product? This means that at hour X the product is released, and by this point everyone should be ready to fulfill their part of the work. Until hour X, money is spent on a marketing campaign (for example, “February 21 in New York, Yevgeny Kaspersky presents his new product KESB”). This is a huge scope of work that we, the developers, do not see. If we delay the release for a day, this is big money, a very serious decision and big consequences.
We, as a product sales team, have come to the conclusion that we have customers who need to be sure that we have done everything qualitatively, what we need, and on time. And for this to work a huge team. And thus, we have precisely determined that we have consumers of traces at different levels.
• Ease of creating requirements for a team of analysts - to the maximum
• Convenience of supporting trace requirements - as much as possible
• Changes for the development team - minimal
• Time spent on reformatting / changing formats - minimal
• The invention of bicycles is minimal
As a team leader of analysts, I tried to minimize changes for the team: i.e. the developer should not concern the process of obtaining requirements. But the developers went further - they wanted each developer’s task to be a requirement. In fact, this is an analyst's dream: so that developers and testers work on the requirements. Look at the dream come true (below).
The diagram shows the product traces that were used in the second release. Of course, the number of systems used for this is large. But below I will tell you what we did.
First, business requirements were driven by SharePoint. We did not put pressure on our business division: they want the consolidated requirements and it’s okay: we’ll break them down. And after that, we (the analysts) will have real business requirements that can be calculated (for example, the release will consist of not 40 requirements, but 40). In addition, SharePoint maintained statuses for each request (included / not included in the release, if so, which, responsible analyst, etc.). we showed a picture of the scope of the release of the product, which could be managed and which could be considered. We took the idea and the tool for the implementation of the list from the adjacent team (reuse of ideas ).
In TFS, we had business requirements, system requirements and requests for changes to third-party teams. I will say briefly: we developed requirements in Enterprise Architect in order to be able to create single requirements, from which SRS could be assembled, and then harmonized, etc. The developers wrote a one-way synchronizer from EA to TFS so that the entire team could work with the requirements in TFS.
I draw your attention, the requirements were created and stored in EA, as a source of knowledge, and requirements management took place in TFS. It was organized because it was more convenient for the team. It turns out that in EA we make a Word file that is consistent with customers (including lawyers who do not want to read Excel). And the development team uses TFS. As for external commands, not all of them have TFS. Previously, we had to make change requests for them in a separate system, and now it has become available to us to create in their scheme a dummy element, which is a link to ours. As a result, we use the experience that we have, and do not invent extra bikes. For example, a team of analysts of another product came up with a scheme for working in EA. The development of third products was also carried out. Work with change requests in TeamTrack was developed by another team and so on. As a result, we’ve got the requirement transfer scheme presented on the slide, and we have lived with it the whole release 2.
In detail: analysts lead macro-business requirements (Epic-BRQ) in TFS. At the same time, they lay them out into smaller ones (BRQ). Next to each BRQ traced system requirements (SR) that implement it. However, one SR may be necessary to implement multiple business requirements. Down from SR, an arrow goes to a third-party command to implement change-requests (CR): for each such request, a reference to the specification of the business requirement that needs to be implemented (SRS) is indicated.
But here we had an interesting moment: before that, the team adopted the practice, when component requirements are traced to product requirements, so that it is clear through which component requirement the given product requirement is implemented. In our case, the RMs agreed not to conduct such traces. We will not discuss why they decided so, but we can look at the results of the experiment: in the end, after 3 months, we had to map 400 system requirements to all CRs. And it took because if the requirement is a development task, then the developer must know whether he can do it already or not. And if the component has not yet been delivered, there is nothing for him to work with - this task cannot be taken yet. It turns out that tracing between the requirements for different teams will help PM to make a plan and will give information on what can be implemented and what cannot. This analyst only needs to describe the business requirement and then transfer the system requirements to other commands. And that's all. And in RM, everything does not end there: he understands what requirements he is already ready to implement, he understands which commands and which CRs he has to deliver (and he will not do this without a trace).When CR is implemented, we can proceed to the implementation of the associated SR.
The scheme is, of course, complicated, but in our architecture it is justified.
Question: what is the green block of the SRS, which includes streams of two SR and CR?
Answer: This is a report that we generate for system requirements in the context of BRQ and lay out on the SharePoint portal. There is also a link “up”: in each business requirement we give a link to the set of requirements that implements it. If the implementation of a business requirement requires the work of several teams, then we give a link to the general SRS, which indicates which teams and which CRs are performed. This is done not by an analyst, but by an architect, i.e. makes breaking into systems.
Question: Why was the connection between SR in EA and TFS created? What does it mean?
Answer: We need the system requirements in EA in order to work and receive reports conveniently. And on demand the developer and the tester working on it are registered in TFS. And at the time of iteration planning, we distribute a set of system requirements for implementation to each developer. Of course, it was possible to give a link to SRS (which is on SharePoint), but our developers did not want to work on the document - it’s more convenient for them to work on a single element. And with EA, they do not work at all.
As a result, we publish SR in 2 places (TFS and SRS) and we understand that there is a risk of their divergence, so we carefully (manually) monitor their relevance.
Question: Non-functional requirements you traced?
Answer: Yes, they are included in the functional area. It should be understood that SR is not just one kind of element: it can be use-case, atomic requirement, etc. But the difficulty is: for the programmer, SR becomes a unit of work, and we begin to consider release in the requirements.
Question: Use-case is an atomic requirement? What to do if it can not be implemented in the unit of work? Is the UseCase 2.0 approach used?
Answer: Yes, use-case is a unit of requirements. And if in one iteration it cannot be implemented, then the development team decomposes it as it is more convenient for it. But at the same time he himself bore responsibility for the final result on all the requirements for the final iteration. And as a result, the entire requirement was tested. And the partially made use-case didn't get out.
Question: The release development cycle for you is 1 year. Did you track changes in requirements?
Answer: Yes, we immediately synchronized the requirements. Moreover, the situation is possible 2 types: either corrected the comma and changes in the code is not required, or I make a significant change in the requirement, which can be already implemented and tested. In this case, it becomes active, and it will be redone. This was done through synchronization options. And in TFS there is a history of the requirement. And we made such a revision so that when synchronizing, the changes were displayed there as in the Word review mode. We used versioning of requirements in EA, but this is a slightly different topic outside of this report.
Question: How did the requirements communicate with the test cases?
Answer: In the diagram, they are not drawn by themselves. We in the second release could not make demands on test cases. Purely physically failed to make so many changes in the process.
• (Time Driven Development)
•
• ,
2
• - (BRQ) SharePoint scope ( — )
• EA ( — xls-)
• SRS
• SRS — doc- - ( — xls- )
• — BRQ ( — )
• BRQ (C4, C5)
• - ( «in scope/out of scope» .) (1)
• , BRQ, ( . , ) (6)
• SRS BRQ (4)
• SR, (2, 3, 6, 8)
,
• : BRQ — SRS ( )
• /: BRQ- CR- SRS (BRQ-InfraCR-InfraSRS)
• : —
• : BRQ-SR
• : SR — DevTask
• : BRQ-CR
• : SR — CR
• BRQ () — SR (), BRQ () — CR ()
, - «» . , , .. - . :
• -
•
•
During the release of the second release on other projects, life did not stop, and they also did something. In this case, the entire RnD began to translate to TFS. There was a dedicated team that made improvements in it.
Everything that you see on the diagram will be implemented in TFS. This is a "poet's dream"!
The first level is business requirements. Get away from SharePoint. The neighboring project made this scheme on TFS (hooray!) And now no mysterious links are needed: everything is in one place.
The next level is change requests. Now neighboring teams also work in TFS. C them understand how to work.
The third level is the system requirements, which are tied to both business requirements and change-requests. Duplication of links is left in order to understand what is ready and what is not at a given point in time. Developers are working on a specific requirement and this is a great burden for the analyst: now his requirement will definitely be read. And if earlier it was possible to overlook something in Excel, then surely they will come to you anyway. Because of this, we found a bunch of holes in the functionality that were never tested. Not because the testers are bad, but because they did not have time or something else. Accordingly, the accuracy and quality really improved.
At the same time, there were fights with analysts like “divide my business in half”, when the developers wanted to get a problem statement instead of requirements. If you implement this approach at home, think over the question of such battles in advance - this is a very important point. It is easy to slide into the level of “what needs to be done” instead of “what should be the result.”
The last level is the binding of requirements for test cases and bugs. Some projects are already working this way. It turns out that all project artifacts (including those belonging to other project teams) are in one place, and we hope that this release will be the happiest.
: . , -, TFS ( TFS – , ). , RTF PDF - TFS, – .
. TFS , baseline . , «», « » baseline TFS . . , , , . , , , .. , , .
• - (LL1)
• / (LL2), ( )
• (TestCase ) (C7, LL3)
• (LL4)
, ( « » Agile)
• -
, - – , . , , .
•
• ,
• ( ) – --,
• – , , , . – , .. .
• !
o ( , , ; .. )
o
•
•
• , ,
•
Video report:
')
Presentation of the report:
www.slideshare.net/VLDCORP/ss-21934265
Good afternoon, we will now talk about tracing. I’ll say right away that this is a project in which the traces turned out. And it was a project, one might say, a very long test cat, because in the beginning we could look at the production of this product, at its release, in which there were no traces at all, then they appeared and now they are improving. Those. such laboratory studies are obtained: when we take the zero level, then we have some changes, there is an opportunity to see what happened and change something further.
About me
- In IT for more than 10 years, of which
- 5 years in system analysis
- experience of managing a team of analysts
- experience in creating and supporting development processes
- Co-author of the Applied Systems Analysis Club of the Stratoplan.ru project
- Member of the analyst community uml2.ru
- responsible for tools in the analysis department
.
I do system analysis in IT. I myself have been working on IT for a very long time, in system analysis for about 5 years. Passed from testers and, moreover, being a tester, the idea that connections between requirements and test cases are needed did not leave me. Since then, I really love this topic. And now I have such an opportunity, and I decided to tell you about it.
In general, on the slide you can see what I do, what I am proud of, with what questions I can contact.
About the topic
• Tracing in educational literature - silver bullet
• Tracing in practice is difficult and difficult.
• Traces in research - challenges and hopes (see ollygotel.com/requirements-traceability )
• I am a perfectionist and practitioner
• There are experimental data - I want to share
Briefly about the definition:
Requirements traceability of the need to follow up and iteration in any of these phases.)
[Gotel and Finkelstein 1994]
Or speaking in Russian:
Requirements tracing refers to the ability to describe and follow the life of requirements, in both the forward and reverse direction (that is, from their origins, through their development and specifications, to their consistent implementation and subsequent use, and through all periods of constant refinement in any of these phase.)
If we look into educational literature (Wigers, etc.), everyone says that tracing needs to be conducted, without them impact analysis is impossible, accurate planning and current understanding of where the project is located is impossible. But in reality, most often it is very difficult and difficult. Such an experience, when it is difficult and difficult, is also on my project. And at the same time, apart from the experience of people like Wigers, i.e. people from the industry who write textbooks, and my own experience, here is information about how the computer science community is related to the traceability topic. Here I gave a link to the site of the lady who deals with this as an independent researcher in New York. She enters the community of people who develop the Traceability Body of Knowledge. They have a whole roadmap, how they plan to develop this discipline and her site has a lot of interesting, but very scientific. Accordingly, in my report there will be slides that I will turn over, because there is a lot of text there. Then you can watch them. Basically, I’ll talk about most of this. I want this presentation to help in some specific cases later. It is clear that now everyone is tired and wants pictures. So now there will be pictures, and then, if you want, look at the text.
I have experimental data based on 3 releases: release 1, which passed without tracing, release 2, which passed with traces, and now we have planned release 3, which will take place with improved traces.
Release 1. Context
• 1 of Kaspersky Lab products
• 3 releases (past, present, future)
• Impact of business requirements
• Influence of architecture
• Influence of the environment (tools, practices, etc.)
So, this is the context of our project. This is one of the Kaspersky Lab products. Release about once a year. The product is quite complicated. I came about mid-release 1. We will discuss a little how requirements management was built and, in principle, creating requirements in all releases, and also see what affects it: collecting business requirements, project structure, company culture. The fact is that in our company several products are being developed at the same time, and accordingly, it turns out that the environment can greatly influence this all.
Project Artifacts
Picture №1. When I came to this project, these were the systems (white squares in the diagram) that we used to manage the requirements. This is Excel, it contained a list of business requirements at the level “I want this feature to be” (quite large). For example, "device management." Plus a paragraph like "flash drives to manage on laptops." This is a business requirement. And such - the whole list in Excel.
In addition, because the product was built on the basis of another product, besides such business requirements there were also requirements in the form “take a feature from there and insert it into a new product” or “I want this, but please, at that angle and so that the user rights management system was”. This is the first level of description - the level of business requirements.
The second level is the system requirements for the functional area, i.e. clarifying business requirements that are contained in another Excel file. Those. there is a feature “device management” - excellent, we are doing an Excel file “device management” and requirements are being kept in it. In some places there are pictures, diagrams, etc. But the main set is a file in which all requirements are maintained. Those. we can supply, say, a link from a business requirement to an Excel file, but I would not say that it is a trace. In addition, it turns out, we see the artifacts with which the system analysts work, and I also highlighted the project plan and mail here.
Accordingly, when we talk about business requirements and system requirements, we always remember what they are for. RM needs them to plan a project, and the team needs them to develop. In any case, the project plan and other artifacts. Because On this project, analysts worked with Excel and mail, I do not cite the rest of the systems (for example, there are systems for maintaining versions of code files, but this is outside the scope of the presentation).
What problems did I, as an analyst, face? The project manager asks: "Is this business requirement part of the scop or not?" But I don’t know, because business requirements are conducted by business analysts and they don’t allow them to dig into their files. I can't be sure. The only thing that I can mark somewhere in my house and follow them carefully. And since There are 4 analysts, then there is a situation where they have saved something wrong or wrong, and inaccuracies accumulate.
When the questions “and we even implemented this functionality?” Begin, we again do not know. We issued the requirements in the form of an Excel file, but we don’t know how they were tested and developed, if the developer read all the lines. Until the end of the release, there is still the fear that “suddenly something will happen tomorrow?”, Another business requirement was forgotten, something was not implemented. And nothing is clear, all the relationships are “on the fingers”.
Because release only once a year and for the year the product manager always has new ideas, then we are trying to add something to this release. And it turns out, when we want to understand how much the scop will change, if a new business requirement is added, we again do not know. Because the project plan is a kind of “sausage” that is not related to this: it is very difficult to count. It takes a lot of team-lead time for analytics and development, as well as a test manager, to make any decision in order to understand the current state of the project. And when the development was finished and testing started, the situation becomes even worse, because even though there are bugs, everything is tested from what has been done, it is not known. And the absence of bugs says either that everything is well developed, or everything is bad, or we just haven't tested it properly.
At the time of release, we have a rather unclear scheme with which, I think, many are familiar. Someone happens similar, but not so terrible. Others have no requirements at all, and, in fact, have nothing to follow (this is typical for small Agile projects, but their practice does not scale well for large projects).
Revealed difficulties:
We received a list of calls (people) - these are the questions that were asked of me and I could not answer them.
• It is difficult to determine whether this business requirement is included in the release rate or not (C1)
• It is difficult to determine how adding a business requirement will affect the scop release (C2)
• Difficult to determine release status (what remains to do, what has already been done) (C3)
• Difficult to reconcile requirements (SRS for functional area - large, tabular format is not convenient for working with changes and comments (C4 - not applicable to traces)
• It is difficult to accurately plan team work, because The functional area is too large to plan (C5).
• It is difficult to accurately plan the amount of development work in a particular area, since number of requirements unknown (C6).
• It is difficult to quickly determine the quality of development of a specific functional, since the requirements and test cases are not related (C7).
• It is difficult to precisely plan the implementation of the functional, since it is unclear what the adjacent teams have already put and what is not (C8).
Release 1 finished
In the end, we somehow survived, passed the release, were happy when it ended. But life does not end there. I came to the previous release as an analysts timlid, and I had to reach for the release (“everything for the front, everything for victory!”) - on my word of honor, on professionalism, on excellent testers, on a good team (and also on snot ” and "on the claws" ). But this does not end there - we need to take on the next release.
We must do the next release
Release 2
Thinking about the future
We had a little time to think about what needs to be taken into account in order to make it easier, and than we can answer the same challenges: business requirements are made or not, are they in or out, tested or not, what's going on and what we have to do before the release.
Then we began to consider what needs to be considered, what affects our project - not even the release, but its preparation. We considered that we have 3 groups of sources of requirements for the release:
Gartner's Magic Quadrant and other industry analysis reports
What is the meaning of it? It helps to understand where features come from, especially in a high-tech (semi-scientific) boxed product such as an antivirus. In fact, the quadrant is one of the analytical studies that makes an overview of industry trends. And how does this happen? The company receives a questionnaire with a lot of questions in the style of "is there such a feature in your product", "how many clients do you have", etc. As a result, each company fills in all the sections in the questionnaire for its product, and after processing by an analytical agency, a report is obtained containing a display of the companies supplying the products in a specific segment of the industry where the position of the companies is determined by 2 axes: fullness of vision and ability to implement. square divided into 4 categories: niche players, applicants, visionaries and leaders. And the source of the new functionality in the product will be such questions as “Do you have data encryption? If so, which one? ”Etc.
In fact, in boxed products, the source of requirements for the product is not only questionnaires, but the overall trend of the industry: you need to look at the competitors, at their planned developments. However, reviews of analytical agencies are an important source of requirements.
As a result, the degree of detail of the features required to implement, in principle, coincides with what was requested in the first release in the form “I want a feature”. Those. Certain sections of the questionnaire were successfully covered with product functionality. And now we need to understand exactly how this functionality should be implemented. (Tracing is required)
Participation in tests of independent laboratories.
The following is a graph of software tests of the VB100 group on the level of threat detection. Our company sells, in fact, its expertise, so our product must win in the tests. In external tests for detection, performance, etc. Those. we give it for testing, they chase it and get some kind of result. The organization and support of this process is carried out by our research divisions, which, unlike analytical agencies that track industry trends, exceed trends, “look beyond the horizon”, invent new things that will allow us to become better, and if we don’t become better, then would enter the top three. Those. a constant requirement is to ensure entry into the top 3. It turns out that it is important for these people to know how the technologies that they invented were implemented, how the analysts wrote the requirements, with the help of which tests it was tested. Thus, it is important for them to have traces from the initial business requirements to both the requirements in the specifications and the test cases.
External environment
Another (or rather two) source of requirements is the requirements of vendor certifications and regulators (for example, FSTEC in Russia, Common Criteria abroad, legislation of different countries). If you want to specify on the package "Works under Windows 8", your product needs to meet the requirements of Microsoft. If you want to deliver your product to government agencies, you must first certify it with the FSTEC, and sometimes with the FSB. And for each of these cases, we also need traces to track compliance with the initial certification requirements in the product.
Thus, we see at least 3 groups of consumers of trace requirements. It turned out that for our products it is critically important to have traces. We have customers who need to keep track of whether certain features are implemented in the product. One of the first lessons: no consumer - no need to do this, find those who really need it.
Product architecture
Next we discuss the product architecture. 2 products are drawn here: for individuals (for example, Kaspersky Internet Security 2013) and one that stands in the corporate network. It turns out that the network product is similar to the first, but also communicates with the administrative part. At the same time, both products are recruited from the component kit (or PDK - product development kit). These are "cubes" from which we collect our product. For example, a little earlier I was talking about innovative anti-virus technologies: they are “packaged” into a cube, and then we insert it into products. The result is a complex internal structure from different cubes wrapped in product logic, plus if it is a corporate product, then it will work under the control of another product. In such a situation, we are still obliged to provide the user with reliable protection and quick response to the call: only the virus has appeared, we must immediately provide the treatment and deliver it to the user quickly. To do this, we have infrastructure services that live in the cloud.
Thus, each product consists of components, sometimes it can work under the administration of the administrative part and it always communicates with cloud services. And in order (in the cloud) to have relevant data that will help users protect themselves, research units that carry out virus analysis work (by the way, virus analysts are much more important than system analysts at Kaspersky Lab) should be able to transfer their results from internal systems (Kaspersky network in the diagram) on the front-end to the cloud.
And now the funniest. We see a set of components on the product chart, each of which is done by completely different people; other teams do this. And in order to release a single product, they all have to do their job on time. Those. we must first tell them in time what we want from them (pass on the demands to them). And then monitor whether we are able to implement or not have time to put them in part (this is possible only through traces). For example, while developing an infrastructure service, they can still wait (the user does not have the product yet, no one is accessing the service), but if it concerns components, their absence will delay the product developers who cannot write the code on time.
What is a boxed product? This means that at hour X the product is released, and by this point everyone should be ready to fulfill their part of the work. Until hour X, money is spent on a marketing campaign (for example, “February 21 in New York, Yevgeny Kaspersky presents his new product KESB”). This is a huge scope of work that we, the developers, do not see. If we delay the release for a day, this is big money, a very serious decision and big consequences.
We, as a product sales team, have come to the conclusion that we have customers who need to be sure that we have done everything qualitatively, what we need, and on time. And for this to work a huge team. And thus, we have precisely determined that we have consumers of traces at different levels.
Process Change Requirements
• Ease of creating requirements for a team of analysts - to the maximum
• Convenience of supporting trace requirements - as much as possible
• Changes for the development team - minimal
• Time spent on reformatting / changing formats - minimal
• The invention of bicycles is minimal
As a team leader of analysts, I tried to minimize changes for the team: i.e. the developer should not concern the process of obtaining requirements. But the developers went further - they wanted each developer’s task to be a requirement. In fact, this is an analyst's dream: so that developers and testers work on the requirements. Look at the dream come true (below).
The diagram shows the product traces that were used in the second release. Of course, the number of systems used for this is large. But below I will tell you what we did.
First, business requirements were driven by SharePoint. We did not put pressure on our business division: they want the consolidated requirements and it’s okay: we’ll break them down. And after that, we (the analysts) will have real business requirements that can be calculated (for example, the release will consist of not 40 requirements, but 40). In addition, SharePoint maintained statuses for each request (included / not included in the release, if so, which, responsible analyst, etc.). we showed a picture of the scope of the release of the product, which could be managed and which could be considered. We took the idea and the tool for the implementation of the list from the adjacent team (reuse of ideas ).
In TFS, we had business requirements, system requirements and requests for changes to third-party teams. I will say briefly: we developed requirements in Enterprise Architect in order to be able to create single requirements, from which SRS could be assembled, and then harmonized, etc. The developers wrote a one-way synchronizer from EA to TFS so that the entire team could work with the requirements in TFS.
I draw your attention, the requirements were created and stored in EA, as a source of knowledge, and requirements management took place in TFS. It was organized because it was more convenient for the team. It turns out that in EA we make a Word file that is consistent with customers (including lawyers who do not want to read Excel). And the development team uses TFS. As for external commands, not all of them have TFS. Previously, we had to make change requests for them in a separate system, and now it has become available to us to create in their scheme a dummy element, which is a link to ours. As a result, we use the experience that we have, and do not invent extra bikes. For example, a team of analysts of another product came up with a scheme for working in EA. The development of third products was also carried out. Work with change requests in TeamTrack was developed by another team and so on. As a result, we’ve got the requirement transfer scheme presented on the slide, and we have lived with it the whole release 2.
In detail: analysts lead macro-business requirements (Epic-BRQ) in TFS. At the same time, they lay them out into smaller ones (BRQ). Next to each BRQ traced system requirements (SR) that implement it. However, one SR may be necessary to implement multiple business requirements. Down from SR, an arrow goes to a third-party command to implement change-requests (CR): for each such request, a reference to the specification of the business requirement that needs to be implemented (SRS) is indicated.
But here we had an interesting moment: before that, the team adopted the practice, when component requirements are traced to product requirements, so that it is clear through which component requirement the given product requirement is implemented. In our case, the RMs agreed not to conduct such traces. We will not discuss why they decided so, but we can look at the results of the experiment: in the end, after 3 months, we had to map 400 system requirements to all CRs. And it took because if the requirement is a development task, then the developer must know whether he can do it already or not. And if the component has not yet been delivered, there is nothing for him to work with - this task cannot be taken yet. It turns out that tracing between the requirements for different teams will help PM to make a plan and will give information on what can be implemented and what cannot. This analyst only needs to describe the business requirement and then transfer the system requirements to other commands. And that's all. And in RM, everything does not end there: he understands what requirements he is already ready to implement, he understands which commands and which CRs he has to deliver (and he will not do this without a trace).When CR is implemented, we can proceed to the implementation of the associated SR.
The scheme is, of course, complicated, but in our architecture it is justified.
Question 1
Question: what is the green block of the SRS, which includes streams of two SR and CR?
Answer: This is a report that we generate for system requirements in the context of BRQ and lay out on the SharePoint portal. There is also a link “up”: in each business requirement we give a link to the set of requirements that implements it. If the implementation of a business requirement requires the work of several teams, then we give a link to the general SRS, which indicates which teams and which CRs are performed. This is done not by an analyst, but by an architect, i.e. makes breaking into systems.
Question 2
Question: Why was the connection between SR in EA and TFS created? What does it mean?
Answer: We need the system requirements in EA in order to work and receive reports conveniently. And on demand the developer and the tester working on it are registered in TFS. And at the time of iteration planning, we distribute a set of system requirements for implementation to each developer. Of course, it was possible to give a link to SRS (which is on SharePoint), but our developers did not want to work on the document - it’s more convenient for them to work on a single element. And with EA, they do not work at all.
As a result, we publish SR in 2 places (TFS and SRS) and we understand that there is a risk of their divergence, so we carefully (manually) monitor their relevance.
Question 3
Question: Non-functional requirements you traced?
Answer: Yes, they are included in the functional area. It should be understood that SR is not just one kind of element: it can be use-case, atomic requirement, etc. But the difficulty is: for the programmer, SR becomes a unit of work, and we begin to consider release in the requirements.
Question 4
Question: Use-case is an atomic requirement? What to do if it can not be implemented in the unit of work? Is the UseCase 2.0 approach used?
Answer: Yes, use-case is a unit of requirements. And if in one iteration it cannot be implemented, then the development team decomposes it as it is more convenient for it. But at the same time he himself bore responsibility for the final result on all the requirements for the final iteration. And as a result, the entire requirement was tested. And the partially made use-case didn't get out.
Question 4
Question: The release development cycle for you is 1 year. Did you track changes in requirements?
Answer: Yes, we immediately synchronized the requirements. Moreover, the situation is possible 2 types: either corrected the comma and changes in the code is not required, or I make a significant change in the requirement, which can be already implemented and tested. In this case, it becomes active, and it will be redone. This was done through synchronization options. And in TFS there is a history of the requirement. And we made such a revision so that when synchronizing, the changes were displayed there as in the Word review mode. We used versioning of requirements in EA, but this is a slightly different topic outside of this report.
Question 5
Question: How did the requirements communicate with the test cases?
Answer: In the diagram, they are not drawn by themselves. We in the second release could not make demands on test cases. Purely physically failed to make so many changes in the process.
Release Context 2
• (Time Driven Development)
•
• ,
2
• - (BRQ) SharePoint scope ( — )
• EA ( — xls-)
• SRS
• SRS — doc- - ( — xls- )
• — BRQ ( — )
• BRQ (C4, C5)
• - ( «in scope/out of scope» .) (1)
• , BRQ, ( . , ) (6)
• SRS BRQ (4)
• SR, (2, 3, 6, 8)
,
• : BRQ — SRS ( )
• /: BRQ- CR- SRS (BRQ-InfraCR-InfraSRS)
• : —
• : BRQ-SR
• : SR — DevTask
• : BRQ-CR
• : SR — CR
• BRQ () — SR (), BRQ () — CR ()
2.
, - «» . , , .. - . :
• -
•
•
3.
During the release of the second release on other projects, life did not stop, and they also did something. In this case, the entire RnD began to translate to TFS. There was a dedicated team that made improvements in it.
Preparing for release 3. How will
Everything that you see on the diagram will be implemented in TFS. This is a "poet's dream"!
The first level is business requirements. Get away from SharePoint. The neighboring project made this scheme on TFS (hooray!) And now no mysterious links are needed: everything is in one place.
The next level is change requests. Now neighboring teams also work in TFS. C them understand how to work.
The third level is the system requirements, which are tied to both business requirements and change-requests. Duplication of links is left in order to understand what is ready and what is not at a given point in time. Developers are working on a specific requirement and this is a great burden for the analyst: now his requirement will definitely be read. And if earlier it was possible to overlook something in Excel, then surely they will come to you anyway. Because of this, we found a bunch of holes in the functionality that were never tested. Not because the testers are bad, but because they did not have time or something else. Accordingly, the accuracy and quality really improved.
At the same time, there were fights with analysts like “divide my business in half”, when the developers wanted to get a problem statement instead of requirements. If you implement this approach at home, think over the question of such battles in advance - this is a very important point. It is easy to slide into the level of “what needs to be done” instead of “what should be the result.”
The last level is the binding of requirements for test cases and bugs. Some projects are already working this way. It turns out that all project artifacts (including those belonging to other project teams) are in one place, and we hope that this release will be the happiest.
: . , -, TFS ( TFS – , ). , RTF PDF - TFS, – .
. TFS , baseline . , «», « » baseline TFS . . , , , . , , , .. , , .
Lessons Learned
• - (LL1)
• / (LL2), ( )
• (TestCase ) (C7, LL3)
• (LL4)
findings
, ( « » Agile)
• -
, - – , . , , .
•
• ,
• ( ) – --,
• – , , , . – , .. .
• !
o ( , , ; .. )
o
•
•
• , ,
•
Source: https://habr.com/ru/post/221909/
All Articles