What answers do I expect at the testing interview?
I conduct interviews for testers. I sometimes have a headache.
I was going to write an article for a long time ... And finally, I fulfilled my intention. The questions raised in the article have been discussed more than once or twice, but the diligent search for compiling answers to these questions has not been crowned with success. But, as my experience suggests, such a compilation is very necessary. First of all, it is required for juniors, because on the net, on request, “testing”, they (applicants) are hit by a huge amount of informational garbage, which is poorly structured and often contradicts itself.
First a few words about yourself. At the moment I am the head of the testing and maintenance department of a company engaged in corporate GIS. Prior to that, he worked as the head of a testing group in a company developing commercial DLS (Distance Learning Systems). And before that, the lead testing engineer at a company that provided electronic trading for the Federal Law No. 94. And I started my career more than 11 years ago as a system administrator (in three different organizations). The intern programmer was a little less than two years old (zero at the beginning - VB). Freelance software engineer: he wrote his own bug tracker for state-owned companies ... Based on the above, it can be argued that a certain experience (testing - more than 5 years in total) has been gained ...
In addition, on duty, I constantly have to pick up employees in the testing department. And the more I do it, the more I am convinced that sometimes it is easier to take a challenger without experience than a person with testing experience in a Russian company (by the way, not without exceptions). Along the way, it should be noted that applicants without experience in the overwhelming majority use the following sources of information about the profession: Internet resources, books, opinions of testers' friends.
')
At the interview I always ask the same questions:
The applicant, who comes to the eighth question in an hour and a half of conversation, is a rarity, I will take this job as a junior. Arriving at the same time up to 11 questions can be accepted as the lead tester, however, for 240 interviews conducted, only 5 people turned out to be such!
Maybe I'm too picky about answers? No, I'm just waiting for the applicant to understand what he will have to do. Here's how the interview goes: I start talking to the applicant, preferably in the form of a dialogue, asking him the indicated questions. If I get an answer, correct or close to correct, then I proceed to the next question. If the applicant “wanders”, leads a learned formulation, or simply cannot justify it, I try to lead him to the correct answer and why this answer is correct. Trying to make reason. Last year, instead of interviews, I get improvised lectures. And it's not just that applicants are less knowledgeable or have little experience. There were also interviews for the position of lead test engineer with applicants with 10 years of experience ... the result is almost always depressing. In my opinion, the fact is that there is a lot of contradictory information and "unprofitable" experience, because many Russian companies build the testing process according to the model of S. Kaner - when two or three highly qualified testers completely generate, select and describe cases, and check out 10 -15,100, 500+ "testers" not really delving into the very essence of the process.
With this text I will try to slightly bring yesterday's, today's and tomorrow's applicants to the position of the tester to understand, and what is “testing” after all. Next, I will answer some of the interview questions and substantiate my opinion, and also give some of the most frequent answers of applicants and explain why I consider them wrong.
The most frequent answer is: "because it is simple and interesting (!)". That is, the candidate thinks that he will be paid for clicking on the mouse in VK ... Or they will give the software and say - break it ... Or he just did not prepare for this issue and has a very weak idea of ​​the profession.
The second answer is: “because I want to work in IT and testing is the easiest way” (read: IT specialists have high sn, but testing doesn’t need any knowledge or skills, but sn is also quite high!).
There were answers: “my mother / husband / wife made me go for an interview”.
The only correct answer is no, but these three and their derivatives are definitely wrong, because testing is difficult and monotonous, it requires certain skills for which there are no textbooks, and leads to a professional deformation of the worldview.
What would you like to hear? Perhaps something like: "because without testing it is impossible to identify the true state of the product being produced, and how it meets the expectations of the consumer."
The most frequent answer (directly registered with S. Kaner and R. Savin) is “search for errors”. And for some reason nobody in the entire literature on testing indicates that this simplification is very rude, and in general, this answer is simply wrong!
Testing is a set of measures aimed at conducting checks on the conformity of the product being produced to the requirements imposed on it (direct and indirect).
Yes, indeed, during the inspections errors / incidents / observations are revealed, but this is only a by-product of the process. The main information is about the product's compliance with the requirements that apply to it .
Well, here, thank God, almost everyone answers: “incorrect work of the program ...”. But then chaos begins when you ask: “how do we know if the work is correct or not?”
The correct answer is on almost all testing resources known to me:
Error - the discrepancy between the product produced and the requirements, direct or indirect.
In order not to wander in contradictions / assumptions, etc., this is the only correct answer.
Here people begin to repeat the answer to the second question with different variations. The most attentive applicants are trying to retell what I told them when answering the second question. And the answer is extremely simple:
The purpose of testing is to provide up-to-date information on the conformity of the product to the requirements
Everything. No more and no less. Well, of course, you can also say that the purpose of testing is to provide information on the number of errors in the product. Namely this is wrong. Why? This is just the same everyday case of many testers / PM / analysts: the customer’s call is “how is my product?”. “You know, there are 60 more bugs in it!” Is the answer of the tester / PM ... And then what? It's a lot? Few? Fine? You can, of course, tell in detail about the criticality of these bugs, their priorities, but this is not the answer to the customer's question, it is the issuance of raw raw information to fiberboard. Now the same case. “How is my product?”, Asks the customer. “35% of the requirements are fully implemented, another 5% with remarks and another 2% are now in implementation,” the PM / tester answers. Do you think this answer is clearer? And let these 5% include the already mentioned 60 bugs, comments ... The answer to the question is given as accurate as possible in this format. This is exactly the purpose of the test. And, accordingly, the process itself in essence should be reduced to the achievement of this goal.
A lot has been said about life cycle software, and it strongly depends on the organization of the implementation process as a whole. Nevertheless, there is a certain “golden mean”, but even here they manage to fantasize wild things, reducing everything to three points, painting the scheme into three pages ... To everyone who conducted / passed the interview, and so clearly, what mistakes are made and how many options do correct answers. I will not stop in more detail, I will say only that there is a whole pool of candidates who have firmly stalled on this issue (approximately 7%).
Only 50% of applicants reach this question in an hour and a half ... Although I do not demand answers “letter to letter”, the main thing, as lawyers call it, is to preserve “spirit”.
The most frequent case: applicants begin to list the types of technical documentation that they know or have heard about ... I will definitely listen, leave and ask: “anything else?”. Few people remember the division into "functional" / "non-functional", and who remember, often cannot explain the difference.
But there is one category that is forgotten. In this article I have already mentioned several times about "... the requirements of direct and indirect ...". At the interview, I say this phrase five or six times. A very small percentage of applicants asks again and thereby excludes this question from the interview. And the full answer is: “Requirements are direct (i.e., formalized in technical documentation, specs, user-story and other formal artifacts) and indirect (i.e., resulting from direct, or unspoken standard for this product or based on experience and common sense of using this product or products like it). All requirements are also divided into functional (describing what functions the product should perform) and non-functional (requirements for the environment, maintainability, reliability and other characteristics of the product). Direct demands are always higher than indirect ones. ”
The most obvious and “simple” example: in the TK - “the button must be red” is a direct requirement, indirect results flow from it - it should not be blue, green, gray or black, etc. ... Of course, this is a strong simplification, but very revealing. But the main thing is that such an approach cuts off an overly formal attitude to testing and raises the bar for testing qualification as such, because for competent testing it is not enough to know only the TK and user-story, one must also study the applied area and the specificity of consumption of the product. Such testing is much more effective.
There is a little sin behind me: I deny the existence of negative checks because:
Based on the concept of indirect requirements, this is not necessary, since all the checks become positive, but some of them are for compliance with direct requirements, and some for indirect compliance. And the qualification of a specialist is just revealed by an understanding of the indirect requirements for each specific product.
There is a lot of information about the classification of testing, there are also a lot of options for the correct answers. I ask this question in order to see whether the applicant prepared himself, at least in a small degree or not at all bothered. The fact is that the previous questions can be answered simply by reasoning and having a general idea of ​​the sphere as a whole. This question requires basic knowledge of terms. Perhaps I will consider it in other articles, because it is quite large and deserves a separate article.
Test documentation is probably the biggest problem. On it are such battles in communities, firms, etc.! About her so much controversial information. About her published multi-volumes in different languages. There is such a mess in her head ... What kind of answers have you heard (yes, yes, including the TK and the design decision are also test documentation) ... Therefore, I will express my thoughts on this.
Test documentation is of two types: external and internal. And she and the other - a tool that makes life easier for the project team. No more and no less.
External documentation:
Internal documentation:
The goal is to record the generated and selected exponential test in the form that allows the tester of any qualification to conduct it and be able to analyze the results obtained.
As you can see, each subsequent type of internal test documentation to a certain extent details the previous one. Each document has its own purpose and together they are a tool to facilitate the generation, selection and reproduction of test cases. In addition, well-structured, supported, readable, organized and accessible test documentation allows in the long term:
But keep in mind that there are some disadvantages:
Lists of both pros and cons can continue, I indicated only those that lie on the surface. But understanding at least this list is extremely important for the current or future testing specialist. The question regarding test documentation is overcome by a very small percentage of applicants.
Naturally, the types of documentation in testing are much more, but without an understanding of the purpose and features of these documents, working in this profession is very difficult. And in order not to increase the size of the article at all, consider the last (for this article) question.
Most often they answer approximately this way: “preparation, testing, report ...” That's how it is, only any process consists of these steps. And the answer does not reflect the understanding of the applicant testing processes. More like cheating ... Therefore, I will allow myself the statement of my vision:
More information about these stages:
Initiation - an event that notifies the testing team of the need for a testing session, and also ensures that the product requirements for testing are met.
For software production requirements include:
There are other conditions, but they are less significant and highly dependent on the specific process in the company.
Identifying requirements is perhaps one of the main steps in the testing process. Unknown requirements - no testing. It is necessary to collect all available information about the subject of testing, use cases, etc. The first source - technical documentation and user-story - these are direct requirements. The quality of indirect requirements largely depends on the good faith, responsibility, qualifications of the tester and the entire project team.
The generation of test cases is the identification of all possible uses of the product, its characteristics and features during operation. This means: of all cases that a tester can “invent” on the basis of direct and indirect requirements known to him. This stage requires highly qualified testing specialist.
Selection of test cases - the selection of the most significant, significant and reproducible test cases. This stage determines how useful the testing will be, effective and analyzed. For example, in the “simple” example with a red button, it is clear that the number of indirect requirements tends to infinity, and checking them all is absurd, but such cases should be generated at least in the head of the examiner. And in order for them not to enter the checks, it is necessary to perform the appropriate selection and only check whether the button is really red.
The example is primitive, but after voicing it, the applicants no longer try to pour radium on the test task J into the glass (those who took part in the interview for the position of tester know this simple task to generate and select test cases).
Carrying out checks - everything is clear. Either according to the documentation or ad hoc (intuitive, free search, without documentation). In any case, this is carried out according to the list of selected checks. For some reason, most of this item calls testing. And in the head of the philistine, unfamiliar with the profession, only this one point is contained in J.
Recording results - the creation of internal and external test documentation in a formalized form or in the form of records, etc. At this stage, the test report, even if it is created, is not considered complete.
Analysis of the results - the decision on the compliance of the proven product requirements. Formalization of this decision and its justification in the form of a test report. This also includes procedures for assessing the coverage of requirements with checks, time-sharing, etc. Thus, not only the results are analyzed, but also the testing session itself.
Transmission of product compliance information. Formally: the transfer of external test documentation to interested parties, often the initiator of a testing session. In general: in addition to the documentation, information is provided on the risks that have been identified in the product, requirements, processes, recommendations are given on how to work out these risks, etc. But this is already QA J!
The attentive reader noted that questions are gaining complexity from the beginning to the end of the conversation. At the same time, it is worth noting that all these questions should not make it difficult for testing specialists, that is, those who didn’t "just click with the mouse, test", but try to understand the essence of what he is doing, those who didn’t stop at one book or one resource, but thought out and justified his actions ... at least for himself.
This concludes this article. If a dear reader finds a subject for discussion in my statements, I will be glad to participate in it. If there are questions, they will be topics for future articles. Always happy to constructive criticism. In any case, write to me. If an interest is shown in the article, I will continue to analyze the interviews, and, perhaps, I will try to cover other aspects of the profession, about which almost everyone has heard, but few of those who know it from the inside.
I was going to write an article for a long time ... And finally, I fulfilled my intention. The questions raised in the article have been discussed more than once or twice, but the diligent search for compiling answers to these questions has not been crowned with success. But, as my experience suggests, such a compilation is very necessary. First of all, it is required for juniors, because on the net, on request, “testing”, they (applicants) are hit by a huge amount of informational garbage, which is poorly structured and often contradicts itself.
Introduction
First a few words about yourself. At the moment I am the head of the testing and maintenance department of a company engaged in corporate GIS. Prior to that, he worked as the head of a testing group in a company developing commercial DLS (Distance Learning Systems). And before that, the lead testing engineer at a company that provided electronic trading for the Federal Law No. 94. And I started my career more than 11 years ago as a system administrator (in three different organizations). The intern programmer was a little less than two years old (zero at the beginning - VB). Freelance software engineer: he wrote his own bug tracker for state-owned companies ... Based on the above, it can be argued that a certain experience (testing - more than 5 years in total) has been gained ...
In addition, on duty, I constantly have to pick up employees in the testing department. And the more I do it, the more I am convinced that sometimes it is easier to take a challenger without experience than a person with testing experience in a Russian company (by the way, not without exceptions). Along the way, it should be noted that applicants without experience in the overwhelming majority use the following sources of information about the profession: Internet resources, books, opinions of testers' friends.
')
At the interview I always ask the same questions:
- Why did you decide to become a tester?
- What is testing? What is its essence as a process?
- What is a mistake?
- What is the purpose of testing?
- What do you know about the software life cycle?
- What are the requirements?
- What types / types / classes / testing methods do you know, and how do they differ?
- Tell us about the test documentation: types, goals.
- What are the stages of the testing process?
- Automated testing - a separate type of testing?
- What type / type of testing class makes sense to automate?
The applicant, who comes to the eighth question in an hour and a half of conversation, is a rarity, I will take this job as a junior. Arriving at the same time up to 11 questions can be accepted as the lead tester, however, for 240 interviews conducted, only 5 people turned out to be such!
Maybe I'm too picky about answers? No, I'm just waiting for the applicant to understand what he will have to do. Here's how the interview goes: I start talking to the applicant, preferably in the form of a dialogue, asking him the indicated questions. If I get an answer, correct or close to correct, then I proceed to the next question. If the applicant “wanders”, leads a learned formulation, or simply cannot justify it, I try to lead him to the correct answer and why this answer is correct. Trying to make reason. Last year, instead of interviews, I get improvised lectures. And it's not just that applicants are less knowledgeable or have little experience. There were also interviews for the position of lead test engineer with applicants with 10 years of experience ... the result is almost always depressing. In my opinion, the fact is that there is a lot of contradictory information and "unprofitable" experience, because many Russian companies build the testing process according to the model of S. Kaner - when two or three highly qualified testers completely generate, select and describe cases, and check out 10 -15,100, 500+ "testers" not really delving into the very essence of the process.
With this text I will try to slightly bring yesterday's, today's and tomorrow's applicants to the position of the tester to understand, and what is “testing” after all. Next, I will answer some of the interview questions and substantiate my opinion, and also give some of the most frequent answers of applicants and explain why I consider them wrong.
Why did you decide to become a tester?
The most frequent answer is: "because it is simple and interesting (!)". That is, the candidate thinks that he will be paid for clicking on the mouse in VK ... Or they will give the software and say - break it ... Or he just did not prepare for this issue and has a very weak idea of ​​the profession.
The second answer is: “because I want to work in IT and testing is the easiest way” (read: IT specialists have high sn, but testing doesn’t need any knowledge or skills, but sn is also quite high!).
There were answers: “my mother / husband / wife made me go for an interview”.
The only correct answer is no, but these three and their derivatives are definitely wrong, because testing is difficult and monotonous, it requires certain skills for which there are no textbooks, and leads to a professional deformation of the worldview.
What would you like to hear? Perhaps something like: "because without testing it is impossible to identify the true state of the product being produced, and how it meets the expectations of the consumer."
What is testing? What is its essence as a process?
The most frequent answer (directly registered with S. Kaner and R. Savin) is “search for errors”. And for some reason nobody in the entire literature on testing indicates that this simplification is very rude, and in general, this answer is simply wrong!
Testing is a set of measures aimed at conducting checks on the conformity of the product being produced to the requirements imposed on it (direct and indirect).
Yes, indeed, during the inspections errors / incidents / observations are revealed, but this is only a by-product of the process. The main information is about the product's compliance with the requirements that apply to it .
What is a mistake?
Well, here, thank God, almost everyone answers: “incorrect work of the program ...”. But then chaos begins when you ask: “how do we know if the work is correct or not?”
The correct answer is on almost all testing resources known to me:
Error - the discrepancy between the product produced and the requirements, direct or indirect.
In order not to wander in contradictions / assumptions, etc., this is the only correct answer.
What is the purpose of testing?
Here people begin to repeat the answer to the second question with different variations. The most attentive applicants are trying to retell what I told them when answering the second question. And the answer is extremely simple:
The purpose of testing is to provide up-to-date information on the conformity of the product to the requirements
Everything. No more and no less. Well, of course, you can also say that the purpose of testing is to provide information on the number of errors in the product. Namely this is wrong. Why? This is just the same everyday case of many testers / PM / analysts: the customer’s call is “how is my product?”. “You know, there are 60 more bugs in it!” Is the answer of the tester / PM ... And then what? It's a lot? Few? Fine? You can, of course, tell in detail about the criticality of these bugs, their priorities, but this is not the answer to the customer's question, it is the issuance of raw raw information to fiberboard. Now the same case. “How is my product?”, Asks the customer. “35% of the requirements are fully implemented, another 5% with remarks and another 2% are now in implementation,” the PM / tester answers. Do you think this answer is clearer? And let these 5% include the already mentioned 60 bugs, comments ... The answer to the question is given as accurate as possible in this format. This is exactly the purpose of the test. And, accordingly, the process itself in essence should be reduced to the achievement of this goal.
What do you know about the software life cycle?
A lot has been said about life cycle software, and it strongly depends on the organization of the implementation process as a whole. Nevertheless, there is a certain “golden mean”, but even here they manage to fantasize wild things, reducing everything to three points, painting the scheme into three pages ... To everyone who conducted / passed the interview, and so clearly, what mistakes are made and how many options do correct answers. I will not stop in more detail, I will say only that there is a whole pool of candidates who have firmly stalled on this issue (approximately 7%).
What are the requirements?
Only 50% of applicants reach this question in an hour and a half ... Although I do not demand answers “letter to letter”, the main thing, as lawyers call it, is to preserve “spirit”.
The most frequent case: applicants begin to list the types of technical documentation that they know or have heard about ... I will definitely listen, leave and ask: “anything else?”. Few people remember the division into "functional" / "non-functional", and who remember, often cannot explain the difference.
But there is one category that is forgotten. In this article I have already mentioned several times about "... the requirements of direct and indirect ...". At the interview, I say this phrase five or six times. A very small percentage of applicants asks again and thereby excludes this question from the interview. And the full answer is: “Requirements are direct (i.e., formalized in technical documentation, specs, user-story and other formal artifacts) and indirect (i.e., resulting from direct, or unspoken standard for this product or based on experience and common sense of using this product or products like it). All requirements are also divided into functional (describing what functions the product should perform) and non-functional (requirements for the environment, maintainability, reliability and other characteristics of the product). Direct demands are always higher than indirect ones. ”
The most obvious and “simple” example: in the TK - “the button must be red” is a direct requirement, indirect results flow from it - it should not be blue, green, gray or black, etc. ... Of course, this is a strong simplification, but very revealing. But the main thing is that such an approach cuts off an overly formal attitude to testing and raises the bar for testing qualification as such, because for competent testing it is not enough to know only the TK and user-story, one must also study the applied area and the specificity of consumption of the product. Such testing is much more effective.
There is a little sin behind me: I deny the existence of negative checks because:
- they are hard to justify before the leadership,
- it's hard for them to get time
- they are almost impossible to justify economically in front of the customer in the preparation of estimates for testing.
Based on the concept of indirect requirements, this is not necessary, since all the checks become positive, but some of them are for compliance with direct requirements, and some for indirect compliance. And the qualification of a specialist is just revealed by an understanding of the indirect requirements for each specific product.
What types / types / classes / testing methods do you know, and how do they differ?
There is a lot of information about the classification of testing, there are also a lot of options for the correct answers. I ask this question in order to see whether the applicant prepared himself, at least in a small degree or not at all bothered. The fact is that the previous questions can be answered simply by reasoning and having a general idea of ​​the sphere as a whole. This question requires basic knowledge of terms. Perhaps I will consider it in other articles, because it is quite large and deserves a separate article.
Tell us about the test documentation: types, goals.
Test documentation is probably the biggest problem. On it are such battles in communities, firms, etc.! About her so much controversial information. About her published multi-volumes in different languages. There is such a mess in her head ... What kind of answers have you heard (yes, yes, including the TK and the design decision are also test documentation) ... Therefore, I will express my thoughts on this.
Test documentation is of two types: external and internal. And she and the other - a tool that makes life easier for the project team. No more and no less.
External documentation:
- Remark - a short note, a comment about a small inaccuracy in the implementation of the product.
- Bug-report - a description of the detected case of non-compliance of the product being produced with the requirements, which are put forward by the error or its manifestations. It must contain the following elements:
- The idea of ​​the test case that caused the error.
- Description of the initial state of the system to perform the case.
- The steps required to identify the error or its manifestation.
- Expected result, i.e., what was supposed to happen in accordance with the requirements.
- The actual result, i.e., what actually happened.
- The input data that was used during the playback of the case.
- Other information, without which to repeat the case will not work.
- Criticality and / or priority.
- Screen shot (screen).
- Version, build, resource and other data about the environment.
- Request for change (improvement) - description of implicit / non-critical indirect requirements that were not taken into account when planning / implementing the product, but non-compliance, which may cause rejection of the end user. And ways / recommendations for product modification to match them.
- Testing Report (test report) - a document that provides information on the compliance / non-compliance of the product with the requirements. It may also contain a description of some details of the test session held, for example, elapsed time, types of testing used, a list of verified cases, etc. In an ideal variant, the phrase “Test passed. The error is not reproduced / the functional is working correctly / Complies with the requirements "means that the product or its part fully complies with the requirements of direct and indirect (in software production).
Internal documentation:
- Test plan (test plan) - a formalized and integrated description of one test session in one or several areas of testing. Those. a list of areas of testing to be conducted during the testing session (and, in accordance with these areas, requirements). It may also contain the necessary information about the environment, methodology, and other conditions that are important for the exponential nature of this testing session. The direction of inspections can also be understood as more detailed test documentation (by referring to it): checklists, test suites, test scenarios, which must be relied upon during the testing session. The main purpose of the document is to describe the boundaries of the testing session, to stabilize the exponential character of this session.
- A test scenario is a sequence of actions on a product that are connected by a single limited business process of use, and corresponding to them checks of the correctness of the product behavior during these actions. It may contain information about the initial state of the product to run the script, input data and other information that are crucial for a successful and demonstrative scenario checks. A special feature is the linearity of actions and checks, i.e. the dependence of subsequent actions and checks on the success of previous ones. The purpose of the document is to stabilize the coverage of aspects of the product, necessary for the performance of a functional task, with exponential necessary and sufficient checks. In fact, with the successful completion of the entire test scenario, we can conclude that the product can perform one or another of the functions assigned to it.
- A test suite is a set of formalized test cases interconnected according to a common logical feature.
- Check-list (check list) - a list of formalized test cases in the form of a convenient for conducting checks. Test cases in the checklist should not be dependent on each other. It must necessarily contain information about: test ideas, sets of input data, expected results, a boolean mark about the passage / non-passage of the test case, a boolean mark about the coincidence / discrepancy of the actual and the expected result for each test. It may also contain steps for conducting an inspection, data on environmental features and other information necessary for conducting inspections. The goal is to ensure the stability of the coverage of the requirements with the checks necessary and sufficient for a conclusion on the compliance of the product with them. The peculiarity is that the checklists are arranged by those test cases that are indicative of a particular requirement.
- Test case (test case) - a formalized description of one indicative test for compliance with the requirements of direct or indirect. Must contain the following information:
- The idea of ​​verification.
- Description of the requirement to be verified or the part of the requirement to be verified.
- Used to test the test environment.
- The initial state of the product before testing.
- Steps to bring the product to a state to be checked.
- Input data to use when playing back steps.
- Expected Result.
- Other information required for the inspection.
The goal is to record the generated and selected exponential test in the form that allows the tester of any qualification to conduct it and be able to analyze the results obtained.
As you can see, each subsequent type of internal test documentation to a certain extent details the previous one. Each document has its own purpose and together they are a tool to facilitate the generation, selection and reproduction of test cases. In addition, well-structured, supported, readable, organized and accessible test documentation allows in the long term:
- Ensure the stability of the coverage of requirements with checks.
- Ensure that all inspections are indicative.
- Ensure the necessity and sufficiency of inspections.
- Save time on the testing stages, reducing them to testing and analysis and transfer of results.
- Reduce the input qualification level of the tester to conduct inspections.
- Increase the predictability of testing sessions in terms of time and resources.
- Increase the transparency of the testing process for other participants in the product manufacturing process.
- Provide a knowledge base about the product and its development history.
But keep in mind that there are some disadvantages:
- Coverage stability With tending to infinity with probability, if testing is carried out on documentation, then only those checks that are in this documentation will be carried out. The likelihood of missing an error (most often non-compliance with an indirect requirement, uncovered documentation) increases.
- Bad error localization by tester. Or a complete lack of localization. The actual result did not match the expected - an error. And what it really is: a mistake; error manifestation; an incident, an already described error, the tester will not check (in the overwhelming number of cases).
- The high required skill level of a specialist to create and maintain test documentation.
- Large time spent on creating and maintaining test documentation.
- Weakly predicted time of relevance of test documentation.
Lists of both pros and cons can continue, I indicated only those that lie on the surface. But understanding at least this list is extremely important for the current or future testing specialist. The question regarding test documentation is overcome by a very small percentage of applicants.
Naturally, the types of documentation in testing are much more, but without an understanding of the purpose and features of these documents, working in this profession is very difficult. And in order not to increase the size of the article at all, consider the last (for this article) question.
What are the stages of the testing process?
Most often they answer approximately this way: “preparation, testing, report ...” That's how it is, only any process consists of these steps. And the answer does not reflect the understanding of the applicant testing processes. More like cheating ... Therefore, I will allow myself the statement of my vision:
- initiation
- identification of direct and indirect requirements,
- test case generation
- selection of indicative test cases
- conducting inspections
- fixation of results
- analysis of results
- transfer of information about the compliance of the proven product requirements.
More information about these stages:
Initiation - an event that notifies the testing team of the need for a testing session, and also ensures that the product requirements for testing are met.
For software production requirements include:
- the necessary test environment is available,
- build / resource / test item is available,
- the code, database, other components of the test object are “frozen”, i.e. they do not change during the entire testing session,
- modification of requirements (at least direct) "frozen",
- know the direction of testing,
- known dates for the testing session.
There are other conditions, but they are less significant and highly dependent on the specific process in the company.
Identifying requirements is perhaps one of the main steps in the testing process. Unknown requirements - no testing. It is necessary to collect all available information about the subject of testing, use cases, etc. The first source - technical documentation and user-story - these are direct requirements. The quality of indirect requirements largely depends on the good faith, responsibility, qualifications of the tester and the entire project team.
The generation of test cases is the identification of all possible uses of the product, its characteristics and features during operation. This means: of all cases that a tester can “invent” on the basis of direct and indirect requirements known to him. This stage requires highly qualified testing specialist.
Selection of test cases - the selection of the most significant, significant and reproducible test cases. This stage determines how useful the testing will be, effective and analyzed. For example, in the “simple” example with a red button, it is clear that the number of indirect requirements tends to infinity, and checking them all is absurd, but such cases should be generated at least in the head of the examiner. And in order for them not to enter the checks, it is necessary to perform the appropriate selection and only check whether the button is really red.
The example is primitive, but after voicing it, the applicants no longer try to pour radium on the test task J into the glass (those who took part in the interview for the position of tester know this simple task to generate and select test cases).
Carrying out checks - everything is clear. Either according to the documentation or ad hoc (intuitive, free search, without documentation). In any case, this is carried out according to the list of selected checks. For some reason, most of this item calls testing. And in the head of the philistine, unfamiliar with the profession, only this one point is contained in J.
Recording results - the creation of internal and external test documentation in a formalized form or in the form of records, etc. At this stage, the test report, even if it is created, is not considered complete.
Analysis of the results - the decision on the compliance of the proven product requirements. Formalization of this decision and its justification in the form of a test report. This also includes procedures for assessing the coverage of requirements with checks, time-sharing, etc. Thus, not only the results are analyzed, but also the testing session itself.
Transmission of product compliance information. Formally: the transfer of external test documentation to interested parties, often the initiator of a testing session. In general: in addition to the documentation, information is provided on the risks that have been identified in the product, requirements, processes, recommendations are given on how to work out these risks, etc. But this is already QA J!
Instead of an afterword
The attentive reader noted that questions are gaining complexity from the beginning to the end of the conversation. At the same time, it is worth noting that all these questions should not make it difficult for testing specialists, that is, those who didn’t "just click with the mouse, test", but try to understand the essence of what he is doing, those who didn’t stop at one book or one resource, but thought out and justified his actions ... at least for himself.
This concludes this article. If a dear reader finds a subject for discussion in my statements, I will be glad to participate in it. If there are questions, they will be topics for future articles. Always happy to constructive criticism. In any case, write to me. If an interest is shown in the article, I will continue to analyze the interviews, and, perhaps, I will try to cover other aspects of the profession, about which almost everyone has heard, but few of those who know it from the inside.
Source: https://habr.com/ru/post/254209/
All Articles