Examination of the task on the example of the task of checking the analyzer RBNF declarations
Dear colleagues, what is the tester's professionalism when he approaches the next task? In the ability to timely make up the missing knowledge, to plunge "to the bottom" in its context; to identify in her statement of uncertainty / incorrectness, to clearly formulate the "misunderstandings" and eliminate all this by agreement.
In this article I will give a detailed illustration of how to conduct an examination of the task for testing.
For what? Honestly, I do not know a lot of theory for the development of the above mentioned professional qualities. For me, testing is an art that I want to touch here, as well as learn from you with those who will then make good comments or additions.
')
For whom? For those who learn to evaluate the content of the work according to the tasks; who needs to decide for themselves: stop doing the tasks without thinking, and then look at the results with sadness. About the side, because of the "mysterious" demands of which a lot of time and energy is wasted, I don’t even speak, but even more so the "dark" side, which is used to catching fish in muddy water, especially.
Consider the task. The problem statement (hereinafter referred to as the PP ) stated:
Our goal is to perform an examination of this assignment and do it delicately in relation to the PZ.
So, we have to perform a search and study (or refresh in memory) of the relevant specifications:
1. ISO / IEC 14977: 1996 (E) - Meta-Language Specification Extended Backus-Naur Form
2. The Java Language Specification, Java SE 7 Edition - JAVA v1.7 Documentation
REMARK_1: agree with the PZ - are these materials relevant for the assignment, is their list sufficient for the solution ?!
In the process of analyzing the specifications and the provided condition of the problem, it becomes necessary to clarify the problem statement, correct it to get rid of ambiguities.
Let us deal with this (see below), and then ask PZ to consider an adapted version of the grammar and pay attention to our comments, questions and comments.
(*
NOTE_2: in the original rule, the line immediately draws attention to itself:
"procedure <return type> <name> ([<param type> <param name> {, <param type> <param name>}]) ;."
What is critical here:
1.1 in the fragment "procedure <return type>" the equal sign between the procedure identifier and its definition is missing, i.e. the expression to his right;
1.2 in the fragment "<param name> {, <param type>" the comma can be interpreted as a concatenation character;
1.3 in the fragment "<param name>}]) ;." each of the last two characters can be interpreted as a terminating expression;
1.4 use of angle brackets "<", ">" is an outdated construction, not in the style of extended syntax.
*)
procedure = return type, blank parenting, args, closing parenthesis, semicolon;
(*
COMMENT_1: here, in the proposed version, ambiguities are eliminated according to our understanding of the RBNF and the presentation as, for example, in the java program the method declaration looks, but if we make a mistake, ask the PP to correct us!
QUESTION_1: within the framework of the java specification - if the body method is not defined when declaring a class method, then the method must be specified as abstract. We ask PZ: in the given initial grammar this definition is not implied, it is not critical, i.e. Consider an interface method declaration?
*)
blank = "" | (* code analog *) "\ u0020";
(* NOTE_3: we suppose that in the 1st line of the grammar in the appropriate places only one space should be used and the translation of the line is not allowed, etc. Otherwise, we ask PZ to correct us here and in Remarks 4 and 5 below! *)
args = [arg, {comma, blank, arg}];
(* NOTE_4: the signature can be empty or contain from one to an infinite number of pairs. Type Parameter parameter Name. The delimiters between the pairs are 'comma and one space' *)
arg = param type, blank, param name;
(* NOTE_5: between the Parameter type and the Parameter name the single space delimiter *)
comma = "," | (* code analog *) "\ u002c";
semicolon = ";" | (* code analog *) "\ u003b";
opening parenthesis = "(" | (* code analog *) "\ u0028";
closing parenthesis = ")" | (* code analog *) "\ u0029";
return type = ("v" | "\ u0076"), ("o" | "\ u006f"), ("i" | "\ u0069"), ("d" | "\ u0064") | ("i" | "\ u0069"), ("n" | "\ u006e"), ("t" | "\ u0074"); (* char or code mix for "void" | "int" *)
param type = ("i" | "\ u0069"), ("n" | "\ u006e"), ("t" | "\ u0074") | ("l" | "\ u006c"), ("o" | "\ u006f"), ("n" | "\ u006e"), ("g" | "\ u0067"); (* char or code mix for "int" | "long" *)
(*
CONDITIONED: "identifiers that meet java requirements"
Further, in terms of the RBNF, we will reveal what should be meant by this.
*)
param name = java identifier; (* parameter name *)
(*
We try to exclude doubts that arise from the questions to the PZ.
QUESTION_2: if in the test case the method has arguments, then should the analyzer:
2.1 consider a valid declaration in which there are identical parameter names?
2.2 recognize arguments as different if their names consist of the same character set, but do they differ in case?
2.2 define as a correct declaration, in which the same names for the method and its parameter? Does this result depend on a different register?
NOTE_1: these questions here and below are obviously necessary in order to form positive and negative test suites.
IN CONDITION: "the implementation of this task does not require special knowledge, not related to programming"
Again, questions to the PP exclude uncertainties in the task.
QUESTION_3: Do we understand correctly that in the task in <param name> the possibility of constructions should not be considered:
3.1 ellipsis before the name of the last argument in the signature, (i.e. the analyzer does not recognize the method with a variable number of parameters as valid)?
3.2 assigning values after specifying the parameter name, (i.e., the method with the default values of the parameters also should not be recognized)?
*)
method name = java identifier; (* method name *)
(* The grammatical rules for the compilation of parameter names apply to method names *)
java identifier = definition - (keyword | boolean literal | null literal);
(* We propose to the attention of the PP a specific rule for the formation of an identifier based on the analysis of java 1.7 documentation *)
definition = java letter, {java letter | java digit};
(*
COMMENT_2: the scope of the definition is such that there is no need to exclude the following two sets defined by the rules:
spec simbol = "[" | "]" | "(" | ")" | "{" | "}" | "." | "," | "~" | "!" | "?" | "@" | "#" | "%" | "^" | "&" | "*" | "+" | "-" | "=" | ":" | ";" | "\" "|" "" | "` "|" | "|" \ "|" / "|" <"|"> "(* and their code analog / is not shown here / *);
escape sequence = "\ b" | "\ t" | "\ n" | "\ f" | "\ r" | "\" "|" \ '"|" \\ "(* and their code analog / is not given here / *);
but they (and not only them) must be kept in mind for the future to build negative sets.
*)
java letter = latin letter | cyrillic letter | spec letter;
(*
JAVA v1.7 fully supports UNICODE v6.0 in the names of a parameter or method, which includes almost all modern scripts, including: Arabic; Armenian; Bengali Burmese; verb; Greek Georgian; Devanagari; Jewish; Cyrillic; Chinese Coptic; Khmer; Latin Tamil; Korean (Hangul); Cherokee; Ethiopian; Japanese (includes, in addition to Chinese characters, the syllable alphabet) and much more.
QUESTION_4: we will propose a PZ definition with Latin and Cyrillic (Russian alphabet) - if this is not enough, we ask:
4.1 indicate which specific international dictionaries need to be added for verification in order to satisfy the relevant requirement for the analyzer to work?
4.2 to clarify whether there are restrictions in the requirements, respectively, for example, each operating system, etc. from the variants of the environment where the analyzer program should be executed?
*)
latin letter = "A" | ... | "Z" | (* code analog *) "\ u0041" | ... | "\ u005a"
| "a" | ... | "z" | (* code analog *) "\ u0061" | ... | "\ u007a";
cyrillic letter = "A" | ... | "I" | "a" | ... | "i" | (* code analog *) "\ u0410" | ... | "\ u044F"
| "Yo" | "" | (* code analog *) "\ u0401" | "\ u451";
spec letter = "$" | (* code analog *) "\ u0024"
| "_" | (* code analog *) "\ u005f";
java digit = "0" | ... | "9" | (* code analog *) "\ u0030" | ... | "\ u0039";
keyword =
"abstract" | "continue" | "for" | "new" | "switch"
| "assert" | "default" | "if" | "package" | "synchronized"
| "boolean" | "do" | "private" | "this"
| "break" | "double" | "implements" | "protected" | "throw"
| "byte" | "else" | "import" | "public" | "throws"
| "case" | "enum" | "instanceof" | "return" | "transient"
| "catch" | "extends" | "int" | "short" | "try"
| "char" | "final" | "interface" | "static" | "void"
| "class" | "finally" | "long" | "strictfp" | "volatile"
| "const" | "float" | "native" | "super" | "while"
| (* reserved *) "goto"; (* here for each word is not given, but the corresponding construction is char or code mix *)
(*
QUESTION_5: clarify with the PP - is it correct to assume that the deprecated "goto" should be considered as an invalid name for the parameter and method?
*)
boolean literal = "true" | "false"; (* is not given here, but the corresponding char or code mix * is implied)
null literal = ("n" | "\ u006e"), ("u" | "\ u0075"), ("l" | "\ u006c"), ("l" | "\ u006c"); (* char or code mix for "null" *)
With the analysis and correction of grammar finished. It remains to clarify a few nuances.
CONDITIONED: "identifiers that meet java requirements"
NOTE2: java identifiers can theoretically have unlimited length. But in practice, obviously, the restriction occurs in accordance with the finite hardware / system resources of the test complex (test bench). For example, if test cases are planned to be entered via a web interface, the specifics of the browser and server must be taken into account, imposing a restriction in accordance with the allowed sizes of GET or POST requests, by means of which the test version will be sent to the analyzer.
Corresponding information is required to work with boundary conditions, we request it.
QUESTION_6: We turn to the PP (or, through it, to experts who know the characteristics of the environment where the analyzer will be tested):
6.1 to give an estimate of the MAX'mal length of the text of the correct test version, when submitted to the input of the lexical analyzer, the input set will not be distorted or (if the declaration is transmitted without distortion), the analyzer program does not fail.
6.2 If test suites must be prepared in a file located on the server side, what should be its MAX 'minimum allowable size (as part of the issue of free disk space required for normal operation of the analyzer program).
We ask the PZ - can be given specific information on paragraphs 6.1, 6.2? Otherwise, we either do not fulfill the check for max. The boundary condition or we will have to describe the method of identifying the upper limit of the length of the text of a separate declaration (with a successful check) empirically in the process of checking. In the latter case, we will require the PZ to formulate this as an additional subtask within the framework of the analyzer’s study!
NOTE_3: PZ pays attention to the fact that the task does not require special knowledge related to testing the white box.
QUESTION_7: Knowing how many varieties of testing exist outside the white box and what knowledge is needed there, we urgently clarify whether we understood correctly that the task required providing only test sets of declarations and no need to prepare a testing methodology, for example, describing the organization of parallel flow to the input of the analyzer, to monitor memory leaks, measure performance and other characteristics related to load testing or safety assessment, etc.? Otherwise, we will require the PZ to formulate this as an additional subtask within the framework of the research program of the analyzer (and we warn that this will require separate coordination)!
Thus, we have carried out an examination of the assignment, including proposals for its correction.
It was important not to leave "holes" in the future coverage of test sets (and this is already a matter of technology, beyond the framework of this article, although not everything is so simple here ... scientific papers are written on this subject).
It remains to appeal to the PP with a request to comment on 7 questions and 5 comments (see above) to clarify: the requirements of the problem; necessary documentation; specifics imposed by the testing environment and the expected types of testing, i.e. coordination of its understandable and unambiguously interpreted correct conditions by both parties.
PS It depends on how clever and in the case were the questions of the tester, whether they want to transfer the corresponding task to his hands on the execution.
It depends on how complete and competent the answer of the PP is, to be or not to be a qualitative solution of the problem.
In this article I will give a detailed illustration of how to conduct an examination of the task for testing.
For what? Honestly, I do not know a lot of theory for the development of the above mentioned professional qualities. For me, testing is an art that I want to touch here, as well as learn from you with those who will then make good comments or additions.
')
For whom? For those who learn to evaluate the content of the work according to the tasks; who needs to decide for themselves: stop doing the tasks without thinking, and then look at the results with sadness. About the side, because of the "mysterious" demands of which a lot of time and energy is wasted, I don’t even speak, but even more so the "dark" side, which is used to catching fish in muddy water, especially.
Consider the task. The problem statement (hereinafter referred to as the PP ) stated:
Prepare test kits for checking a lexical analyzer that parses declarations of the following type (in the proposed RBNF notation):
procedure <return type> <name> ([<param type> <param name> {, <param type> <param name>}]) ;.
<return type> = void | int.
<param type> = int | long.
<name>, <param name>
- identifiers that meet the requirements of java.
PZ draws attention to the fact that to perform the task does not require special knowledge related to programming or testing a white box.
Our goal is to perform an examination of this assignment and do it delicately in relation to the PZ.
So, we have to perform a search and study (or refresh in memory) of the relevant specifications:
1. ISO / IEC 14977: 1996 (E) - Meta-Language Specification Extended Backus-Naur Form
2. The Java Language Specification, Java SE 7 Edition - JAVA v1.7 Documentation
REMARK_1: agree with the PZ - are these materials relevant for the assignment, is their list sufficient for the solution ?!
In the process of analyzing the specifications and the provided condition of the problem, it becomes necessary to clarify the problem statement, correct it to get rid of ambiguities.
Let us deal with this (see below), and then ask PZ to consider an adapted version of the grammar and pay attention to our comments, questions and comments.
(*
NOTE_2: in the original rule, the line immediately draws attention to itself:
"procedure <return type> <name> ([<param type> <param name> {, <param type> <param name>}]) ;."
What is critical here:
1.1 in the fragment "procedure <return type>" the equal sign between the procedure identifier and its definition is missing, i.e. the expression to his right;
1.2 in the fragment "<param name> {, <param type>" the comma can be interpreted as a concatenation character;
1.3 in the fragment "<param name>}]) ;." each of the last two characters can be interpreted as a terminating expression;
1.4 use of angle brackets "<", ">" is an outdated construction, not in the style of extended syntax.
*)
procedure = return type, blank parenting, args, closing parenthesis, semicolon;
(*
COMMENT_1: here, in the proposed version, ambiguities are eliminated according to our understanding of the RBNF and the presentation as, for example, in the java program the method declaration looks, but if we make a mistake, ask the PP to correct us!
QUESTION_1: within the framework of the java specification - if the body method is not defined when declaring a class method, then the method must be specified as abstract. We ask PZ: in the given initial grammar this definition is not implied, it is not critical, i.e. Consider an interface method declaration?
*)
blank = "" | (* code analog *) "\ u0020";
(* NOTE_3: we suppose that in the 1st line of the grammar in the appropriate places only one space should be used and the translation of the line is not allowed, etc. Otherwise, we ask PZ to correct us here and in Remarks 4 and 5 below! *)
args = [arg, {comma, blank, arg}];
(* NOTE_4: the signature can be empty or contain from one to an infinite number of pairs. Type Parameter parameter Name. The delimiters between the pairs are 'comma and one space' *)
arg = param type, blank, param name;
(* NOTE_5: between the Parameter type and the Parameter name the single space delimiter *)
comma = "," | (* code analog *) "\ u002c";
semicolon = ";" | (* code analog *) "\ u003b";
opening parenthesis = "(" | (* code analog *) "\ u0028";
closing parenthesis = ")" | (* code analog *) "\ u0029";
return type = ("v" | "\ u0076"), ("o" | "\ u006f"), ("i" | "\ u0069"), ("d" | "\ u0064") | ("i" | "\ u0069"), ("n" | "\ u006e"), ("t" | "\ u0074"); (* char or code mix for "void" | "int" *)
param type = ("i" | "\ u0069"), ("n" | "\ u006e"), ("t" | "\ u0074") | ("l" | "\ u006c"), ("o" | "\ u006f"), ("n" | "\ u006e"), ("g" | "\ u0067"); (* char or code mix for "int" | "long" *)
(*
CONDITIONED: "identifiers that meet java requirements"
Further, in terms of the RBNF, we will reveal what should be meant by this.
*)
param name = java identifier; (* parameter name *)
(*
We try to exclude doubts that arise from the questions to the PZ.
QUESTION_2: if in the test case the method has arguments, then should the analyzer:
2.1 consider a valid declaration in which there are identical parameter names?
2.2 recognize arguments as different if their names consist of the same character set, but do they differ in case?
2.2 define as a correct declaration, in which the same names for the method and its parameter? Does this result depend on a different register?
NOTE_1: these questions here and below are obviously necessary in order to form positive and negative test suites.
IN CONDITION: "the implementation of this task does not require special knowledge, not related to programming"
Again, questions to the PP exclude uncertainties in the task.
QUESTION_3: Do we understand correctly that in the task in <param name> the possibility of constructions should not be considered:
3.1 ellipsis before the name of the last argument in the signature, (i.e. the analyzer does not recognize the method with a variable number of parameters as valid)?
3.2 assigning values after specifying the parameter name, (i.e., the method with the default values of the parameters also should not be recognized)?
*)
method name = java identifier; (* method name *)
(* The grammatical rules for the compilation of parameter names apply to method names *)
java identifier = definition - (keyword | boolean literal | null literal);
(* We propose to the attention of the PP a specific rule for the formation of an identifier based on the analysis of java 1.7 documentation *)
definition = java letter, {java letter | java digit};
(*
COMMENT_2: the scope of the definition is such that there is no need to exclude the following two sets defined by the rules:
spec simbol = "[" | "]" | "(" | ")" | "{" | "}" | "." | "," | "~" | "!" | "?" | "@" | "#" | "%" | "^" | "&" | "*" | "+" | "-" | "=" | ":" | ";" | "\" "|" "" | "` "|" | "|" \ "|" / "|" <"|"> "(* and their code analog / is not shown here / *);
escape sequence = "\ b" | "\ t" | "\ n" | "\ f" | "\ r" | "\" "|" \ '"|" \\ "(* and their code analog / is not given here / *);
but they (and not only them) must be kept in mind for the future to build negative sets.
*)
java letter = latin letter | cyrillic letter | spec letter;
(*
JAVA v1.7 fully supports UNICODE v6.0 in the names of a parameter or method, which includes almost all modern scripts, including: Arabic; Armenian; Bengali Burmese; verb; Greek Georgian; Devanagari; Jewish; Cyrillic; Chinese Coptic; Khmer; Latin Tamil; Korean (Hangul); Cherokee; Ethiopian; Japanese (includes, in addition to Chinese characters, the syllable alphabet) and much more.
QUESTION_4: we will propose a PZ definition with Latin and Cyrillic (Russian alphabet) - if this is not enough, we ask:
4.1 indicate which specific international dictionaries need to be added for verification in order to satisfy the relevant requirement for the analyzer to work?
4.2 to clarify whether there are restrictions in the requirements, respectively, for example, each operating system, etc. from the variants of the environment where the analyzer program should be executed?
*)
latin letter = "A" | ... | "Z" | (* code analog *) "\ u0041" | ... | "\ u005a"
| "a" | ... | "z" | (* code analog *) "\ u0061" | ... | "\ u007a";
cyrillic letter = "A" | ... | "I" | "a" | ... | "i" | (* code analog *) "\ u0410" | ... | "\ u044F"
| "Yo" | "" | (* code analog *) "\ u0401" | "\ u451";
spec letter = "$" | (* code analog *) "\ u0024"
| "_" | (* code analog *) "\ u005f";
java digit = "0" | ... | "9" | (* code analog *) "\ u0030" | ... | "\ u0039";
keyword =
"abstract" | "continue" | "for" | "new" | "switch"
| "assert" | "default" | "if" | "package" | "synchronized"
| "boolean" | "do" | "private" | "this"
| "break" | "double" | "implements" | "protected" | "throw"
| "byte" | "else" | "import" | "public" | "throws"
| "case" | "enum" | "instanceof" | "return" | "transient"
| "catch" | "extends" | "int" | "short" | "try"
| "char" | "final" | "interface" | "static" | "void"
| "class" | "finally" | "long" | "strictfp" | "volatile"
| "const" | "float" | "native" | "super" | "while"
| (* reserved *) "goto"; (* here for each word is not given, but the corresponding construction is char or code mix *)
(*
QUESTION_5: clarify with the PP - is it correct to assume that the deprecated "goto" should be considered as an invalid name for the parameter and method?
*)
boolean literal = "true" | "false"; (* is not given here, but the corresponding char or code mix * is implied)
null literal = ("n" | "\ u006e"), ("u" | "\ u0075"), ("l" | "\ u006c"), ("l" | "\ u006c"); (* char or code mix for "null" *)
With the analysis and correction of grammar finished. It remains to clarify a few nuances.
CONDITIONED: "identifiers that meet java requirements"
NOTE2: java identifiers can theoretically have unlimited length. But in practice, obviously, the restriction occurs in accordance with the finite hardware / system resources of the test complex (test bench). For example, if test cases are planned to be entered via a web interface, the specifics of the browser and server must be taken into account, imposing a restriction in accordance with the allowed sizes of GET or POST requests, by means of which the test version will be sent to the analyzer.
Corresponding information is required to work with boundary conditions, we request it.
QUESTION_6: We turn to the PP (or, through it, to experts who know the characteristics of the environment where the analyzer will be tested):
6.1 to give an estimate of the MAX'mal length of the text of the correct test version, when submitted to the input of the lexical analyzer, the input set will not be distorted or (if the declaration is transmitted without distortion), the analyzer program does not fail.
6.2 If test suites must be prepared in a file located on the server side, what should be its MAX 'minimum allowable size (as part of the issue of free disk space required for normal operation of the analyzer program).
We ask the PZ - can be given specific information on paragraphs 6.1, 6.2? Otherwise, we either do not fulfill the check for max. The boundary condition or we will have to describe the method of identifying the upper limit of the length of the text of a separate declaration (with a successful check) empirically in the process of checking. In the latter case, we will require the PZ to formulate this as an additional subtask within the framework of the analyzer’s study!
NOTE_3: PZ pays attention to the fact that the task does not require special knowledge related to testing the white box.
QUESTION_7: Knowing how many varieties of testing exist outside the white box and what knowledge is needed there, we urgently clarify whether we understood correctly that the task required providing only test sets of declarations and no need to prepare a testing methodology, for example, describing the organization of parallel flow to the input of the analyzer, to monitor memory leaks, measure performance and other characteristics related to load testing or safety assessment, etc.? Otherwise, we will require the PZ to formulate this as an additional subtask within the framework of the research program of the analyzer (and we warn that this will require separate coordination)!
Thus, we have carried out an examination of the assignment, including proposals for its correction.
It was important not to leave "holes" in the future coverage of test sets (and this is already a matter of technology, beyond the framework of this article, although not everything is so simple here ... scientific papers are written on this subject).
It remains to appeal to the PP with a request to comment on 7 questions and 5 comments (see above) to clarify: the requirements of the problem; necessary documentation; specifics imposed by the testing environment and the expected types of testing, i.e. coordination of its understandable and unambiguously interpreted correct conditions by both parties.
PS It depends on how clever and in the case were the questions of the tester, whether they want to transfer the corresponding task to his hands on the execution.
It depends on how complete and competent the answer of the PP is, to be or not to be a qualitative solution of the problem.
Source: https://habr.com/ru/post/243513/
All Articles