Statistics identify software vulnerabilities in the framework of certification tests
Software vulnerability analysis is currently a mandatory activity carried out by experts from testing laboratories of the national systems for certification of information protection means (GIS). This type of work is performed both with certification for compliance with the requirements of protection profiles, which explicitly include the requirements of the AVA_VAN trust analysis "Vulnerability Analysis" (standards under the "Common Criteria"), and during tests for compliance with the requirements of technical conditions or classical guidelines State Technical Commission of Russia.
This study presents statistics on the detection of vulnerabilities in software that has been subject to certification testing in the Testing Laboratory of NPO Echelon in the period 2016-2017.
The vulnerability analysis methodology used by testing laboratories (IL) in the framework of certification tests is based on the General Assessment Methodology (international standard ISO / IEC 18045) and generally consists of the following steps.
Stage 1. Identification of known (confirmed) vulnerabilities of the certification object. At this stage, IL experts search for known (confirmed) vulnerabilities in publicly available information sources, for example: in the FSTEC of Russia Information Security Threat Data Bank, CVE resources, NVD, software developer (software) resource. If information about vulnerabilities is detected, then the tests are suspended until the software updates “closing” known vulnerabilities are transmitted to the IL.
')
Stage 2. Identification of previously unpublished vulnerabilities of the certification object. At this stage, IL experts generally perform the following steps:
As part of this study, when performing step 1, the specialists of the ILO Echelon used the following methods to form a list of potential vulnerabilities:
When identifying a vulnerability in a certification object, detailed technical information is provided to the software developer in order to obtain confirmation of the conclusions made by IL experts and formulate measures to neutralize the identified vulnerability (by updating the software or determining operating instructions). It is assumed that vulnerabilities identified during certification tests / inspection controls should undergo a procedure for disclosing information security threats from the FSTEC of Russia through the Data Bank.
Studies were conducted in the period 2016 - 2017. as part of testing 76 products under the certification system in all certification systems (certification tests, inspection controls) at IL Echelon.
The distribution of the investigated products by type is presented in Figure 1.
Fig.1. The distribution of the investigated products by type
(GIS from unauthorized access control - a means of protecting against unauthorized access; software with GIS - application software with integrated information protection; ME - firewall; SAVZ - anti-virus protection; DBMS - database management system; OS - operating system; IDS - system intrusion detection)
Product developers were both domestic and foreign organizations.
Depending on the certification criteria determined by the potential scope of the certified product, IL experts were given or did not have access to the source texts of the certification object (Fig. 2).
Fig.2. The distribution of the investigated products, depending on access to the source code
Research results
As a result of the research, specialists of IL EE “Echelon” identified 81 vulnerabilities (vulnerabilities were identified in 26 products from 76 studied). For all identified software vulnerabilities, confirmation of their relevance from the software developer was received, and software developers took measures to eliminate the identified vulnerabilities.
In fig. 3 shows the distribution of identified vulnerabilities by degree of criticality (assessment was performed using the CVSS version 3.0 method).
Fig.3. Distribution of identified vulnerabilities depending on the degree of criticality
The most popular type of vulnerable software has become application software with built-in information protection tools (Fig. 4).
Fig.4. Distribution of identified vulnerabilities depending on the type
investigated software
The main types of vectors of successful attacks, which were developed by IL experts to confirm the relevance of the vulnerability, were (Fig. 5):
Fig.5. Distribution of identified vulnerabilities depending on the type of attack vector
The “Other” category included such types of attack vectors as: remote execution of operating system commands by sending data in HTTP requests (CAPEC-76), XML injection (CAPEC-250), session fixation (CAPEC-61), going beyond the limits of the designated directory (CAPEC-126), attacks such as "Reparse-Point", "RegSafe / RegRestore".
The main types of software flaws that caused vulnerabilities are (Fig. 6):
Fig.6. Distribution of identified vulnerabilities depending on software error (shortage)
The types of software errors such as: use of authentication data specified in the program code (CWE-798), buffer overflow (CWE-120), errors leading to session fixation (CWE-384), incorrect use of data, received from an untrusted source, to generate OS commands (CWE-22), etc.
Speaking about the methods of forming a list of potential vulnerabilities, it should be noted that most of the vulnerabilities were discovered due to assumptions made on the basis of studying the documentation on the object of certification and data on vulnerabilities in products similar to the object of certification (Fig. 7).
Fig.7. Distribution of identified vulnerabilities depending on the method of generating a list of potential vulnerabilities
The distribution of the identified vulnerabilities depending on the software error (shortage) and the type of software investigated is shown below:
The distribution of identified vulnerabilities depending on the degree of criticality and the type of software investigated:
This study presents statistics on the detection of vulnerabilities in software that has been subject to certification testing in the Testing Laboratory of NPO Echelon in the period 2016-2017.
Research methodology
The vulnerability analysis methodology used by testing laboratories (IL) in the framework of certification tests is based on the General Assessment Methodology (international standard ISO / IEC 18045) and generally consists of the following steps.
Stage 1. Identification of known (confirmed) vulnerabilities of the certification object. At this stage, IL experts search for known (confirmed) vulnerabilities in publicly available information sources, for example: in the FSTEC of Russia Information Security Threat Data Bank, CVE resources, NVD, software developer (software) resource. If information about vulnerabilities is detected, then the tests are suspended until the software updates “closing” known vulnerabilities are transmitted to the IL.
')
Stage 2. Identification of previously unpublished vulnerabilities of the certification object. At this stage, IL experts generally perform the following steps:
- Step 1: formation of a list of potential vulnerabilities of a certification object based on a study of various types of data sources (project documentation for the certification object, source text, information from open sources);
- Step 2: creating test scenarios (attacks) for each identified potential vulnerability;
- Step 3: Test execution to determine if the assumption is correct.
As part of this study, when performing step 1, the specialists of the ILO Echelon used the following methods to form a list of potential vulnerabilities:
- expert documentary method that provides for the formation of a list of potential vulnerabilities based on information about known vulnerabilities in products of similar functionality, in other versions (for example, older) of the certification object, in borrowed components, typical vulnerabilities characteristic of the technologies used in the development of the object certification;
- Static analysis of source texts of software (in the case of providing access to source texts in the framework of certification tests);
- fuzz testing
When identifying a vulnerability in a certification object, detailed technical information is provided to the software developer in order to obtain confirmation of the conclusions made by IL experts and formulate measures to neutralize the identified vulnerability (by updating the software or determining operating instructions). It is assumed that vulnerabilities identified during certification tests / inspection controls should undergo a procedure for disclosing information security threats from the FSTEC of Russia through the Data Bank.
Objects of study
Studies were conducted in the period 2016 - 2017. as part of testing 76 products under the certification system in all certification systems (certification tests, inspection controls) at IL Echelon.
The distribution of the investigated products by type is presented in Figure 1.
Fig.1. The distribution of the investigated products by type
(GIS from unauthorized access control - a means of protecting against unauthorized access; software with GIS - application software with integrated information protection; ME - firewall; SAVZ - anti-virus protection; DBMS - database management system; OS - operating system; IDS - system intrusion detection)
Product developers were both domestic and foreign organizations.
Depending on the certification criteria determined by the potential scope of the certified product, IL experts were given or did not have access to the source texts of the certification object (Fig. 2).
Fig.2. The distribution of the investigated products, depending on access to the source code
Research results
As a result of the research, specialists of IL EE “Echelon” identified 81 vulnerabilities (vulnerabilities were identified in 26 products from 76 studied). For all identified software vulnerabilities, confirmation of their relevance from the software developer was received, and software developers took measures to eliminate the identified vulnerabilities.
In fig. 3 shows the distribution of identified vulnerabilities by degree of criticality (assessment was performed using the CVSS version 3.0 method).
Fig.3. Distribution of identified vulnerabilities depending on the degree of criticality
The most popular type of vulnerable software has become application software with built-in information protection tools (Fig. 4).
Fig.4. Distribution of identified vulnerabilities depending on the type
investigated software
The main types of vectors of successful attacks, which were developed by IL experts to confirm the relevance of the vulnerability, were (Fig. 5):
- cross-site scripting (CAPEC-63);
- cross-site request forgery (CAPEC-62);
- privilege escalation associated with security feature bypass (CAPEC-233);
- denial of service attacks (CAPEC-2);
- disclosure of critical software information in error messages (CAPEC-54);
- SQL injection (CAPEC-66).
Fig.5. Distribution of identified vulnerabilities depending on the type of attack vector
The “Other” category included such types of attack vectors as: remote execution of operating system commands by sending data in HTTP requests (CAPEC-76), XML injection (CAPEC-250), session fixation (CAPEC-61), going beyond the limits of the designated directory (CAPEC-126), attacks such as "Reparse-Point", "RegSafe / RegRestore".
The main types of software flaws that caused vulnerabilities are (Fig. 6):
- incorrect use of data received from an untrusted source to generate an HTML page (CWE-79);
- using authentication data (cookie data) to authorize the request (CWE-352);
- incorrect use of data obtained from an untrusted source when performing security functions (CWE-807);
- lack of authorization when performing critical operations (CWE-862);
- incorrect use of data received from an untrusted source to generate a query to a DBMS (CWE-89);
- Incorrect error message generation (CWE-209).
Fig.6. Distribution of identified vulnerabilities depending on software error (shortage)
The types of software errors such as: use of authentication data specified in the program code (CWE-798), buffer overflow (CWE-120), errors leading to session fixation (CWE-384), incorrect use of data, received from an untrusted source, to generate OS commands (CWE-22), etc.
Speaking about the methods of forming a list of potential vulnerabilities, it should be noted that most of the vulnerabilities were discovered due to assumptions made on the basis of studying the documentation on the object of certification and data on vulnerabilities in products similar to the object of certification (Fig. 7).
Fig.7. Distribution of identified vulnerabilities depending on the method of generating a list of potential vulnerabilities
The distribution of the identified vulnerabilities depending on the software error (shortage) and the type of software investigated is shown below:
The distribution of identified vulnerabilities depending on the degree of criticality and the type of software investigated:
findings
- It is necessary to recognize that the share of vulnerabilities found in domestic software is somewhat higher than the share of vulnerabilities found in foreign software, even against the background of a significant difference between the volumes of products of domestic and foreign production studied. It seems that this is due to significant differences in the maturity levels of the life cycle processes of developing secure software implemented in foreign and domestic software developers. We hope that this situation will change with the introduction of the standard for safe development management in the country.
It should be noted that when conducting research for foreign-made software, in most cases, developers did not provide access to the source code of the objects of study, which made it impossible in principle to generate a list of potential vulnerabilities based on a study of the source texts of the program. - Most of the vulnerabilities identified in the framework of the research could be detected by software developers in the early stages of software development using software architecture analysis from the point of view of implementing information security threats and static software source code analysis.
- In order to reduce the number of vulnerabilities, software developers are encouraged to embed basic activities in the life cycle processes aimed at developing secure software (GOST R 56939) - analyzing the software architecture from the point of view of implementing information security threats, static source code analysis, and security testing. The introduction of such procedures into the practice of domestic software developers, in our opinion, will increase the level of security of the software being created and, as a result, significantly reduce the number of information security incidents.
Source: https://habr.com/ru/post/341648/
All Articles