Avionics Software Development
At the heart of avionics software development is the fundamental standard RTCA \ DO-178B . Despite the first glance at his detachment from the programmer’s immediate routine, he describes the entire development process and places demands on such software. Nevertheless, this article will also discuss how everything actually happens, based on personal experience in the development of flight monitoring and control systems, landing systems, etc., for airplanes and helicopters.
Modern solutions for the control and monitoring of aircraft control systems ( Flight Control System ) is a complex hardware and software system, the work of which, perhaps, as a whole, none of the employees and developers know. This is akin to the projects of the development of the atomic bomb of the Second World War, where everyone knows a part of his work, but does not really know why it works together. However, avionics is not the only example of such complex systems, and the complexity of the same Microsoft Excel or GNU GCC, of ​​course, creates similar problems, but, nevertheless, for aviation software, there are nuances and typical solutions that I will try to focus separately on. Facing the problem of the effectiveness of managing the development process, the management, trying to follow the optimization of the cost parameters and the quality of the project, generates an informational and organizational deficit. This is primarily due to the high cost of specialists and / or their training in the field of aviation software, i.e. with staff because Often in large projects, its size can reach about 2-3 thousand people per one control system (not to mention the dynamic model and the physical performance of the airframe, and especially the product). In the second place - with the organization of communication and submission of information, its synchronization between the participants of the development, as well as restrictions on excessively large amounts of data passing through a particular level. Therefore, for the development of such systems, a special, carefully documented and regulated technical process of developing requirements, creating hardware and software, as well as performing and debugging the system, as well as testing it and drawing up certification documentation has been approved. Nevertheless, the process is constantly being modified and improved, based on the realities of the project and from the picture of the surrounding world.
For the development of critical systems, it is necessary to ensure the minimum possibility of error (for the highest level in avionics, the probability of failure is set to 10 ^ -9), as well as to minimize development costs and code corrections. Due to the complexity of the system and interconnection with other parts (other software, other equipment), a waterfall model or a flexible development may not be the best option, therefore, the V-shaped development model was chosen as the basic principle for developing such software.
')
Figure 1. V-shaped development model.
First of all, this model allows you to ensure synchronization of all project participants at each iteration, as well as provides the opportunity to use the already accumulated data and ready-made methodology, since at the start of any project, the V-shaped model (Fig. 1) can be adapted for this project, since this model does not depend on the types of organizations and projects. V-model allows you to divide the activity into separate steps, each of which will include the necessary actions for him, instructions to them, recommendations and a detailed explanation of the activity. This is especially important for a multi-iteration cycle of development and testing of avionics software, since allows, in fact, to directly break down software development into separate sub-cycles. Usually the V-model is generalized to a spiral model of development (Figure 2). Which, in turn, allows assessing the risks at each development stage, as well as optimally distributes the employment of specialists (workload) in the conditions of staff shortage and time for short periods of time (Iteration Packages synchronized with the V-shaped model in each Baseline).
Figure 2. Spiral Software Development Testing Model
For control of each stage and for the subsequent possibility of certification, the process is divided into various levels, each of which has its own document, for which a document is created that controls it (report). As a result, each development stage, all errors and corrections are classified and documented. By repeating the development iterations each time, the probability of error is reduced. These documents are also created on the basis of the company's internal standards and requirements submitted by the customer.
Figure 3. The relationship of documents and requirements
At the head of everything is, of course, the customer, who often doesn’t really have much idea what he needs, but is quite able to say that he wants his plane to fly, has a taxiing and air-conditioning system for all occasions, as well as this system to work the way he wants it, and even with bonuses. Therefore, the first part is the analysis of customer requirements and determination of the basic functionality of the system, on the basis of which a general concept and system diagram is created, including the technical details of the equipment used, i.e. Creation of the initial specifications of the system (Equipment Specification) and requirements (System Requirements).
When the basis on which the system will be created is determined, a plan is approved according to which the software development plan and its Qualification plan will be carried out. Despite the fact that the main thing for the customer is to get ready-to-use management software, a parallel process is the development of hardware, which in software development cannot be ignored, because development of avionics software is very closely connected with the hardware; In the majority of software, although it is portable and embedded code, it is highly dependent on the system layout, but more on that later.
Figure 4. V-shaped model, divided into stages according to the statutory documents.
Before proceeding further, it is worth saying that the initial design is based on the fundamental principles that are present throughout the development process, and the main one is the “dissimilarity”, which determines that each part of the control systems must be implemented by different groups of people on different hardware stuffing using different software tools (including development tools, programming languages). Thus, the system is divided into software and hardware-independent parts, and the development process is controlled by different groups of people for different tasks and at different levels, respectively, in accordance with the higher requirements and plan.
The result of the initial design is a system model, usually performed in Matlab \ Simulink, Labview environments. Based on the model, documents are created that regulate what hardware should be used and what connections they should have between themselves. At a minimum, the result of this stage is the creation of two documents: those that define the hardware component (hardware) and the software-hardware (hardware-software).
Then begins the stage of the engineering process of preparation, assembly of boards and finished modules (Control Electronic, etc.), i.e. directly mounting, distributing the necessary microcontrollers, microcircuits, peripherals, for which the necessary software will be written. To interact with the hardware, there must be drivers and an interaction layer (framework layer), on the basis of which an application must be built. However, often the work of a programmer begins here, when it is necessary to add the necessary drivers / functionality, and most often make changes to the "comprehensive library", based on the HSI document (Hardware-Software Interface). So, the most common practice is to “cut back” the functionality of the system to the limit of the equipment and drivers used, as well as to change some of the calibration settings, including unusual pinout or optimization for the selected real-time parameters.
Figure 5. Software organization
Accordingly, the universal library Framework includes all sorts of drivers for devices certified for use in aviation, as well as certified standard functions and procedures.
This is a fundamental point, because the most common function strcmp, for example, can not be used directly, it must be rewritten according to standards and be certified. A set of such certified standard functions, prototypes, templates is Common in the Framework layer. This attitude is particularly critical to safe mathematical operations (fast fractional division for integer processors, module, root, as an example), and for working with memory. All algorithms, though a bit (at least in the style of the code), are different from STL.
Figure 6. The architecture of the test cluster, with the ability to use different interfaces for monitoring and analysis *
For use on various devices, the Framework includes driver sets having the structure DrvHigh <-> DrvLow. Here, the DrvHigh package contains all sorts of interfaces for device drivers (Flash, Eeprom memory, digital-analog, analog-digital converters, real-time clock, interrupts, CAN chips, ARINC, LAN, etc.). In turn, each of these driver interfaces can use one or several drivers for a specific device (this or that memory chip, converter, etc.). For optimization purposes, such drivers are reconfigured or even used directly without the DrvHigh level. Perhaps this is not the most beautiful solution, but unlike application programs, working with hard real-time embedded systems, where “640kb should be enough for everyone” is not just an aphorism, but a reality. For high-load and fault-tolerant systems, the realities are as peak chip loads, 90-100% load on data buses, synchronization of channels, devices and frame loading scheduling depending on input parameters (frame scheduling) with error control (including multi-level monitoring with confirmation of static and oscillation errors), and putting all software and data objects in volumes of the order of 64-128kbytes.
But back to the software development cycle. Once the hardware is installed and configured, a Software Requirement Document is created that describes what the software should do and how to do it. This is the document on the basis of which the application should be developed. Here, in fact, the work of the programmer, along with the work of the tester, begins.
So, in other words, the avionics software programmer does not see a complete picture of what he is actually creating the program for, but he operates with the necessary requirements and the architecture he has to create based on the requirements. Requirements for the programmer are the following documents:
On the basis of the first document, the developer is determined how to solve his problem: what technical means he can use, what and where he should put the results, as well as what rules (rules) and advice (guidelines) he should follow. Speaking quite simply, this document sets the developer's toolkit (programming language, development environment and reports).
On the basis of the second document, the developer is set the style of his code and the permitted techniques. For example, the use of the Hungarian notation, arithmetic operations, the style of writing code and its complexity.
Also for the work of the developer, as I mentioned, you may need knowledge of low-level requirements (HSI, ICD (Interface Communication), Datasheets (documents describing the operation of devices, usually from the creators of devices (on different chips)).
The development process itself consists of the following steps:
1. Design - design (design development in UML and / or modeling systems (using software such as Ameos, SCADE, Simulink, etc.) -
2. Low-Level Requirements - Description of the functionality and algorithm for solving the realizable requirement for the tester (using the black box model: description of the input and the expected reaction). Those. low-level specification.
3. Coding - the process of directly writing code (anything, if you do not use the automatic code generator, as in SCADE \ Matlab, because the development environment (IDE) can be any and can be used under any OS (I use Eclipse, CodeBlocks , although other solutions are not forbidden).
4. Debugging - the process of debugging, or rather the process of processing and assembly to the state of no errors (no Errors, Warnings with the parameters of the selected compiler installed).
5. Static check - the process of checking and correcting code based on code analyzers (xLite, Polyspace, MISRA, QAC).
6. Engineering tests - the process of launching and integrating the software on the simulator (i.e., on the prototype of the equipment, the final version of which will later be installed in flight, with interfaces and tools of manipulation derived as far as possible (usually a Labview + Trace32 debugger bundle)). In some cases, the functionality of the simulator is expanded by installing additional devices (sensors, disconnecting circuits, signal generators, and even control and monitoring devices such as a pilot's handle, etc.). In very rare cases for a few control systems, this can be done on a real full-scale bench model aircraft.
7. Entering the results into the version control system and reports (IBM Rational ClearCase \ ClearQuest).
Figure 7. "Electronic Bird" Sukhoi SuperJet 100 *
All these seven points make up one iteration, usually performed only for its part of the requirements. Due to changes in functionality or corrections / amendments to an already tested code, it is necessary to create Change Requests, on the basis of which, as a document, corrections will be made to existing reporting documentation or code, usually this happens in the system. Upon closing one of the Baseline, subsequent changes to the code or documentation are not made, but can only be initiated on the basis of the Problem Reports. Such complexity is needed to avoid unauthorized and dangerous changes to the code and documentation, and to stabilize the code as it is before the corresponding activity. The very same code and \ or specification change after permission to Change Request, which documents which revisions were made.
Upon completion of each of Baseline, an SDD (Software Description Document) document is created, which contains information about the design of the application, as well as the low-level requirements provided by the developer to the tester. However, before transferring it to testers, an analysis and review (design review) of this document is made for errors and testing capabilities in the requirements given in the document (made by another developer, usually responsible for another part of the functionality). This is where the developer’s work ends, and he proceeds either to the next Baseline or, if the project is completed, to the next project. In this case, of course, the developer enters as a consultant in connection with the tester, providing him with the necessary assistance.
Nevertheless, it is necessary to mention that each of the links (system engineer, programmer, tester) should be divided in order to avoid possible influence and pressure from their side. But, of course, there are always controversial issues and nuances, and sometimes, despite the clarity of the process, in order not to overload the technical process with iterations due to typing errors in the comments (for example), the agile development model is not used that affects the main functionality.
The job of the tester is almost half, if not 2/3 of the entire software development time. It is painstaking and long work, which includes:
which consists, in turn, of:
In subsequent iterations, the process can be minimized to delta-testing and delta-review, i.e. Testing only changed parts of the program \ document or correcting previous testing errors. This should save time, however, often, as practice shows, errors are present until the end of the entire development process, therefore, testers often test the system completely every time based on ready-made tests. This can be considered as regression testing, with the exception of high time costs and often an increased percentage of changes in the code \ tests. Here, I emphasize that this is due to the fact that tests are created on the basis of the system, and not in parallel with it. As you might guess, because of this approach, the main problems fall on the shoulders of programmers, who must provide an excellent balance of architecture, code and low-level specification. Parallel development is almost impossible due to the need to be able to trace the error down to the function and the variable that causes the crash. Not only in tests, but also at the level of requirements. Although it looks like a heavy minus, from such a model, they gradually try to move to more flexible models, and make the black-box process so that the tests do not come from the code, but go from the architecture design stage. Ideally, write specifications before writing code.
Figure 8. Test benches for electronics and related systems *
Slightly less often, at the final stages of development, purely hardware testing occurs, which implies testing on real equipment with drawing up an act. Usually this happens on the bench tests of the assembled model of the aircraft, and later in flight on a real aircraft.
Between each stage, naturally, a “delivery package” is formed, which includes both the specification of the used equipment, the versions of all devices and documents, and the software itself and related documents. This is done by specialized managers (package manager), and coordinator manager (coordinator manager) is responsible for coordinating and tracking the status of various groups. The stages of development and testing are regulated by internal plans (roadmap), which are also a reporting document for managers (actual status).
Based on tests, a general SVR document (Software Verification Review) is compiled, which at a particular development stage determines the error condition. On the basis of which, depending on their importance, a document on the end of the stage (SAS, Software Accomplishment Summary) is being compiled. This document determines whether the start of a new stage of development / processing (including processing of SWRD) is necessary, or the development is terminated, and all documentation is transferred to the certification and to the customer. This document is final for the technical control department, whose work is carried out constantly for each Baseline in the background, usually without having a strong influence on the technical process.
At this point, the audit and verification of the entire project begins, on the basis of which the last three documents are created:
Naturally, if problems arise at one of these stages, the situation is quite possible to initiate another (at least) software processing. As a rule, such certification due to the abundance of documentation (this is about 100-200 thousand pages) is checked exclusively selectively at a low level, and requirements for revision are drawn up at a high level according to the customer, certification commission and test pilots. As a rule, the number of flight test stages is two to three, and certification - 1-2.
As for the huge amounts of work, the time allotted for the development of a relatively simple system (system of steering and chassis release) is 1.5-2 years, for surface control systems (electric actuators and hydraulic systems) is 5-6 years. Thus, on average, the system runs from 2-3 large iterations (baseline) to 18-20 for large and complex systems, and more than 40 for the Framework layer.
Due to the extreme complexity and cumbersome reporting systems and testing routines, outsourcing resources are attracted to work in India, and less often in China and eastern Europe. All certification, as a rule, takes place in the territory where the certificate is valid (for EASA - Europe, for FAA - America), and for Russian standards - Russia. The equipment is certified separately, or it should already have its own certificate, therefore, unfortunately, or fortunately, relatively “outdated” models and solutions tested in temperature, time and aggressive operating conditions are used in aviation. Despite the enormous complexity and relevance, there are not so many good specialists, and even in America and Europe, electronic control systems are only a beginning direction, which, of course, contains a small but a portion of errors ... however, not to scare, o security and fault-tolerant architecture will be discussed next time.
* Test system prepared in collaboration with Cosateq .
Introduction
Modern solutions for the control and monitoring of aircraft control systems ( Flight Control System ) is a complex hardware and software system, the work of which, perhaps, as a whole, none of the employees and developers know. This is akin to the projects of the development of the atomic bomb of the Second World War, where everyone knows a part of his work, but does not really know why it works together. However, avionics is not the only example of such complex systems, and the complexity of the same Microsoft Excel or GNU GCC, of ​​course, creates similar problems, but, nevertheless, for aviation software, there are nuances and typical solutions that I will try to focus separately on. Facing the problem of the effectiveness of managing the development process, the management, trying to follow the optimization of the cost parameters and the quality of the project, generates an informational and organizational deficit. This is primarily due to the high cost of specialists and / or their training in the field of aviation software, i.e. with staff because Often in large projects, its size can reach about 2-3 thousand people per one control system (not to mention the dynamic model and the physical performance of the airframe, and especially the product). In the second place - with the organization of communication and submission of information, its synchronization between the participants of the development, as well as restrictions on excessively large amounts of data passing through a particular level. Therefore, for the development of such systems, a special, carefully documented and regulated technical process of developing requirements, creating hardware and software, as well as performing and debugging the system, as well as testing it and drawing up certification documentation has been approved. Nevertheless, the process is constantly being modified and improved, based on the realities of the project and from the picture of the surrounding world.
Development model
For the development of critical systems, it is necessary to ensure the minimum possibility of error (for the highest level in avionics, the probability of failure is set to 10 ^ -9), as well as to minimize development costs and code corrections. Due to the complexity of the system and interconnection with other parts (other software, other equipment), a waterfall model or a flexible development may not be the best option, therefore, the V-shaped development model was chosen as the basic principle for developing such software.
')
Figure 1. V-shaped development model.
First of all, this model allows you to ensure synchronization of all project participants at each iteration, as well as provides the opportunity to use the already accumulated data and ready-made methodology, since at the start of any project, the V-shaped model (Fig. 1) can be adapted for this project, since this model does not depend on the types of organizations and projects. V-model allows you to divide the activity into separate steps, each of which will include the necessary actions for him, instructions to them, recommendations and a detailed explanation of the activity. This is especially important for a multi-iteration cycle of development and testing of avionics software, since allows, in fact, to directly break down software development into separate sub-cycles. Usually the V-model is generalized to a spiral model of development (Figure 2). Which, in turn, allows assessing the risks at each development stage, as well as optimally distributes the employment of specialists (workload) in the conditions of staff shortage and time for short periods of time (Iteration Packages synchronized with the V-shaped model in each Baseline).
Figure 2. Spiral Software Development Testing Model
Design and documentation
For control of each stage and for the subsequent possibility of certification, the process is divided into various levels, each of which has its own document, for which a document is created that controls it (report). As a result, each development stage, all errors and corrections are classified and documented. By repeating the development iterations each time, the probability of error is reduced. These documents are also created on the basis of the company's internal standards and requirements submitted by the customer.
Figure 3. The relationship of documents and requirements
At the head of everything is, of course, the customer, who often doesn’t really have much idea what he needs, but is quite able to say that he wants his plane to fly, has a taxiing and air-conditioning system for all occasions, as well as this system to work the way he wants it, and even with bonuses. Therefore, the first part is the analysis of customer requirements and determination of the basic functionality of the system, on the basis of which a general concept and system diagram is created, including the technical details of the equipment used, i.e. Creation of the initial specifications of the system (Equipment Specification) and requirements (System Requirements).
When the basis on which the system will be created is determined, a plan is approved according to which the software development plan and its Qualification plan will be carried out. Despite the fact that the main thing for the customer is to get ready-to-use management software, a parallel process is the development of hardware, which in software development cannot be ignored, because development of avionics software is very closely connected with the hardware; In the majority of software, although it is portable and embedded code, it is highly dependent on the system layout, but more on that later.
Figure 4. V-shaped model, divided into stages according to the statutory documents.
Development
Concept
Before proceeding further, it is worth saying that the initial design is based on the fundamental principles that are present throughout the development process, and the main one is the “dissimilarity”, which determines that each part of the control systems must be implemented by different groups of people on different hardware stuffing using different software tools (including development tools, programming languages). Thus, the system is divided into software and hardware-independent parts, and the development process is controlled by different groups of people for different tasks and at different levels, respectively, in accordance with the higher requirements and plan.
Hardware and software
The result of the initial design is a system model, usually performed in Matlab \ Simulink, Labview environments. Based on the model, documents are created that regulate what hardware should be used and what connections they should have between themselves. At a minimum, the result of this stage is the creation of two documents: those that define the hardware component (hardware) and the software-hardware (hardware-software).
Then begins the stage of the engineering process of preparation, assembly of boards and finished modules (Control Electronic, etc.), i.e. directly mounting, distributing the necessary microcontrollers, microcircuits, peripherals, for which the necessary software will be written. To interact with the hardware, there must be drivers and an interaction layer (framework layer), on the basis of which an application must be built. However, often the work of a programmer begins here, when it is necessary to add the necessary drivers / functionality, and most often make changes to the "comprehensive library", based on the HSI document (Hardware-Software Interface). So, the most common practice is to “cut back” the functionality of the system to the limit of the equipment and drivers used, as well as to change some of the calibration settings, including unusual pinout or optimization for the selected real-time parameters.
Figure 5. Software organization
Framework
Accordingly, the universal library Framework includes all sorts of drivers for devices certified for use in aviation, as well as certified standard functions and procedures.
This is a fundamental point, because the most common function strcmp, for example, can not be used directly, it must be rewritten according to standards and be certified. A set of such certified standard functions, prototypes, templates is Common in the Framework layer. This attitude is particularly critical to safe mathematical operations (fast fractional division for integer processors, module, root, as an example), and for working with memory. All algorithms, though a bit (at least in the style of the code), are different from STL.
Figure 6. The architecture of the test cluster, with the ability to use different interfaces for monitoring and analysis *
For use on various devices, the Framework includes driver sets having the structure DrvHigh <-> DrvLow. Here, the DrvHigh package contains all sorts of interfaces for device drivers (Flash, Eeprom memory, digital-analog, analog-digital converters, real-time clock, interrupts, CAN chips, ARINC, LAN, etc.). In turn, each of these driver interfaces can use one or several drivers for a specific device (this or that memory chip, converter, etc.). For optimization purposes, such drivers are reconfigured or even used directly without the DrvHigh level. Perhaps this is not the most beautiful solution, but unlike application programs, working with hard real-time embedded systems, where “640kb should be enough for everyone” is not just an aphorism, but a reality. For high-load and fault-tolerant systems, the realities are as peak chip loads, 90-100% load on data buses, synchronization of channels, devices and frame loading scheduling depending on input parameters (frame scheduling) with error control (including multi-level monitoring with confirmation of static and oscillation errors), and putting all software and data objects in volumes of the order of 64-128kbytes.
Programming
Requirements
But back to the software development cycle. Once the hardware is installed and configured, a Software Requirement Document is created that describes what the software should do and how to do it. This is the document on the basis of which the application should be developed. Here, in fact, the work of the programmer, along with the work of the tester, begins.
So, in other words, the avionics software programmer does not see a complete picture of what he is actually creating the program for, but he operates with the necessary requirements and the architecture he has to create based on the requirements. Requirements for the programmer are the following documents:
- Software Design Standard - a standard that defines the overall style of the application and the approach to creating an architecture.
- Programming Standard - a programming standard that defines what is allowed to be written in the code and by what style.
- Software Requirements Document - software requirements, documented and divided into different Baseline and iteration package within them (high-level specificaction).
On the basis of the first document, the developer is determined how to solve his problem: what technical means he can use, what and where he should put the results, as well as what rules (rules) and advice (guidelines) he should follow. Speaking quite simply, this document sets the developer's toolkit (programming language, development environment and reports).
On the basis of the second document, the developer is set the style of his code and the permitted techniques. For example, the use of the Hungarian notation, arithmetic operations, the style of writing code and its complexity.
Also for the work of the developer, as I mentioned, you may need knowledge of low-level requirements (HSI, ICD (Interface Communication), Datasheets (documents describing the operation of devices, usually from the creators of devices (on different chips)).
Process
The development process itself consists of the following steps:
1. Design - design (design development in UML and / or modeling systems (using software such as Ameos, SCADE, Simulink, etc.) -
2. Low-Level Requirements - Description of the functionality and algorithm for solving the realizable requirement for the tester (using the black box model: description of the input and the expected reaction). Those. low-level specification.
3. Coding - the process of directly writing code (anything, if you do not use the automatic code generator, as in SCADE \ Matlab, because the development environment (IDE) can be any and can be used under any OS (I use Eclipse, CodeBlocks , although other solutions are not forbidden).
4. Debugging - the process of debugging, or rather the process of processing and assembly to the state of no errors (no Errors, Warnings with the parameters of the selected compiler installed).
5. Static check - the process of checking and correcting code based on code analyzers (xLite, Polyspace, MISRA, QAC).
6. Engineering tests - the process of launching and integrating the software on the simulator (i.e., on the prototype of the equipment, the final version of which will later be installed in flight, with interfaces and tools of manipulation derived as far as possible (usually a Labview + Trace32 debugger bundle)). In some cases, the functionality of the simulator is expanded by installing additional devices (sensors, disconnecting circuits, signal generators, and even control and monitoring devices such as a pilot's handle, etc.). In very rare cases for a few control systems, this can be done on a real full-scale bench model aircraft.
7. Entering the results into the version control system and reports (IBM Rational ClearCase \ ClearQuest).
Figure 7. "Electronic Bird" Sukhoi SuperJet 100 *
All these seven points make up one iteration, usually performed only for its part of the requirements. Due to changes in functionality or corrections / amendments to an already tested code, it is necessary to create Change Requests, on the basis of which, as a document, corrections will be made to existing reporting documentation or code, usually this happens in the system. Upon closing one of the Baseline, subsequent changes to the code or documentation are not made, but can only be initiated on the basis of the Problem Reports. Such complexity is needed to avoid unauthorized and dangerous changes to the code and documentation, and to stabilize the code as it is before the corresponding activity. The very same code and \ or specification change after permission to Change Request, which documents which revisions were made.
Specification
Upon completion of each of Baseline, an SDD (Software Description Document) document is created, which contains information about the design of the application, as well as the low-level requirements provided by the developer to the tester. However, before transferring it to testers, an analysis and review (design review) of this document is made for errors and testing capabilities in the requirements given in the document (made by another developer, usually responsible for another part of the functionality). This is where the developer’s work ends, and he proceeds either to the next Baseline or, if the project is completed, to the next project. In this case, of course, the developer enters as a consultant in connection with the tester, providing him with the necessary assistance.
Nevertheless, it is necessary to mention that each of the links (system engineer, programmer, tester) should be divided in order to avoid possible influence and pressure from their side. But, of course, there are always controversial issues and nuances, and sometimes, despite the clarity of the process, in order not to overload the technical process with iterations due to typing errors in the comments (for example), the agile development model is not used that affects the main functionality.
Testing
The job of the tester is almost half, if not 2/3 of the entire software development time. It is painstaking and long work, which includes:
Low Level Testing
which consists, in turn, of:
- Code review - review the source code for compliance with the programming standard, as well as for compliance with the code and requirements (SDD).
- Low-Level Testing - low-level testing, such as unit-testing and unit-integration-testing (Razorcat Tessy + environment and processor emulators), i.e. testing based on the requirements of the code itself, using the McCabe metric and Modified Condition / Decision Coverage (using the NASA MCDC standard). Also, all boundary values ​​are checked here, as well as the system’s response to exiting acceptable conditions (reaction to invalid mathematical operations, to operations with memory, pointers, going beyond the limits of the range, etc. (robustness testing)).
- Creating a reporting document (Software Verification Cases and Procedures \ Software Unit and Integration Verification Cases and Procedures).
- VoV (Verification of Verification) is a verification process that verifies the validity of the tests as well as the created document, and records the results in a QAR (Quality Assurance Record), which will later be used at the next iteration to correct any errors (performed by another tester ). Naturally, all errors are fixed in addition to QAR and in the bug tracking system so that Iteration can find and fix the problem.
In subsequent iterations, the process can be minimized to delta-testing and delta-review, i.e. Testing only changed parts of the program \ document or correcting previous testing errors. This should save time, however, often, as practice shows, errors are present until the end of the entire development process, therefore, testers often test the system completely every time based on ready-made tests. This can be considered as regression testing, with the exception of high time costs and often an increased percentage of changes in the code \ tests. Here, I emphasize that this is due to the fact that tests are created on the basis of the system, and not in parallel with it. As you might guess, because of this approach, the main problems fall on the shoulders of programmers, who must provide an excellent balance of architecture, code and low-level specification. Parallel development is almost impossible due to the need to be able to trace the error down to the function and the variable that causes the crash. Not only in tests, but also at the level of requirements. Although it looks like a heavy minus, from such a model, they gradually try to move to more flexible models, and make the black-box process so that the tests do not come from the code, but go from the architecture design stage. Ideally, write specifications before writing code.
Figure 8. Test benches for electronics and related systems *
High Level Testing
- Document review (SWRD review) - a review of the document for errors and testability in terms of system requirements and specifications, as well as Hardware Compliance Software (ICD, HSI).
- Hardware-Software testing is the process of creating and running high-level tests on the simulator, which can be automated (scripted), manual (scripted, with the need to switch hardware somewhere, measure with a multimeter or an oscilloscope), Unit-tests (in the case of when it is impossible to check on the simulator, the test / tests from low-level ones are withdrawn). Such testing is very close to Engineering tests that are conducted by the developer, except that their results are recorded in a report file (Hardware / Software Integration Verification Cases and Procedures), and the whole process is usually automated.
- HwSw VoV is a verification verification process that verifies the correctness of test execution as well as the created document, and records the results in a QAR (Quality Assurance Record), which will later be used in the next iteration to correct any errors (performed by another tester).
Slightly less often, at the final stages of development, purely hardware testing occurs, which implies testing on real equipment with drawing up an act. Usually this happens on the bench tests of the assembled model of the aircraft, and later in flight on a real aircraft.
Between each stage, naturally, a “delivery package” is formed, which includes both the specification of the used equipment, the versions of all devices and documents, and the software itself and related documents. This is done by specialized managers (package manager), and coordinator manager (coordinator manager) is responsible for coordinating and tracking the status of various groups. The stages of development and testing are regulated by internal plans (roadmap), which are also a reporting document for managers (actual status).
Certification
Based on tests, a general SVR document (Software Verification Review) is compiled, which at a particular development stage determines the error condition. On the basis of which, depending on their importance, a document on the end of the stage (SAS, Software Accomplishment Summary) is being compiled. This document determines whether the start of a new stage of development / processing (including processing of SWRD) is necessary, or the development is terminated, and all documentation is transferred to the certification and to the customer. This document is final for the technical control department, whose work is carried out constantly for each Baseline in the background, usually without having a strong influence on the technical process.
At this point, the audit and verification of the entire project begins, on the basis of which the last three documents are created:
- First Delivery Review (FDR) - package delivery document,
- First Flight Review (FFR) - First Flight Report,
- Software Certification Review (SCR) - decision of the certification commission.
Naturally, if problems arise at one of these stages, the situation is quite possible to initiate another (at least) software processing. As a rule, such certification due to the abundance of documentation (this is about 100-200 thousand pages) is checked exclusively selectively at a low level, and requirements for revision are drawn up at a high level according to the customer, certification commission and test pilots. As a rule, the number of flight test stages is two to three, and certification - 1-2.
Conclusion
As for the huge amounts of work, the time allotted for the development of a relatively simple system (system of steering and chassis release) is 1.5-2 years, for surface control systems (electric actuators and hydraulic systems) is 5-6 years. Thus, on average, the system runs from 2-3 large iterations (baseline) to 18-20 for large and complex systems, and more than 40 for the Framework layer.
Due to the extreme complexity and cumbersome reporting systems and testing routines, outsourcing resources are attracted to work in India, and less often in China and eastern Europe. All certification, as a rule, takes place in the territory where the certificate is valid (for EASA - Europe, for FAA - America), and for Russian standards - Russia. The equipment is certified separately, or it should already have its own certificate, therefore, unfortunately, or fortunately, relatively “outdated” models and solutions tested in temperature, time and aggressive operating conditions are used in aviation. Despite the enormous complexity and relevance, there are not so many good specialists, and even in America and Europe, electronic control systems are only a beginning direction, which, of course, contains a small but a portion of errors ... however, not to scare, o security and fault-tolerant architecture will be discussed next time.
* Test system prepared in collaboration with Cosateq .
Source: https://habr.com/ru/post/144686/
All Articles