Partitioning the requirements of embedded systems by input/output dependency analysis for compositional creation of parallel test models

Abstract

In industry, embedded systems continue to become even more complex. The development is done increasingly in a virtual environment to accelerate the prototyping and the implementation. Independent from this fact, testing is still the essential activity for verification and validation and obtaining confidence in the quality and reliability of the product. The main driver behind innovation is software, and due to its relevance in September 2013 the first international standard for software testing, i.e. the ISO/IEC/IEEE 29119, was published. It states the main purpose of requirements based testing as to determine whether the test item meets end-user requirements [1, Part 1, p.31]. In the same section it is mentioned, that testing may suffer if the requirements are incomplete or not consistently specified. This highlights, that with the importance of testing, the crucial issue is the quality of the testing activities. Research and industry have researched, how to derive test cases e.g. to cover requirements within minimal time or to achieve coverage criteria like pairwise testing. For the automation of the test execution, the assessment and reporting a broad tool support is available. Yet, one question is increasingly difficult: What are good, i.e. significant test cases. This leads to the question, from what should test cases be derived, to represent the most significant scenarios of use. Constructive requirements modeling methods guarantee that the requirements are analyzed and the final test model is complete and unambiguous. To achieve this output, however, all observable states of use and their combinations are considered during creation of the model. This is apparently cumbersome and hardly feasible in the industrial routine. In this paper we present a novel approach to reduce the effort for creating the model and facilitate its appliance in industry. We stick to the foundations of constructive enumeration to create a complete, traceably correct, and consistent model, but we do first decompose the task into manageable units by input/output dependency analysis. The expected behavior is formalized in temporal logic. The resulting model is a composition of all models, that run in parallel. As time is explicitly considered during the creation of the model, timing information is available for structured testing of non functional, e.g. real time requirements, as well as for the determination of measures and dependability estimators. By this approach, the subsequent activities for quality assurance, such as validation and verification, measurement of coverage criteria, and dependability estimators, e.g. of reliability, safety, and risk, profit from this approach, as they rely on a provably correct basis. We applied the method to an embedded system of a German automotive OEM, that was designed in Matlab Simulink and architectured with AUTOSAR 3.2 methodology. An existing test suite was at hand, that was created with the established method. This existing test suite served as benchmark to assess the quality of the new test suite, derived from the model. We compared the reachability of the test cases inside the implementation with code coverage measures and examined the variance of use imposed by the test suites. We present the promising results in this paper.