Test generation for large automotive models

Abstract

Modeling tools such as Statemate, Simulink and Stateflow are widely used in the automotive industry to specify low level requirements and design. Systematic testing of models to achieve structural coverage such as state coverage, transition coverage or modified condition decision coverage (MCDC) helps in early defect detection. Automatic generation of test data can help in reducing the cost and improving the quality of systematic testing. Test data can be automatically generated either 1) directly from the models or 2) from the code generated from these models. In this paper we argue for and recommend the second approach. We propose generating test data from C, a formalism-independent intermediate language, as it is widely used in the embedded domain and most modeling tools have C code generators. Accurate representation of floating point number type is possible in C (that being the representation in the final executable) and there are various analysis tools that are available for C. A major challenge in using code to generate test data is scalability. To overcome this problem, we built a tool that combines available static slicing and model-checking techniques to generate test data. We conducted experiments to check if this tool can generate test data for large complex models from the automotive domain. To demonstrate formalism independence and scalability we chose industry size Statemate as well as Simulink/Stateflow models. The set up and the findings of these experiments are also presented in this paper. We successfully generated test data for code sizes as large as 50KLOC and detected several bugs in four already tested industry models thus proving the benefits of this approach.