An intuitive approach to determine test adequacy in safety-critical software

Abstract

Safety-critical software must adhere to stringent quality standards and is expected to be thoroughly tested. However, exhaustive testing of software is usually impractical. The two main challenges faced by a software testing team are generation of effective test cases and demonstration of testing adequacy. This paper proposes an intuitive and conservative approach to determine the test adequacy in safety-critical software. The approach is demonstrated through a case study: the core temperature monitoring system of a nuclear reactor. We combine conservative test coverage of unique execution path test cases, and the results from mutation testing to determine the test adequacy. Although mutation testing is a powerful technique, the difficulty in identifying equivalent mutants has limited its practical utility. To gain confidence on the computed test adequacy: (i) faults during mutation testing must be induced at all possible execution paths of the code, (ii) properties of unkilled mutants must be studied, and (iii) all equivalent mutants must be detected. In this regard; results of static, dynamic and coverage analysis of the mutants is presented, and a technique to identify the likely equivalent mutants is proposed.