Axiomatic assessment of control flow-based software test adequacy criteria

Abstract

Software test data adequacy criteria are rules to determine whether a piece of software has been adequately tested. Since Goodenough and Gerhard pointed out that the central problem of software testing is ‘what is a test data adequacy criterion’, many test data adequacy criteria have been proposed and investigated in the literature. Each criterion has its own strength and weakness. The comparison of these adequacy criteria has long been desirable but unsatisfactory. One of the approaches to compare existing software test adequacy criteria is to assess them against our intuitive concept of software test adequacy, by presenting a set of axioms of the ideal test adequacy criterion and checking if existing criteria satisfy these axioms. Weyuker is perhaps the first computer scientist who explicitly employs the notion of axiom systems in the study of software testing. However, an open problem in the application of this approach is how to avoid the weakness in the use of negative properties as axioms. In the study of control flow graphs, Baker et al. have proposed five properties as the requirements of control flow-based adequacy criteria. These properties are positive requirements of test adequacy criteria, but informal and less systematic. Some of the properties conflict with Weyuker's axioms. In the paper, we propose an axiom system combining the framework of Weyuker's system with Baker et al's positive properties. These properties are analysed, formalised and modified. A set of control flow-based test adequacy criteria is assessed against these axioms.