Experimental evaluation of the tolerance for control-flow test criteria: Research Articles

Abstract

Fault-detection effectiveness of coverage criteria has remained one of the controversial issues in recent years. In order to detect a fault, a test set must execute the faulty statement, cause infection of the data state and then propagate the faulty data state to bring about a failure. This paper sheds some light on the earlier contradictory results by investigating the infection aspect of coverage criteria.For a given test criterion, the number of test sets satisfying the criterion may be very large, with varying fault-detection effectiveness. In a recent work the measure of variation in effectiveness of a test criterion was defined as ‘tolerance’. This paper presents an experimental evaluation of tolerance for control-flow test criteria by exhaustive test set generation, wherever possible. The approach used here is complementary to earlier empirical studies that adopted analysis of some test sets using random selection techniques. Four industrially used control-flow testing criteria, Condition Coverage (CC), Decision Condition Coverage (DCC), Full Predicate Coverage (FPC) and Modified Condition Decision Coverage (MCDC) have been analysed against four types of faults. A new test criterion, Reinforced Condition Decision Coverage (RCDC), is also analysed and compared. Copyright © 2004 John Wiley & Sons, Ltd.A version of this paper was originally presented at SoftTest II: The Second U.K. Workshop on Software Testing Research, held at the University of York, U.K., 4–5 September 2003. It is reproduced here in modified form with the permission of the Workshop organizers