Mitigating the Effect of Coincidental Correctness in Spectrum Based Fault Localization

Abstract

Coincidentally correct test cases are those that execute faulty statements but do not cause failures. Such test cases reduce the effectiveness of spectrum-based fault localization techniques, such as Ochiai. These techniques calculate a suspiciousness score for each statement. The suspiciousness score estimates the likelihood that the program will fail if the statement is executed. The presence of coincidentally correct test cases reduces the suspiciousness score of the faulty statement, thereby reducing the effectiveness of fault localization. We present two approaches that predict coincidentally correct test cases and use the predictions to improve the effectiveness of spectrum based fault localization. In the first approach, we assign weights to passing test cases such that the test cases that are likely to be coincidentally correct obtain low weights. Then we use the weights to calculate suspiciousness scores. In the second approach, we iteratively predict and remove coincidentally correct test cases, and calculate the suspiciousness scores with the reduced test suite. In this dissertation, we investigate the cost and effectiveness of our approach to predicting coincidentally correct test cases and utilizing the predictions. We report the results of our preliminary evaluation of effectiveness and outline our research plan.