FCCI: A fuzzy expert system for identifying coincidental correct test cases

Abstract

Spectrum-based fault localization (SBFL) is a promising approach to reduce the cost of program debugging and there has been a large body of research on introducing effective SBFL techniques. However, performance of these techniques can be adversely affected by the existence of coincidental correct (CC) test cases in the test suites. Such test cases execute the faulty statement but do not cause failures. Given that coincidental correctness is prevalent, it is necessary to precisely identify CC test cases and eliminate their effects from test suites. To do so, in this paper, we propose several important factors to identify CC test cases and model the CC identification process as a decision making system by constructing a fuzzy expert system and proposing a novel fuzzy CC identification method, namely FCCI. FCCI estimates the CC likelihood of passed test cases using the designed fuzzy rules, which effectively correlate the proposed CC identification factors. We evaluated FCCI by conducting extensive experiments on 17 popular and open source subject programs ranging from small- to large-scale containing both artificial and real faults. The experimental results indicate that FCCI successfully improves the accuracy of the CC identification as well as the accuracy of the representative SBFL techniques.