Backward-Slice-Based Statistical Fault Localization without Test Oracles

Abstract

A recent promising technique for fault localization, Backward-Slice-based Statistical Fault Localization (BSSFL), statistically analyzes the backward slices and results of a set of test cases to evaluate the suspiciousness of a statement being faulty. However, BSSFL like many existing fault localization approaches assumes the existence of a test oracle to determine whether the result of a test case is a failure or pass. In reality, test oracles do not always exist, and therefore in such cases BSSFL can be severely infeasible. Among current research, metamorphic testing has been widely studied as a technique to alleviate the oracle problem. Hence, we leverage metamorphic testing to conduct BSSFL without test oracles. With metamorphic testing, our approach uses the backward slices and the metamorphic result of violation or non-violation for a metamorphic test group, rather than the backward slice and the result of failure or pass for an individual test case in BSSFL. Because our approach does not need the execution result of a test case, it implies that BSSFL can be extended to those application domains where no test oracle exists. The experimental results on 8 programs and 2 groups of the maximal suspiciousness evaluation formulas show that our proposed approach demonstrates the effectiveness comparable to that of existing BSSFL techniques in the cases where test oracles exist.