How test suites impact fault localisation starting from the size

Abstract

Although a test suite is indispensable for conducting effective fault localisation, not much work has been done to study how the test suite impacts fault localisation. This study presents a systematic study for a deeper understanding of their relation. Specifically, the authors' study reveals an interesting fact that there is no strong correlation between localisation effectiveness and the size of the test suite. Furthermore, they show that, in a test suite, (i) the passing test cases that do not execute the faulty statements and the failing test cases have a positive effect on the fault localisation effectiveness, while (ii) the passing test cases that exercise the faulty statements have a negative impact on localisation performance. Their result is drawn from a large-scale empirical analysis on the localisation effectiveness with respect to randomly sampled test suites. This study presents the details of the study and their follow-up investigation on the findings. Their work provides a new perspective on fault localisation and suggests fresh directions of research on an extensively studied topic.