Exploring the usefulness of unlabelled test cases in software fault localization

Abstract

The usefulness of unlabelled test cases in fault localization is explored.An approach using test classification is proposed to utilize unlabelled test cases.A classifier utilizing the domain knowledge of fault localization is designed.Experiments are conducted to demonstrate the usefulness of our approach. In automatic software fault localization techniques, both the coverage and the testing outcomes of the provided test suite are considered to be essential information. The problem occurs when test oracles do not exist. Specifically, the test suite will contain a large number of unlabelled test cases, i.e., test cases whose output is not identified as being either correct (passing) or incorrect (failing). Such unlabelled test cases cannot be directly used, thereby leading to a degradation of localization effectiveness. In this paper, we propose an approach based on test classification to enable the use of unlabelled test cases in localizing faults. In our approach, unlabelled test cases are classified based on their execution information and are then assigned corresponding estimated labels to allow them to be utilized in fault localization. Experimental results show that with the utilization of these newly labelled test cases, the effectiveness of fault localization can indeed be improved.