Spectrum Based Fault Localisation Research Articles

Empirical Evaluation of Fault Localisation Using Code and Change Metrics

Fault localisation aims to reduce the debugging efforts of human developers by highlighting the program elements that are suspected to be the root cause of the observed failure. Spectrum Based Fault Localisation (SBFL), a coverage based approach, has been widely studied in many researches as a promising localisation technique. Recently, however, it has been proven that SBFL techniques have reached the limit of further improvement. To overcome the limitation, we extend SBFL with code and change metrics that have been mainly studied in defect prediction, such as size, age, and churn. FLUCCS, our fault learn-to-rank localisation technique, employs both existing SBFL formulæ and these metrics as input. We investigate the effect of employing code and change metrics for fault localisation using four different learn-to-rank techniques: Genetic Programming, Gaussian Process Modelling, Support Vector Machine, and Random Forest. We evaluate the performance of FLUCCS with 386 real world faults collected from Defects4J repository. The results show that FLUCCS with code and change metrics places 144 faults at the top and 304 faults within the top ten. This is a significant improvement over the state-of-art SBFL formulæ, which can locate 65 and 212 faults at the top and within the top ten, respectively. We also investigate the feasibility of cross-project transfer learning of fault localisation. The results show that, while there exist project-specific properties that can be exploited for better localisation per project, ranking models learnt from one project can be applied to others without significant loss of effectiveness.

A dynamic slicing-based approach for effective SBFL technique

Fault finding is an activity to locate the fault or bug present in a software. It is a very time taking job and needs much more effort if done manually. Hence, automated fault localisation is always in high demand which reduces the human effort and also makes the task more accurate. Among different existing debugging techniques, spectrum-based debugging is the most efficient for automated fault localisation. Dynamic program slicing is another technique which can reduce the debugging time by reducing the unaffected source codes depending on slicing criteria. In this paper, we present a spectrum-based fault localisation technique by using dynamic slicing. Context-sensitive slicing is used to diminish fault localisation time and makes the process more effective. SBFL metrics are used in the sliced program to find the suspiciousness score of individual program statements. The efficiency of the proposed approach is evaluated on three open-source programs. From results, we notice that due to dynamic slicing the technique takes less time to find the suspiciousness score of individual statements in the sliced program compared to original program. We have also observed that the programmer needs to inspect less source code to detect the buggy statement. The result says that the proposed approach outperforms the pure spectrum-based fault localisation techniques.

Robustness of spectrum-based fault localisation in environments with labelling perturbations

Most fault localisation techniques take as inputs a faulty program and a test suite, and produce as output a ranked list of suspicious code locations at which the program may be defective. If only a small portion of the executions are labelled erroneously, we expect a fault localisation technique to be robust to these errors. However, it is not known which fault localisation techniques with high accuracy are robust and which techniques are best at finding faults under the trade-off between accuracy and robustness.In this paper, a theoretical analysis of the impacts of labelling perturbations on spectrum-based fault localisation techniques (SBFL) is presented from different aspects first. We theoretically analyse the influence of labelling perturbations on three relations among risk evaluation formulas and the effect of mislabelling cases on the ranking of faulty statements. Then, we conduct controlled experiments on 18 programs with 3079 faulty versions from different domains to compare the robustness of 23 classes of risk evaluation formulas. Besides, experiments are conducted for evaluating the robustness of two neural network-based techniques. The impacts of perturbation degrees, number of faults and types of labelling perturbation on the robustness of formulas are empirically studied, and several interesting findings are obtained.

Kernel-based detection of coincidentally correct test cases to improve fault localisation effectiveness

Although empirical studies have confirmed the effectiveness of spectrum-based fault localisation (SBFL) techniques, their performance may be degraded due to presence of some undesired circumstances such as the existence of coincidental correctness (CC) where one or more passing test cases exercise a faulty statement and thus causing some confusion to decide whether the underlying exercised statement is faulty or not. This article aims at improving SBFL effectiveness by mitigating the effect of CC test cases. In this regard, a new method is proposed that uses a support vector machine (SVM) with a customised kernel function. To build the kernel function, we applied a new sequence-matching algorithm that measures the similarities between passing and failing executions. We conducted some experiments to assess the proposed method. The results show that our method can effectively improve the performance of SBFL techniques.

Human Competitiveness of Genetic Programming in Spectrum-Based Fault Localisation

We report on the application of Genetic Programming to Software Fault Localisation, a problem in the area of Search-Based Software Engineering (SBSE). We give both empirical and theoretical evidence for the human competitiveness of the evolved fault localisation formulæ under the single fault scenario, compared to those generated by human ingenuity and reported in many papers, published over more than a decade. Though there have been previous human competitive results claimed for SBSE problems, this is the first time that evolved solutions have been formally proved to be human competitive. We further prove that no future human investigation could outperform the evolved solutions. We complement these proofs with an empirical analysis of both human and evolved solutions, which indicates that the evolved solutions are not only theoretically human competitive, but also convey similar practical benefits to human-evolved counterparts.