Existing Fault Localization Techniques Research Articles

Model-domain failing test augmentation with Generative Adversarial Networks

It is undoubtedly that test suites are crucial for fault localization techniques. Through the running of a single test case in the test suite, a fault localization technology is able to obtain the dynamic coverage information of the statements, which is a vector and could be analyzed to identify the corresponding coefficients between statements and program failures. This vector is commonly referred to as a model domain test sample. There are two types of model domain test samples: passing ones and failing ones. However, the number of passing test samples often far exceeds the number of failing ones, which undoubtedly causes the asymmetry phenomenon. Previous studies have proved that the imbalance of model domain test samples may hamper the effectiveness of fault localization techniques. In practice, it is difficult to create failing test cases in input domain. With the rapid development of deep learning techniques, Generative Adversarial Networks (GAN) have achieved promising results in many fields, which may bring a new perspective for augmenting model domain failing test samples. Thus, we try to provide a new method named MAG: Model-domain failing test Augmentation with Generative Adversarial Networks. MAG first constructs a Generative Adversarial Network, and then generates vectors with the abstract representation of execution information as input. At last, MAG utilizes influential global and local contexts to enhance the generated vectors, and to formulate model domain test samples with failing labels. MAG augments model domain failing test samples. Different from traditional methods of generating test cases directly from input domain, MAG seeks to augment failing test samples of model domain, which could improve the effectiveness of existing fault localization techniques. Experimental results illustrate that MAG significantly improves the effectiveness of 13 typical fault localization approaches and is better than two representative data optimization techniques.

Effective Isolation of Fault-Correlated Variables via Statistical and Mutation Analysis

It is a widely-adopted strategy for developers to monitor the values of program variables when debugging in practice. In particular, developers often set breakpoints at specific locations or execute the program step by step in the debugging mode to inspect if abnormal values or status will be observed for concerned variables. Such a practical debugging strategy can facilitate developers in understanding and localizing the target fault. This study aims to identify suspicious program variables of a given fault (i.e., denoted as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fault-correlated variables</i> ) automatically, thus facilitating the debugging activities for developers. To the best of our knowledge, this is the finest granularity in fault localization (FL) so far, which can address the limitations of being coarse-grained as faced by existing FL techniques. However, isolating fault-correlated variables precisely is challenging since there are usually substantially different variables used or defined in a program, and plenty of them are in the same basic block which cannot be well discriminated from each other since they will be either executed or not against the given test suite. To address such challenges, this study presents IsoVar, a two-phase model to isolate fault-correlated variables. Specifically, IsoVar first performs statistical analysis based on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">variable execution matrices</i> , which is a novel concept proposed in this study, to identify a set of suspicious variables. It then observes the impacts of those variables on the program dynamically after applying subtle mutations at the bytecode level, to further isolate fault-correlated variables. Extensive experiments on Defects4J and Bears demonstrate that IsoVar can outperform state-of-the-art techniques significantly ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$13.0\%$</tex-math></inline-formula> for MAP and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$19.3\%$</tex-math></inline-formula> for MRR). More importantly, we incorporated IsoVar into 11 existing FL techniques as well as 14 automated program repair techniques, and found that IsoVar can significantly boost their performance.

An Empirical Study of Boosting Spectrum-Based Fault Localization via PageRank

Manual debugging is notoriously tedious and time-consuming. Therefore, various automated fault localization techniques have been proposed to help with manual debugging. Among the existing fault localization techniques, spectrum-based fault localization (SBFL) is one of the most widely studied techniques due to being lightweight. The focus of the existing SBFL techniques is to consider how to differentiate program entities (i.e., one dimension in program spectra); indeed, this focus is aligned with the ultimate goal of finding the faulty lines of code. Our key insight is to enhance the existing SBFL techniques by additionally considering how to differentiate tests (i.e., the other dimension in program spectra), which, to the best of our knowledge, has not been studied in prior work. We present our basic approach, PRFL, a lightweight technique that boosts SBFL by differentiating tests using PageRank algorithm. Specifically, given the original program spectrum information, PRFL uses PageRank to recompute the spectrum by considering the contributions of different tests. Next, traditional SBFL techniques are applied on the recomputed spectrum to achieve more effective fault localization. On top of PRFL, we explore PRFL+ and PRFL <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MA</sub> , two variants which extend PRFL by optimizing its components and integrating Method-level Aggregation technique, respectively. Though being simple and lightweight, PRFL has been demonstrated to outperform state-of-the-art SBFL techniques significantly (e.g., ranking 39.2% / 82.3% more real/artificial faults at Top-1 compared with the most effective traditional SBFL technique) with low overhead (e.g., around 6 minutes average extra overhead on real faults) on 395 real faults from 6 Defects4J projects and 96925 artificial (i.e., mutation) faults from 240 GitHub projects. To further validate PRFL's effectiveness, we compare PRFL with multiple recent proposed fault localization techniques (e.g., Multric, Metallaxis and MBFL-hybrid-avg), and the experimental results show that PRFL outperforms them as well. Furthermore, we study the performance of PRFL <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MA</sub> , and the experimental results present it can locate 137 real faults (73.4% / 24.5% more compared with the most effective SBFL/PRFL technique) and 35058 artificial faults (159.6% / 28.1% more than SBFL/PRFL technique) at Top-1. At last, we study the generalizability of PRFL on another benchmark, Bugs.jar, and the result shows PRFL can help locate around 30 percent more faults at Top 1.

MuSim: Mutation-based Fault Localization Using Test Case Proximity

Fault localization techniques aim to localize faulty statements using the information gathered from both passed and failed test cases. We present a mutation-based fault localization technique called MuSim. MuSim identifies the faulty statement based on its computed proximity to different mutants. We study the performance of MuSim by using four different similarity metrics. To satisfactorily measure the effectiveness of our proposed approach, we present a new evaluation metric called Mut_Score. Based on this metric, on an average, MuSim is 33.21% more effective than existing fault localization techniques such as DStar, Tarantula, Crosstab, Ochiai.

Software fault localization using BP neural network based on function and branch coverage

Software failure is inevitable with the increase in scale and complexity of the software. Existing fault localization techniques based on neural networks take statement coverage information and test case execution results into account to train the network. In this paper, we propose an effective approach for fault localization based on back-propagation neural network which utilizes branch and function coverage information along with test case execution results to train the network. We investigated our approach using Siemens suite. Our experimental result shows that our proposed approach performs on average 23.50–44.27% better than existing fault localization techniques.

A Community-Based Fault Isolation Approach for Effective Simultaneous Localization of Faults

During program testing, software programs may be discovered to contain multiple faults. Multiple faults in a program may reduce the effectiveness of the existing fault localization techniques due to the complex relationship between faults and failures in the presence of multiple faults. In an ideal case, faults are isolated into fault-focused clusters, each targeting a single fault for developers to localize them simultaneously in parallel. However, the relationship between faults and failures is not easily identified and depends solely on the accuracy of clustering, such as existing clustering algorithms are not able to isolate failed tests to their causative faults effectively which hinder localization effectiveness. This paper proposes a new approach that makes use of a divisive network community clustering algorithm to isolate faults into separate fault-focused communities that target a single fault each. A community weighting and a selection mechanism that aids in prioritizing highly important fault-focused communities to the available developers to debug the faults simultaneously in parallel is also proposed. The approach is evaluated on eight subject programs ranging from medium-sized to large-sized programs (tcas, replace, gzip, sed, flex, grep, make, and ant). Overall, 540 multiple-fault versions of these programs were generated with 2-5 faulty versions. The experimental results have demonstrated that the proposed approach performs significantly better in terms of localization effectiveness in comparison with two other parallel debugging approaches for locating multiple faults in parallel.

Simultaneous Localization of Software Faults Based on Complex Network Theory

Effective debugging is necessary for producing high quality and reliable software. Fault localization plays a vital role in the debugging process. However, fault localization is the most tedious and expensive activity in program debugging, as such, effective fault localization techniques that can identify the exact location of faults is eminent. Despite various fault localization techniques proposed, their application in multiple-fault programs is limited. The presence of multiple faults in a program reduces the efficacy of the existing fault localization techniques to locate faults effectively due to fault interference. Moreover, most of these techniques are unable to localize faults simultaneously. This has led researchers to adopt alternative approaches, such as one-fault-at-a-time debugging and parallel debugging. In this paper, we propose a novel fault localization technique based on complex network theory (FLCN) to improve localization effectiveness in programs with single-fault and multiple-fault and to aid developers to localize multiple faults simultaneously in a single diagnosis rank list. The proposed technique ranks faulty statements based on their behavioral abnormalities and distance between faulty statements in both passed and failed test executions. Two graph-based centrality measures are adopted for fault diagnosis, namely, degree centrality and closeness centrality and a new ranking formula is proposed. FLCN is evaluated across 14 subjects with both single-fault and multiple-fault programs. Our experimental results show that FLCN is more effective at locating faults when compared with existing state-of-the-art fault localization techniques with improvement of fault localization effectiveness in both single-fault and multiple-fault programs.

Diversity maximization speedup for localizing faults in single-fault and multi-fault programs

Fault localization is useful for reducing debugging effort. Such techniques require test cases with oracles, which can determine whether a program behaves correctly for every test input. Although most fault localization techniques can localize faults relatively accurately even with a small number of test cases, choosing the right test cases and creating oracles for them are not easy. Test oracle creation is expensive because it can take much manual labeling effort (i.e., effort needed to decide whether the test cases pass or fail). Given a number of test cases to be executed, it is challenging to minimize the number of test cases requiring manual labeling and in the meantime achieve good fault localization accuracy. To address this challenge, this paper presents a novel test case selection strategy based on Diversity Maximization Speedup (Dms). Dms orders a set of unlabeled test cases in a way that maximizes the effectiveness of a fault localization technique. Developers are only expected to label a much smaller number of test cases along this ordering to achieve good fault localization results. We evaluate the performance of Dms on 2 different types of programs, single-fault and multi-fault programs. Our experiments with 411 faults from the Software-artifact Infrastructure Repository show (1) that Dms can help existing fault localization techniques to achieve comparable accuracy with on average 67 and 6 % fewer labeled test cases than previously best test case prioritization techniques for single-fault and multi-fault programs, and (2) that given a labeling budget (i.e., a fixed number of labeled test cases), Dms can help existing fault localization techniques reduce their debugging cost (in terms of the amount of code needed to be inspected to locate faults). We conduct hypothesis test and show that the saving of the debugging cost we achieve for the real C programs are statistically significant.

Concurrency bug localization using shared memory access pairs

Non-determinism in concurrent programs makes their debugging much more challenging than that in sequential programs. To mitigate such difficulties, we propose a new technique to automatically locate buggy shared memory accesses that triggered concurrency bugs. Compared to existing fault localization techniques that are based on empirical statistical approaches, this technique has two advantages. First, as long as enough successful runs of a concurrent program are collected, the proposed technique can locate buggy memory accesses to the shared data even with only one single failed run captured, as opposed to the need of capturing multiple failed runs in other statistical approaches. Second, the proposed technique is more precise because it considers memory accesses in those failed runs that terminate prematurely.

BPELDebugger: An effective BPEL-specific fault localization framework

ContextBusiness Process Execution Language (BPEL) is a widely recognized executable service composition language, which is significantly different from typical programming languages in both syntax and semantics, and especially shorter in program scale. How to effectively locate faults in BPEL programs is an open and challenging problem. ObjectiveIn this paper, we propose a fault localization framework for BPEL programs. MethodBased on BPEL program characteristics, we propose two fault localization guidelines to locate the integration and interaction faults in BPEL programs. Our framework formulates the BPEL fault localization problem using the popular fault localization problem settings, and synthesizes BPEL-specific fault localization techniques by reuse of existing fault localization formulas. We use two realistic BPEL programs and three existing fault localization formulas to evaluate the feasibility and effectiveness of the proposed fault localization framework and guidelines. ResultExperiment results show that faults can be located with the fewest code examining efforts. That is, the fault-relevant basic block is assigned the highest suspiciousness score by our fault localization method. The experiment results also show that with the use of the proposed fault localization guidelines, the code examining efforts to locate faults are extraordinarily reduced. ConclusionWe conclude that the proposed framework is feasible in synthesizing effective fault localization techniques, and our fault localization guidelines are very effective to enhance existing fault localization techniques in locating faults in BPEL programs.

Focus section on program debugging

Focus section on program debugging