Spectrum-based Fault Localization Techniques Research Articles

Fault localization in DSLTrans model transformations by combining symbolic execution and spectrum-based analysis

The verification of model transformations is important for realizing robust model-driven engineering technologies and quality-assured automation. Many approaches for checking properties of model transformations have been proposed. Most of them have focused on the effective and efficient detection of property violations by contract checking. However, there remains the fault localization step between identifying a failing contract for a transformation based on verification feedback and precisely identifying the faulty rules. While there exist fault localization approaches in the model transformation verification literature, these require the creation and maintenance of test cases, which imposes an additional burden on the developer. In this paper, we combine transformation verification based on symbolic execution with spectrum-based fault localization techniques for identifying the faulty rules in DSLTrans model transformations. This fault localization approach operates on the path condition output of symbolic transformation checkers instead of requiring a set of test input models. In particular, we introduce a workflow for running the symbolic execution of a model transformation, evaluating the defined contracts for satisfaction, and computing different measures for tracking the faulty rules. We evaluate the effectiveness of spectrum-based analysis techniques for tracking faulty rules and compare our approach to previous works. We evaluate our technique by introducing known mutations into five model transformations. Our results show that the best spectrum-based analysis techniques allow for effective fault localization, showing an average EXAM score below 0.30 (less than 30% of the transformation needs to be inspected). These techniques are also able to locate the faulty rule in the top-three ranked rules in 70% of all cases. The impact of the model transformation, the type of mutation and the type of contract on the results is discussed. Finally, we also investigate the cases where the technique does not work properly, including discussion of a potential pre-check to estimate the prospects of the technique for a certain transformation.

FEDRak: Federated Learning-Based Symmetric Code Statement Ranking Model for Software Fault Forecasting

Software Fault Forecasting (SFF) pertains to timely identifying sections in software projects that are prone to faults and may result in significant development expenses. Deep learning models have become widespread in software fault monitoring and management, and these models rely on the design metrics and the code pattern features for classifying the code as erroneous or safe. The proposed model works based on the collective formulation of the fault localization model, which acquires the model-specific metadata for building a global model that would perform software fault forecasting globally. The proposed model works by ranking the suspicious code blocks based on the symmetry of the semantic features of the erroneous code and the implementation code. The feature selection and scaling process is initially performed to precisely identify the features contributing to fault forecasting. The data extraction that is portrayed as the intermediate phase would assist in focusing on the code statements and ranking them based on the impact of the fault. A fine-tuned spectrum-based fault localization technique is used in ranking the statements. The FEDRak model facilitates ongoing adaptation in instances where there are adjustments in the feature contribution of data over time. The federated learning model would update the feature weights of the global model based on the weights synchronized by locally built fault forecasting approaches. FEDRak is statistically analyzed in relation to other contemporary techniques in fault localization in terms of metrics like sensitivity, specificity, accuracy, F1-score, and ROC curves. The proposed model’s performances are analyzed in terms of local and global models.

Multiple fault localization based on ant colony algorithm via genetic operation

Debugging programs involves a considerable amount of time and resources spent on identifying the source of errors. Spectrum-based fault localization techniques are becoming increasingly popular due to their quick and effective performance when dealing with programs that have a single fault. However, these methods do not take into account the reality that most programs tend to have multiple faults. To address the above challenges, we propose an automatic approach using ant colony algorithm via genetic operation for multiple fault localization (ACOMFL). Our approach first constructs the search graph by the coverage information. Secondly, it designs the fitness function to select the best paths and dynamically adjusts the pheromone updating. Finally, it computes the suspiciousness ranking of each statement by the ant path information in the optimal path set. Four open-source benchmarks are utilized to validate the performance of the ACOMFL approach. The experimental results show ACOMFL achieves better results for multi-fault programs in comparison with six spectrum-based fault localization approaches and a genetic algorithm-based multiple fault localization approach. The ACOMFL is effective at locating multi-fault, serving as a helpful method for locating faults.

The MADAG Strategy for Fault Location Techniques

Spectrum-based fault localization (SBFL), which utilizes spectrum information of test cases to calculate the suspiciousness of each statement in a program, can reduce developers’ effort. However, applying redundant test cases from a test suite to fault localization incurs a heavy burden, especially in a restricted resource environment, and it is expensive and infeasible to inspect the results of each test input. Prioritizing/selecting appropriate test cases is important to enable the practical application of the SBFL technique. In addition, we must ensure that applying the selected tests to SBFL can achieve approximately the effectiveness of fault localization with whole tests. This paper presents a test case prioritization/selection strategy, namely the Minimal Aggregate of the Diversity of All Groups (MADAG). The MADAG strategy prioritizes/selects test cases using information on the diversity of the execution trace of each test case. We implemented and applied the MADAG strategy to 233 faulty versions of the Siemens and UNIX programs from the Software-artifact Infrastructure Repository. The experiments show that (1) the MADAG strategy uses only 8.99 and 14.27 test cases, with an average of 18, from the Siemens and UNIX test suites, respectively, and the SBFL technique has approximate effectiveness for fault localization on all test cases and outperforms the previous best test case prioritization method; (2) we verify that applying whole tests from the test suite may not achieve the better effectiveness in fault localization compared with the tests selected by MADAG strategy.

CRMF: A fault localization approach based on class reduction and method call frequency

AbstractIdentifying the location of faults in real‐world programs is one of the costly processes during software debugging. To reduce the debugging effort, various fault localization techniques have been proposed in recent years. Spectrum‐based fault localization (SBFL) is one kind of widely investigated fault localization technique. Most SBFL techniques first calculate the suspiciousness of program elements (such as statements, methods) to be faulty using the coverage information and execution results of tests. Then a rank list of program elements is generated according to their suspiciousness. However, some SBFL techniques only consider the binary coverage information (i.e., whether the program element is covered) but ignore some of the tests' running behaviors, such as the execution frequency when faults occur in the iteration entities or loop bodies, which are more likely to be faulty followed the propagation‐infection‐execution model. The execution frequency based techniques only replace the feature items of the existing formula limiting their effectiveness in fault localization. In this article, we propose a fault localization technique, class reduction and method call frequency (CRMF), which utilizes mutation analysis and information retrieval techniques. In particular, CRMF first uses mutation analysis to identify and reduce the classes, in which the program elements with a low probability of being faulty. Then we propose a new suspiciousness formula that applies information retrieval and considers method call frequency. To evaluate the effectiveness of CRMF, we conduct empirical studies on 264 real‐world programs from the Defects4J benchmark. Final results show that CRMF outperforms the statement frequency based technique FLSF and SBFL techniques (i.e., Ochiai, OP2, Tarantula, and Dstar) in both single‐fault programs and multiple‐fault programs. Specifically, CRMF can rank 29, 74, and 112 faults at the top 1, 3, 5 ranks and achieve a higher mean reciprocal rank for single‐fault programs and multiple‐fault programs. Finally, we discuss the essence of CRMF and analyze its effectiveness on multi‐fault programs in detail.

Real world projects, real faults: evaluating spectrum based fault localization techniques on Python projects

Spectrum Based Fault Localization (SBFL) is a statistical approach to identify faulty code within a program given a program spectra (i.e., records of program elements executed by passing and failing test cases). Several SBFL techniques have been proposed over the years, but most evaluations of those techniques were done only on Java and C programs, and frequently involve artificial faults. Considering the current popularity of Python, indicated by the results of the Stack Overflow survey among developers in 2020, it becomes increasingly important to understand how SBFL techniques perform on Python projects. However, this remains an understudied topic. In this work, our objective is to analyze the effectiveness of popular SBFL techniques in real-world Python projects. We also aim to compare our observed performance on Python to previously-reported performance on Java. Using the recently-built bug benchmark BugsInPy as our fault dataset, we apply five popular SBFL techniques (Tarantula, Ochiai, OP, Barinel, and DStar) and analyze their performances. We subsequently compare our results with results from Java and C projects reported in earlier related works. We find that 1) the real faults in BugsInPy are harder to identify using SBFL techniques compared to the real faults in Defects4J, indicated by the lower performance of the evaluated SBFL techniques on BugsInPy; 2) older techniques such as Tarantula, Barinel, and Ochiai consistently outperform newer techniques (i.e., OP and DStar) in a variety of metrics and debugging scenarios; 3) claims in preceding studies done on artificial faults in C and Java (such as “OP outperforms Tarantula”) do not hold on Python real faults; 4) lower-performing techniques can outperform higher-performing techniques in some cases, emphasizing the potential benefit of combining SBFL techniques. Our results yield insight into how popular SBFL techniques perform in real Python faults and emphasize the importance of conducting SBFL evaluations on real faults.

Fault localization using function call frequencies

In traditional Spectrum-Based Fault Localization (SBFL), hit-based spectrum is used to estimate a program element’s suspiciousness to contain a fault, i.e., only the binary information is used if the code element was executed by the test case or not. Count-based spectra can potentially improve the localization effectiveness due to the number of executions also being available. In this work, we use function-level granularity and define count-based spectra which use function call frequencies. We investigate the naïve approach, which simply counts the function call instances. We also define a novel method which is based on counting the different function call contexts, i.e., the frequency of the investigated function occurring in unique call stack instances during test execution. The basic intuition is that if a function is called in many different contexts during a failing test case, it will be more probable to be accountable for the fault. We empirically evaluated the fault localization capability of different variations of the approach and compared them to 9 traditional SBFL techniques using the Defects4J benchmark. We show that: (i) naïve counts result in worse rank positions than the hit-based approach, but (ii) unique counts produce better rank positions with some of the algorithm variants.

A Preliminary Investigation on the Performance of SBFL Techniques and Distance Metrics in Parallel Fault Localization

To locate multiple bugs in parallel, one common practice is to generate fault-focused clusters where failed test cases that are likely caused by the same bug are grouped together. With respect to the fault-focused clustering and fault localization, Spectrum-based fault localization (SBFL) technique and distance metric are two critical impact factors. This article proposes a method to evaluate the parallel fault localization performance of SBFL techniques and distance metrics where the fault-focused information for each failed test case is already given. Case studies are conducted using the proposed method to evaluate five SBFL techniques and five distance metrics on three programs with multiple bugs.

A Survey of Challenges in Spectrum-Based Software Fault Localization

In software debugging, fault localization is the most difficult, expensive, tedious, and time-consuming task, particularly for large-scale software systems. This is due to the fact that it requires significant human participation and it is difficult to automate its sub-tasks. Therefore, there is a high demand for automatic fault localization techniques that can help software engineers effectively find the locations of faults with minimal human intervention. This has led to the proposal of implementing different types of such techniques. However, Spectrum Based Fault Localization (SBFL) is considered amongst the most prominent techniques in this respect due to its efficiency and effectiveness. In SBFL, the probability of each program element (e.g., statement, block, or function) being faulty is calculated based on the results of executing test cases and their corresponding code coverage information. However, SBFL techniques are not yet widely adopted in the industry. The rationale behind this is that they pose a number of issues and their performance is affected by several influential factors. For example, the characteristics of bugs, target programs, test suites, and supporting tools make their effectiveness differ dramatically from one case to another. There are massive studies on SBFL that cover its usage, formulas, performance, etc. So far, no dedicated survey points out comprehensively the issues of SBFL. In this paper, various SBFL challenges and issues have been identified, categorized, and discussed alongside many directions. Also, the paper raises awareness of the works being achieved to address the identified issues and suggests some potential solutions too.

Historical Spectrum Based Fault Localization

Spectrum-based fault localization (SBFL) techniques are widely studied and have been evaluated to be effective in locating faults. Recent studies also showed that developers from industry value automated SBFL techniques. However, their effectiveness is still limited by two main reasons. First, the test coverage information leveraged to construct the spectrum does not reflect the root cause directly. Second, SBFL suffers from the tie issue so that the buggy code entities can not be well differentiated from non-buggy ones. To address these challenges, we propose to leverage the information of version histories in fault localization based on the following two intuitions. First, version histories record how bugs are introduced to software projects and this information reflects the root cause of bugs directly. Second, the evolution histories of code can help differentiate those suspicious code entities ranked in tie by SBFL. Our intuitions are also inspired by the observations on debugging practices from large open source projects and industry. Based on the intuitions, we propose a novel technique HSFL (historical spectrum based fault localization). Specifically, HSFL identifies bug-inducing commits from the version history in the first step. It then constructs historical spectrum (denoted as Histrum) based on bug-inducing commits, which is another dimension of spectrum orthogonal to the coverage based spectrum used in SBFL. HSFL finally ranks the suspicious code elements based on our proposed Histrum and the conventional spectrum. HSFL outperforms the state-of-the-art SBFL techniques significantly on the Defects4J benchmark. Specifically, it locates and ranks the buggy statement at Top-1 for 77.8 percent more bugs as compared with SBFL, and 33.9 percent more bugs at Top-5. Besides, for the metrics MAP and MRR, HSFL achieves an average improvement of 28.3 and 40.8 percent over all bugs, respectively. Moreover, HSFL can also outperform other six families of fault localization techniques, and our proposed Histrum model can be integrated with different families of techniques and boost their performance.

Combi-FL: Neural network and SBFL based fault localization using mutation analysis

In this article, we present a hybrid approach for fault localization (FL). We combine three different domains of software FL techniques to realize an effective fault localizer. Spectrum based fault localization techniques and neural network (NN) based techniques are utilized to determine the similarity of different mutants with the faulty program. Ten prominent FL techniques (six techniques from SBFL family and four methods from NN family) are considered in our proposed approach: Combi-FL. The ranking sequences generated by different FL techniques are combined using learning-to-rank algorithm. In this work, we focus on localization of single-fault programs. We have evaluated our proposed Combi-FL technique over seven program suites comprising twenty-one programs and compared its effectiveness with eight popular FL techniques. Our experimental results show that on an average Combi-FL is 28.42% more effective than existing FL techniques such as DStar, Ochiai, Barinel, Tarantula, CNN-FL, DNN, RBFNN, and BPNN.

An Empirical Study of Boosting Spectrum-Based Fault Localization via PageRank

Manual debugging is notoriously tedious and time-consuming. Therefore, various automated fault localization techniques have been proposed to help with manual debugging. Among the existing fault localization techniques, spectrum-based fault localization (SBFL) is one of the most widely studied techniques due to being lightweight. The focus of the existing SBFL techniques is to consider how to differentiate program entities (i.e., one dimension in program spectra); indeed, this focus is aligned with the ultimate goal of finding the faulty lines of code. Our key insight is to enhance the existing SBFL techniques by additionally considering how to differentiate tests (i.e., the other dimension in program spectra), which, to the best of our knowledge, has not been studied in prior work. We present our basic approach, PRFL, a lightweight technique that boosts SBFL by differentiating tests using PageRank algorithm. Specifically, given the original program spectrum information, PRFL uses PageRank to recompute the spectrum by considering the contributions of different tests. Next, traditional SBFL techniques are applied on the recomputed spectrum to achieve more effective fault localization. On top of PRFL, we explore PRFL+ and PRFL <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MA</sub> , two variants which extend PRFL by optimizing its components and integrating Method-level Aggregation technique, respectively. Though being simple and lightweight, PRFL has been demonstrated to outperform state-of-the-art SBFL techniques significantly (e.g., ranking 39.2% / 82.3% more real/artificial faults at Top-1 compared with the most effective traditional SBFL technique) with low overhead (e.g., around 6 minutes average extra overhead on real faults) on 395 real faults from 6 Defects4J projects and 96925 artificial (i.e., mutation) faults from 240 GitHub projects. To further validate PRFL's effectiveness, we compare PRFL with multiple recent proposed fault localization techniques (e.g., Multric, Metallaxis and MBFL-hybrid-avg), and the experimental results show that PRFL outperforms them as well. Furthermore, we study the performance of PRFL <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MA</sub> , and the experimental results present it can locate 137 real faults (73.4% / 24.5% more compared with the most effective SBFL/PRFL technique) and 35058 artificial faults (159.6% / 28.1% more than SBFL/PRFL technique) at Top-1. At last, we study the generalizability of PRFL on another benchmark, Bugs.jar, and the result shows PRFL can help locate around 30 percent more faults at Top 1.

Spectrum-based multi-fault localization using Chaotic Genetic Algorithm

Abstract Context: In the field of software engineering, the most complex and time consuming activity is fault-finding. Due to increasing size and complexity of software, there is a necessity of automated fault detection tool which can detect fault with minimal human intervention. A programmer spends a lot of time and effort on software fault localization. Various Spectrum Based Fault Localization (SBFL) techniques have already been developed to automate the fault localization in single-fault software. But, there is a scarcity of fault localization technique for multi-fault software. In our study, we have found that pure SBFL is not always sufficient for effective fault localization in multi-fault programs. Objective: To address the above challenge, we propose an automated framework using Chaos-based Genetic Algorithm for Multi-fault Localization (CGAML) based on SBFL technique. Methods: Traditional Genetic Algorithm (GA) sometimes stuck in local optima, and it takes more time to converge. Different chaos mapping functions have been applied to GA for better performance. We have used logistic mapping function to achieve chaotic sequence. The proposed technique CGAML first calculates the suspiciousness score for each program statement and then assigns ranks according to that score. The statements having smaller rank means there is a high probability of the statements to be faulty. Results: Five open-source benchmark programs are tested to evaluate the efficiency of CGAML technique. The experimental results show CGAML gives better results for both single-fault and multi-fault programs in comparison with existing spectrum-based fault localization techniques. Conclusion: E X A M metric is used to compare the performance of our proposed technique with other existing techniques. Smaller E X A M score denotes the higher accuracy of the technique. The proposed framework generates smaller E X A M score in comparison with other existing techniques. We found that, overall CGAML works on an average 8.5% better than GA for both single-fault and multi-fault software.

A Framework for Improving Fault Localization Effectiveness Based on Fuzzy Expert System

Many spectrum-based fault localization (SBFL) techniques have been proposed in order to improve debugging efficiency. These SBFL techniques were designed according to different underlying assumptions and then adopt different fault locator functions to evaluate the likelihood of each statement being faulty, called suspiciousness. So far no single SBFL technique claims that it can outperform all of the others under every scenario. That is, the effectiveness of fault localization may vary considerably by just adopting a single SBFL technique. Due to the aforementioned reasons, this study presents a framework for improving fault localization effectiveness by using a Fuzzy Expert System (FES) to integrate different SBFL techniques. In the presented framework, the outputs of several SBFL techniques will be put into the fuzzification and then transferred to fuzzy input sets. After undergoing the fuzzy inference based on the given fuzzy rules, the fuzzy input sets will be transferred to a fuzzy output set. Finally, the fuzzy set will be transferred to a crisp output (called a weighted suspiciousness value). The code statements will then be ranked according to their weighted suspiciousness values. In other words, no additional instrumentations and analyses on the source code and the test suite are necessary for our approach. Our experiment results indicate that our FES-based framework is effective at combining the SBFL techniques from different equivalent groups and achieves high effectiveness on the nine subject programs. It is also noted that in the literature, most of the approaches that combine multiple SBFL techniques are learning-based and they are suitable for long-term projects with sufficient historical data. Since our approach does not reference historical data for model training, it can be applied to new software projects. Thus, the application scenarios of our approach should be complementary to those of the state-of-the-art learning-based approaches.

FATOC: Bug Isolation Based Multi-Fault Localization by Using OPTICS Clustering

Bug isolation is a popular approach for multi-fault localization (MFL), where all failed test cases are clustered into several groups, and then the failed test cases in each group combined with all passed test cases are used to localize only a single fault. However, existing clustering algorithms cannot always obtain completely correct clustering results, which is a potential threat for bug isolation based MFL approaches. To address this issue, we first analyze the influence of the accuracy of the clustering on the performance of MFL, and the results of a controlled study indicate that using the clustering algorithm with the highest accuracy can achieve the best performance of MFL. Moreover, previous studies on clustering algorithms also show that the elements in a higher density cluster have a higher similarity. Based on the above motivation, we propose a novel approach FATOC (One-Fault-at-a-Time via OPTICS Clustering). In particular, FATOC first leverages the OPTICS (Ordering Points to Identify the Clustering Structure) clustering algorithm to group failed test cases, and then identifies a cluster with the highest density. OPTICS clustering is a density-based clustering algorithm, which can reduce the misgrouping and calculate a density value for each cluster. Such a density value of each cluster is helpful for finding a cluster with the highest clustering effectiveness. FATOC then combines the failed test cases in this cluster with all passed test cases to localize a single-fault through the traditional spectrum-based fault localization (SBFL) formula. After this fault is localized and fixed, FATOC will use the same method to localize the next single-fault, until all the test cases are passed. Our evaluation results show that FATOC can significantly outperform the traditional SBFL technique and a state-of-the-art MFL approach MSeer on 804 multi-faulty versions from nine real-world programs. Specifically, FATOC’s performance is 10.32% higher than that of traditional SBFL when using Ochiai formula in terms of metric A-EXAM. Besides, the results also indicate that, when checking 1%, 3% and 5% statements of all subject programs, FATOC can locate 36.91%, 48.50% and 66.93% of all faults respectively, which is also better than the traditional SBFL and the MFL approach MSeer.

Improving Fault-Localization Accuracy by Referencing Debugging History to Alleviate Structure Bias in Code Suspiciousness

Spectrum-based fault localization (SBFL) techniques can automatically localize software faults. They employ the program spectrum, such as code coverage profile with test verdicts, to rank the program entities based on their code suspiciousness. In the past decades, researchers have proposed many approaches to optimize these techniques; however, the program structure, which can influence their performance, is not taken into consideration in developing and improving these techniques. In this article, we identify and analyze the effect of the program structure on the application of SBFL techniques. We observe that some specific program structures may introduce structure bias to code suspiciousness and negatively influence the output of SBFL techniques. To mitigate these effects and improve the performance of fault localization, we propose Delta4Ts, a structure-aware technique. Delta4Ts references debugging history to alleviate the impact of structure bias in the calculation of code suspiciousness. It reasons from the observable suspicious value towards the desired suspicious value and the impact of structure bias. To evaluate Delta4Ts under practical constraints, we conduct a controlled experiment using nine widely-studied SBFL formulae on 12 C programs and 6 Java programs. The experiment results show that Delta4Ts can significantly improve the accuracy of the studied SBFL formulae by an average of 34.8% on 12 C programs and 30.6% on 6 Java programs, and improve more on subject programs associated with more history versions or having larger code sizes.

HMER: A Hybrid Mutation Execution Reduction approach for Mutation-based Fault Localization

Identifying the location of faults in programs has been recognized as one of the most manually and time cost activities during software debugging process. Fault localization techniques, which seek to identify faulty program statements as quickly as possible, can assist developers in alleviating the time and manual cost of software debugging. Mutation-based fault localization(MBFL) has a promising fault localization accuracy, but suffered from huge mutation execution cost. To reduce the cost of MBFL, we propose a Hybrid Mutation Execution Reduction(HMER) approach in this paper. HMER consists of two steps: Weighted Statement-Oriented Mutant Sampling(WSOME) and Dynamic Mutation Execution Strategy(DMES). In the first step, we employ Spectrum-Based Fault Localization(SBFL) techniques to calculate the suspiciousness value of statements, and guarantee that the mutants generated from statements with higher suspiciousness value will have more chance to be remained in the sampling process. Next, a dynamic mutation execution strategy is used to execute the reduced mutant set on test suite to avoid worthless execution. Empirical results on 130 versions from 9 subject programs show that HMER can reduce 74.5%-93.4% mutation execution cost while keeping almost the same fault localization accuracy with the original MBFL. A further Wilcoxonsigned−ranktest indicates that when employing HMER strategy in MBFL, the fault localization accuracy has no statistically significant difference in most cases compared with the original MBFL without any reduction techniques.

FCCI: A fuzzy expert system for identifying coincidental correct test cases

Spectrum-based fault localization (SBFL) is a promising approach to reduce the cost of program debugging and there has been a large body of research on introducing effective SBFL techniques. However, performance of these techniques can be adversely affected by the existence of coincidental correct (CC) test cases in the test suites. Such test cases execute the faulty statement but do not cause failures. Given that coincidental correctness is prevalent, it is necessary to precisely identify CC test cases and eliminate their effects from test suites. To do so, in this paper, we propose several important factors to identify CC test cases and model the CC identification process as a decision making system by constructing a fuzzy expert system and proposing a novel fuzzy CC identification method, namely FCCI. FCCI estimates the CC likelihood of passed test cases using the designed fuzzy rules, which effectively correlate the proposed CC identification factors. We evaluated FCCI by conducting extensive experiments on 17 popular and open source subject programs ranging from small- to large-scale containing both artificial and real faults. The experimental results indicate that FCCI successfully improves the accuracy of the CC identification as well as the accuracy of the representative SBFL techniques.

Spectrum-based Fault Localization Techniques Application on Multiple-Fault Programs: A Review

Software fault localization is one of the most tedious and costly activities in program debugging in the endeavor to identify faults locations in a software program. In this paper, the studies that used spectrum-based fault localization (SBFL) techniques that makes use of different multiple fault localization debugging methods such as one-bug-at-a-time (OBA) debugging, parallel debugging, and simultaneous debugging in localizing multiple faults are classified and critically analyzed in order to extensively discuss the current research trends, issues, and challenges in this field of study. The outcome strongly shows that there is a high utilization of OBA debugging method, poor fault isolation accuracy, and dominant use of artificial faults that limit the existing techniques applicability in the software industry.

A Theoretical and Empirical Analysis of Program Spectra Diagnosability

Current metrics for assessing the adequacy of a test-suite plainly focus on the number of components (be it lines, branches, paths) covered by the suite, but do not explicitly check how the tests actually exercise these components and whether they provide enough information so that spectrum-based fault localization techniques can perform accurate fault isolation. We propose a metric, called DDU, aimed at complementing adequacy measurements by quantifying a test-suite's diagnosability, i.e., the effectiveness of applying spectrum-based fault localization to pinpoint faults in the code in the event of test failures. Our aim is to increase the value generated by creating thorough test-suites, so they are not only regarded as error detection mechanisms but also as effective diagnostic aids that help widely-used fault-localization techniques to accurately pinpoint the location of bugs in the system. We have performed a topology-based simulation of thousands of spectra and have found that DDU can effectively establish an upper bound on the effort to diagnose faults. Furthermore, our empirical experiments using the Defects4J dataset show that optimizing a test suite with respect to DDU yields a 34 percent gain in spectrum-based fault localization report accuracy when compared to the standard branch-coverage metric.