Coverage-based Fault Localization Research Articles

A Passed Test Case Cluster Method to Improve Fault Localization

Coverage-Based Fault Localization (CBFL) techniques are based on the conjecture that the program elements executed by more failed test cases and less passed test cases have more chance to be faulty. Coincidental Correct (CC) test case is one of the negative impacts on CBFL, for the reason that the CC test cases execute the faulty element but not propagate the faulty status to the final output. To alleviate the negative impact of CC test cases, in this paper, we propose a cluster-based technique to identify CC test cases from the passed test suite. To evaluate the effectiveness of our method, we conduct empirical studies on 102 versions from six programs. The experimental results show that, when using our method, it can accurately recognize CC test cases, where the precision and recall rate are both higher than 85%. A further study shows that, after removing identified CC test cases, the fault localization accuracy of SBFL can be improved apparently.

A weighted fuzzy classification approach to identify and manipulate coincidental correct test cases for fault localization

Identifying the location of faults effectively and accurately is highly important in the debugging process of software engineering. Coverage-based Fault Localization (CBFL) has been widely studied that can alleviate the effort of developers to find the faults position using the execution information of test cases. Coincidental Correct (CC) test cases are the specific test cases that execute the faulty statements but with a correct output, which have been illustrated with a negative effect on the accuracy of CBFL. In this paper, we propose a weighted fuzzy classification approach to identify CC test cases and three fuzzy strategies are suggested to manipulate CC test cases for CBFL. Firstly, we present a simple but efficient approach to identify some CC test cases for single fault programs, which provide labeled samples that enable the application of supervised classification algorithms for CC identification. Then, a Fuzzy Weighted K-Nearest Neighbor (FW-KNN) algorithm is proposed to classify potential CC from the passed test cases, in which a ‘weighted’ similarity measure and a “weighted” CC probability computation are presented. Finally, three fuzzy CC test cases manipulation strategies are presented to mitigate the impact of CC test cases in CBFL. Various empirical studies are conducted on 190 faulty versions of 12 programs to investigate the impact of “weighted” and “fuzzy” methods for CC identification by the comparison of the effectiveness and efficiency between FW-KNN and three popular cluster and classification techniques. The results indicate that the proposed FW-KNN has higher accuracy and lower time cost. The Precision, Recall and False Positive Rate of FW-KNN is 96.47%, 83.40% and 2.85%, respectively. Besides, by utilizing code block coverage, the time cost can be reduced by 72.97% in average compared to statement coverage. The experimental results also indicate that the fault localization accuracy of CBFL can be improved by the proposed CC test cases manipulation strategies.

A Test-Suite Reduction Approach for Improving Path-Based Fault-Localization Effectiveness

Path-based fault-localization technique is proved to be more effective than coverage-based fault localization because of analyzing the semantics of a program. Most previous work focused on investigating coverage-based test-suite reduction approach for fault localization. But it is not suitable for path-based fault localization, because it only analyzes coverage information, and does not take the execution path information of test cases into account. In this paper, we propose a test-suite reduction approach for path-based fault localization. This approach analyzes the concrete execution path of test cases. The experimental results show that the proposed method works well in improving the effectiveness of path-based fault localization.

Fault density, fault types, and spectra-based fault localization

This paper presents multiple empirical experiments that investigate the impact of fault quantity and fault type on statistical, coverage-based fault localization techniques and fault-localization interference. Fault-localization interference is a phenomenon revealed in earlier studies of coverage-based fault localization that causes faults to obstruct, or interfere, with other faults' ability to be localized. Previously, it had been asserted that a fault-localization technique's effectiveness was negatively correlated to the quantity of faults in the program. To investigate these beliefs, we conducted an experiment on six programs consisting of more than 72,000 multiple-fault versions. Our data suggests that the impact of multiple faults exerts a significant, but slight influence on fault-localization effectiveness. In addition, faults were categorized according to four existing fault-taxonomies and found no correlation between fault type and fault-localization interference. In general, even in the presence of many faults, at least one fault was found by fault localization with similar effectiveness. Additionally, our data exhibits that fault-localization interference is prevalent and exerts a meaningful influence that may cause a fault's localizability to vary greatly. Because almost all real-world software contains multiple faults, these results affect the practical use and understanding of statistical fault-localization techniques.

Locating Software Faults Based on Control Flow and Data Dependence

Debugging software is a difficult and time-consuming work. Fault localization techniques are becoming extremely important. Coverage Based Fault Localization (CBFL) is very commonly used in fault location technique. Tarantula is a typical one. It uses the coverage statistics of failed execution paths and passed execution paths to calculate the suspiciousness in the software. However, since this technique ignores the data dependency, it is hard to find the bugs which are not in the suspicious code area but have data dependence with it. In order to improve the efficiency of fault locating, we combine control flow coverage information and data dependence from program slicing. We validate our approach experimentally using Siemens benchmark programs. The experimental results show that our approach is more effective than Tarantula.

A coverage and slicing dependencies analysis for seeking software security defects.

Software security defects have a serious impact on the software quality and reliability. It is a major hidden danger for the operation of a system that a software system has some security flaws. When the scale of the software increases, its vulnerability has becoming much more difficult to find out. Once these vulnerabilities are exploited, it may lead to great loss. In this situation, the concept of Software Assurance is carried out by some experts. And the automated fault localization technique is a part of the research of Software Assurance. Currently, automated fault localization method includes coverage based fault localization (CBFL) and program slicing. Both of the methods have their own location advantages and defects. In this paper, we have put forward a new method, named Reverse Data Dependence Analysis Model, which integrates the two methods by analyzing the program structure. On this basis, we finally proposed a new automated fault localization method. This method not only is automation lossless but also changes the basic location unit into single sentence, which makes the location effect more accurate. Through several experiments, we proved that our method is more effective. Furthermore, we analyzed the effectiveness among these existing methods and different faults.

A FAULT LOCALIZATION FRAMEWORK TO ALLEVIATE THE IMPACT OF EXECUTION SIMILARITY

Coverage-based fault localization (CBFL) techniques contrast the execution spectra of a program entity to assess the extent of how much a program entity is being related to faults. However, different test cases may result in similar executions, which further make the execution spectra of program entities be indistinguishable among similar executions. As a consequence, most of the current CBFL techniques are impacted by the noise of indistinguishable spectra. To alleviate the impact of execution similarity and improve the effectiveness of CBFL techniques, we propose a general fault localization framework. This framework is general to current execution spectra based CBFL techniques, which could synthesize a fault localization technique based on a given base technique. To synthesize the new technique, we use the concept of coverage vector to model execution spectra and capture the execution similarity, then reduce the impact of execution similarity by counting distinct coverage vectors, and finally assess the suspiciousness of basic blocks being related to faults with the spectra of distinct coverage vectors. We adopt four representative fault localization techniques as base techniques, use seven Siemens programs and three median-sized real-life UNIX utility programs as subject programs, to conduct an experimental study on the effectiveness of our framework. The empirical evaluation shows that our framework can effectively alleviate the impact of execution similarity and generate more effective fault localization techniques based on existing ones.

A CLUSTERING-BASED STRATEGY TO IDENTIFY COINCIDENTAL CORRECTNESS IN FAULT LOCALIZATION

Coverage-based fault localization techniques leverage the coverage information to identify the faulty elements of a program. However, these techniques can be adversely affected by coincidental correctness, which occurs when the defect is executed but no failure is revealed. In this paper, we propose a clustering-based strategy to identify coincidental correctness in fault localization. The insight behind this strategy is that tests in the same cluster have similar behaviors. Thus a passed test in a cluster with many failed tests is highly possible to be coincidentally correct because it has the potential to execute the faulty elements as those failed ones do. We evaluated this technique from two aspects: the ability to identify coincidental correctness and the effectiveness to improve fault localization. The experimental results show that our strategy can alleviate the coincidental correctness problem and improve the effectiveness of fault localization.

Mining Associations to Improve the Effectiveness of Fault Localization

Coverage-based fault localization(CBFL) techniques find the fault-related positions in programs by comparing the execution statistics of passed executions and failed executions have been proven to be efficient by several empirical studies.However,these techniques assess the suspiciousness of program entities individually,whereas the individual coverage information cannot reflect the complicated control-and data-dependency relationships,and thus oversimplify the execution spectra.In this paper,we first use motivating examples to show the impact of the cause-effect relationship on the effectiveness of CBFL.Second,we propose the rules of program failures and design the execution analysis model based on the coverage of different program execution spectrum.By computing the frequency items for statements with high suspiciousness,we also bring out the coverage vector to mine fault-prone statements.The controlled experiments based on the SIR benchmarks indicate that our technique is promising.

Context-Aware Fault Localization via Control Flow Analysis

Coverage-based fault localization techniques are effective to support program debugging. However, these techniques assess the suspiciousness of program entities individually. Such calculation oversimplifies executions and cannot reflect execution contexts. In this paper, we use control flow paths to analyze the execution context, quantify edge profiles to assess how each block contributes to failures and propose the context-aware fault localization approach FP. We use the edge profile to represent the passed and failed executions, calculate the coverage statistics and edge suspiciousness scores, and then by contrasting edge suspiciousness scores of blocks covered by a failed execution, we propose fault proneness to evaluate how each block contributes to the failure. At last, we take the sum of fault proneness as the suspiciousness to assess the probability of containing faults. We construct controlled experiments to compare our technique with a representative technique. The findings are as follows. 1) the FP technique performs well in locating faults if the infected state propagation is complex, 2) but when the fault is easy to locate, the FP technique may be overly complicated, 3) the integration of the two techniques are more effective than any of them.