Program Entities Research Articles

Ranking co-change candidates suggested by FLeCCS using programmer sensitivity

Identifying co-change candidates when making changes to a particular program entity is an important software engineering task which is known as change impact analysis. A number of techniques for detecting co-change candidates currently exist. Most of these techniques work considering file level, class level, or method level granularity. FLeCCS is a recently introduced technique that suggests co-change candidates considering even fragment level granularity. FLeCCS ranks its co-change candidates considering file proximity. However, programmer sensitivity might be an important factor while ranking co-change candidates. Intuitively, it is more comfortable for a programmer to change her own code rather than changing someone else's code. Considering this fact, we investigate ranking co-change candidates suggested by FLeCCS considering programmer sensitivity.In our research, we have two main contributions: (1) proposing a composite ranking mechanism considering three things - file proximity, code similarity, and programmer sensitivity and (2) an empirical evaluation of our proposed mechanism by comparing it with file proximity ranking and similarity extent ranking mechanisms on thousands of revisions of five subject systems. We compare the ranking mechanisms in ranking the co-change candidates suggested by FLeCCS. According to our investigation incorporating four measures (Precision At K, Recall At K, Mean Average Precision and Mean Reciprocal Rank), composite ranking mechanism significantly outperforms the other two existing ranking mechanisms. Our findings are supported by statistical significance tests. We believe that our Proposed Composite Ranking mechanism is not only applicable to FLeCCS but also to other co-change suggestion techniques.

GBSR: Graph-based suspiciousness refinement for improving fault localization

Fault Localization (FL) is an important and time-consuming phase of software debugging. The essence of FL lies in the process of calculating the suspiciousness of different program entities (e.g., statements) and generating a ranking list to guide developers in their code inspection. Nonetheless, a prevalent challenge within existing FL methodologies is the propensity for program entities with analogous execution information to receive a similar suspiciousness. This phenomenon can lead to confusion among developers, thereby reducing the effectiveness of debugging significantly. To alleviate this issue, we introduce fine-grained contextual information (such as partial code structural, coverage, and features from mutation analysis) to enrich the characteristics of program entities. Graphical structures are proposed to organize such information, where the passed and failed tests are constructed separately with the consideration of their differential impacts. In order to support the analysis of multidimensional features and the representation of large-scale programs, the PageRank algorithm is adopted to compute each program entity’s weight. Rather than altering the fundamental FL process, we leverage these computed weights to refine the suspiciousness produced by various FL techniques, thereby providing developers with a more precise and actionable ranking of potential fault locations. The proposed strategy Graph-Based Suspiciousness Refinement (GBSR) is evaluated on 243 real-world faulty programs from the Defects4J. The results demonstrate that GBSR can improve the accuracy of various FL techniques. Specifically, for the refinement with traditional SBFL and MBFL techniques, the number of faults localized by the first position of the ranking list (Top-1) is increased by 189% and 68%, respectively. Furthermore, GBSR can also boost the state-of-the-art learning-based FL technique Grace by achieving a 2.8% performance improvement in Top-1.

Redusharptor: A Tool to Simplify Developer-Written C# Unit Tests

Modern software systems are complex and locating, isolating and fixing a fault even with a failing test is tedious and time-consuming. Simplifying failing test(s) can significantly reduce the developer effort by reducing the irrelevant program entities that developers need to observe. Delta Debugging (DD) algorithm automatically reduces the failing tests. Hierarchical Delta Debugging (HDD) algorithm improves DD for hierarchical tests like source code and HTML files. Many modern implementations of these algorithms work on a generic tree-like structure and fail to consider complex structures, intricacies, and interdependence of program elements of a particular programming language. We propose a tool ReduSharptor to simplify C# tests that uses language-specific features and interdependence of C# program elements using Roslyn compiler APIs. We evaluate ReduSharptor on a set of 30 failing C# tests to demonstrate its applicability and accuracy.

A qualitative study

A qualitative study

An Empirical Study of Boosting Spectrum-Based Fault Localization via PageRank

Manual debugging is notoriously tedious and time-consuming. Therefore, various automated fault localization techniques have been proposed to help with manual debugging. Among the existing fault localization techniques, spectrum-based fault localization (SBFL) is one of the most widely studied techniques due to being lightweight. The focus of the existing SBFL techniques is to consider how to differentiate program entities (i.e., one dimension in program spectra); indeed, this focus is aligned with the ultimate goal of finding the faulty lines of code. Our key insight is to enhance the existing SBFL techniques by additionally considering how to differentiate tests (i.e., the other dimension in program spectra), which, to the best of our knowledge, has not been studied in prior work. We present our basic approach, PRFL, a lightweight technique that boosts SBFL by differentiating tests using PageRank algorithm. Specifically, given the original program spectrum information, PRFL uses PageRank to recompute the spectrum by considering the contributions of different tests. Next, traditional SBFL techniques are applied on the recomputed spectrum to achieve more effective fault localization. On top of PRFL, we explore PRFL+ and PRFL <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MA</sub> , two variants which extend PRFL by optimizing its components and integrating Method-level Aggregation technique, respectively. Though being simple and lightweight, PRFL has been demonstrated to outperform state-of-the-art SBFL techniques significantly (e.g., ranking 39.2% / 82.3% more real/artificial faults at Top-1 compared with the most effective traditional SBFL technique) with low overhead (e.g., around 6 minutes average extra overhead on real faults) on 395 real faults from 6 Defects4J projects and 96925 artificial (i.e., mutation) faults from 240 GitHub projects. To further validate PRFL's effectiveness, we compare PRFL with multiple recent proposed fault localization techniques (e.g., Multric, Metallaxis and MBFL-hybrid-avg), and the experimental results show that PRFL outperforms them as well. Furthermore, we study the performance of PRFL <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MA</sub> , and the experimental results present it can locate 137 real faults (73.4% / 24.5% more compared with the most effective SBFL/PRFL technique) and 35058 artificial faults (159.6% / 28.1% more than SBFL/PRFL technique) at Top-1. At last, we study the generalizability of PRFL on another benchmark, Bugs.jar, and the result shows PRFL can help locate around 30 percent more faults at Top 1.

ID-correspondence: a measure for detecting evolutionary coupling

Evolutionary coupling is a well investigated phenomenon in software maintenance research and practice. Association rules and two related measures, support and confidence, have been used to identify evolutionary coupling among program entities. However, these measures only emphasize the co-change (i.e., changing together) frequency of entities and cannot determine whether the entities co-evolved by experiencing related changes. Consequently, the approach reports false positives and fails to detect evolutionary coupling among infrequently co-changed entities. We propose a new measure, identifier correspondence (id-correspondence), that quantifies the extent to which changes that occurred to the co-changed entities are related based on identifier similarity. Identifiers are the names given to different program entities such as variables, methods, classes, packages, interfaces, structures, unions etc. We use Dice-Sorensen co-efficient for measuring lexical similarity between the identifiers involved in the changed lines of the co-changed entities. Our investigation on thousands of revisions from nine subject systems covering three programming languages shows that id-correspondence can considerably improve the detection accuracy of evolutionary coupling. It outperforms the existing state-of-the-art evolutionary coupling based techniques with significantly higher recall and F-score in predicting future co-change candidates.

Improving Fault-Localization Accuracy by Referencing Debugging History to Alleviate Structure Bias in Code Suspiciousness

Spectrum-based fault localization (SBFL) techniques can automatically localize software faults. They employ the program spectrum, such as code coverage profile with test verdicts, to rank the program entities based on their code suspiciousness. In the past decades, researchers have proposed many approaches to optimize these techniques; however, the program structure, which can influence their performance, is not taken into consideration in developing and improving these techniques. In this article, we identify and analyze the effect of the program structure on the application of SBFL techniques. We observe that some specific program structures may introduce structure bias to code suspiciousness and negatively influence the output of SBFL techniques. To mitigate these effects and improve the performance of fault localization, we propose Delta4Ts, a structure-aware technique. Delta4Ts references debugging history to alleviate the impact of structure bias in the calculation of code suspiciousness. It reasons from the observable suspicious value towards the desired suspicious value and the impact of structure bias. To evaluate Delta4Ts under practical constraints, we conduct a controlled experiment using nine widely-studied SBFL formulae on 12 C programs and 6 Java programs. The experiment results show that Delta4Ts can significantly improve the accuracy of the studied SBFL formulae by an average of 34.8% on 12 C programs and 30.6% on 6 Java programs, and improve more on subject programs associated with more history versions or having larger code sizes.

Mining Association Rules from Code (MARC) to support legacy software management

This paper presents a methodology for Mining Association Rules from Code (MARC), aiming at capturing program structure, facilitating system understanding and supporting software management. MARC groups program entities (paragraphs or statements) based on similarities, such as variable use, data types and procedure calls. It comprises three stages: code parsing/analysis, association rule mining and rule grouping. Code is parsed to populate a database with records and respective attributes. Association rules are then extracted from this database and subsequently processed to abstract programs into groups containing interrelated entities. Entities are then grouped together if their attributes participate to common rules. This abstraction is performed at the program level or even the paragraph level, in contrast to other approaches that work at the system level. Groups can then be visualised as collections of interrelated entities. The methodology was evaluated using real-life COBOL programs. Results showed that the methodology facilitates program comprehension by using source code only, where domain knowledge and documentation are either unavailable or unreliable.

Automatic method change suggestion to complement multi-entity edits

When maintaining software, developers sometimes change multiple program entities (i.e., classes, methods, and fields) to fulfill one maintenance task. We call such complex changes multi-entity edits. Consistently and completely applying multi-entity edits can be challenging, because (1) the changes scatter in different entities and (2) the incorrectly edited code may not trigger any compilation or runtime error. This paper introduces CMSuggester, an approach to suggest complementary changes for multi-entity edits. Given a multi-entity edit that (i) adds a new field or method and (ii) modifies one or more methods to access the field or invoke the method, CMSuggester suggests other methods to co-change for the new field access or method invocation. The design of CMSuggester is motivated by our preliminary study, which reveals that co-changed methods usually access existing fields or invoke existing methods in common.Our evaluation shows that based on common field accesses, CMSuggester recommended method changes in 463 of 685 tasks with 70% suggestion accuracy; based on common method invocations, CMSuggester handled 557 of 692 tasks with 70% accuracy. Compared with prior work ROSE, TARMAQ, and Transitive Association Rules (TAR), CMSuggester recommended more method changes with higher accuracy. Our research can help developers correctly apply multi-entity edits.

Precise Learn-to-Rank Fault Localization Using Dynamic and Static Features of Target Programs

Finding the root cause of a bug requires a significant effort from developers. Automated fault localization techniques seek to reduce this cost by computing the suspiciousness scores (i.e., the likelihood of program entities being faulty). Existing techniques have been developed by utilizing input features of specific types for the computation of suspiciousness scores, such as program spectrum or mutation analysis results. This article presents a novel learn-to-rank fault localization technique called PRecise machINe-learning-based fault loCalization tEchnique (PRINCE). PRINCE uses genetic programming (GP) to combine multiple sets of localization input features that have been studied separately until now. For dynamic features, PRINCE encompasses both Spectrum Based Fault Localization (SBFL) and Mutation Based Fault Localization (MBFL) techniques. It also uses static features, such as dependency information and structural complexity of program entities. All such information is used by GP to train a ranking model for fault localization. The empirical evaluation on 65 real-world faults from CoREBench, 84 artificial faults from SIR, and 310 real-world faults from Defects4J shows that PRINCE outperforms the state-of-the-art SBFL, MBFL, and learn-to-rank techniques significantly. PRINCE localizes a fault after reviewing 2.4% of the executed statements on average (4.2 and 3.0 times more precise than the best of the compared SBFL and MBFL techniques, respectively). Also, PRINCE ranks 52.9% of the target faults within the top ten suspicious statements.

Exoneration-based fault localization for SQL predicates

Spectrum-based fault localization (SFL) techniques automatically localize faults in program entities (statements, predicates, SQL clauses, etc.) by analyzing information collected from test executions. One application of SFL techniques is to find faulty SQL statements in database applications. However, prior techniques treated each SQL statement as one program entity, thus they could not find faulty elements inside SQL statements. Since SQL statements can be complex, identifying the faulty element within a faulty SQL statement is still time-consuming.In our previous paper, we developed a novel fault localization method based on row-based dynamic slicing and delta debugging techniques that can localize faults in individual clauses within SQL predicates. We call this technique exoneration-based fault localization because it can exonerate “innocent” elements and precisely identify the faulty element, whereas previous SFL techniques simply ranked all the elements in an SQL statement based on suspiciousness.This paper improves the exoneration-based fault localization technique with a new algorithm that considerably reduces the execution time. We also conducted an empirical study that compared nine existing SFL techniques with the exoneration-based technique in localizing faulty clauses in SQL predicates. Results indicate that the new exoneration-based technique surpasses the other techniques both in terms of effectiveness and efficiency.

Towards a Reliable Identification of Deficient Code with a Combination of Software Metrics

Different challenges arise while detecting deficient software source code. Usually a large number of potentially problematic entities are identified when an individual software metric or individual quality aspect is used for the identification of deficient program entities. Additionally, a lot of these entities quite often turn out to be false positives, i.e., the metrics indicate poor quality whereas experienced developers do not consider program entities as problematic. The number of entities identified as potentially deficient does not decrease significantly when the identification of deficient entities is carried out by applying code smell detection rules. Moreover, the intersection of entities identified as allegedly deficient among different code smell detection tools is small, which suggests that the implementation of code smell detection rules are not consistent and uniform. To address these challenges, we present a novel approach for identifying deficient entities that is based on applying the majority function on the combination of software metrics. Program entities are assessed according to selected quality aspects that are evaluated with a set of software metrics and corresponding threshold values derived from benchmark data, considering the statistical distributions of software metrics values. The proposed approach was implemented and validated on projects developed in Java, C++ and C#. The validation of the proposed approach was done with expert judgment, where software developers and architects with multiple years of experiences assessed the quality of the software classes. Using a combination of software metrics as the criteria for the identification of deficient source code, the number of potentially deficient object-oriented program entities proved to be reduced. The results show the correctness of quality ratings determined by the proposed identification approach, and most importantly, confirm the absence of false positive entities.

Evolutionary approach to generating test data for data flow test

Software testing consumes a significant portion of software effort. Program entities such as branch or definition-use pairs (DUPs) are used in diverse software development tasks. In this study, the authors present a novel evolution-based approach to generating test data for all definition-use coverage. First, the subset of DUPs, which can ensure the coverage adequacy, is computed by a reduction algorithm for the whole DUPs. Then they apply a genetic algorithm to generate test data for the subset of DUPs. Furthermore, the fitness of an individual depends on the matching degree between the traversed path and the definition-clear path of each target DUP. They also investigate the coverage and the size of test cases of test data generation by applying the authors' approach on 15 widely used subject programs. The experimental results show that their approach can reduce the size of test cases that generated without affecting the coverage rate.

Hybrid Program Dependence Approximation for Effective Dynamic Impact Prediction

Impact analysis determines the effects that program entities of interest, or changes to them, may have on the rest of the program for software measurement, maintenance, and evolution tasks. Dynamic impact analysis could be one major approach to impact analysis that computes smaller impact setsthan static alternatives for concrete sets of executions. However, existing dynamic approaches often produce impact sets that are too large to be useful, hindering their adoption in practice. To address this problem, we propose to exploit static program dependencies to drastically prune false-positive impacts that are not exercised by the set of executions utilized by the analysis, via hybrid dependence approximation. Further, we present a novel dynamic impact analysis called Diver which leverages both the information provided by the dependence graph and method-execution events to identify runtimemethod-level dependencies, hence dynamic impact sets, much more precisely without reducing safety and at acceptable costs. We evaluate Diver on ten Java subjects of various sizes and application domains against both arbitrary queries covering entire programs and practical queries based on changes actually committed by developers to actively evolving software repositories. Our extensive empirical studies show that Diver can significantly improve the precision of impact prediction, with 100-186 percent increase, with respect to a representative existing alternative thus provide a far more effective option for dynamic impact prediction. Following a similar rationale to Diver, we further developed and evaluated an online dynamic impact analysis called DiverOnline which produces impact sets immediately upon the termination of program execution. Our results show that compared to the offline approach, for the same precision, the online approach can reduce the time by 50 percent on average for answering all possible queries in the given program at once albeit at the price of possibly significant increase in runtime overhead. For users interested in one specific query only, the online approach may compute the impact set for that query during runtime without much slowing down normal program operation. Further, the online analysis, which does not incur any space cost beyond the static-analysis phase, may be favored against the offline approach when trace storage and/or related file-system resource consumption becomes a serious challenge or even stopper for adopting dynamic impact prediction. Therefore, the online and offline analysis together offer complementary options to practitioners accommodating varied application/task scenarios and diverse budget constraints.

A vector table model-based systematic analysis of spectral fault localization techniques

Spectral fault localization (SFL) is an automatic fault-localization technique, which uses risk evaluation formula to rank the risk of fault existence in each program entity after collecting the testing information dynamically. To provide insight into SFL techniques, the evaluation method is an important research topic. In this paper, we present a uniformly systematic investigation framework to evaluate and compare SFL techniques with fixed formulas considering both single-fault and multiple-fault scenarios. Particularly, we design a generic vector table model called VTM as a novel measurement model to thoroughly understand various SFL techniques. By defining different types of faulty statements and investigating suspiciousness factors’ mathematical expression of statements on the basis of VTM, the effectiveness of different SFL techniques could be systematically analyzed and compared. Under the VTM-based evaluation framework, the latest formula D* and the optimal formula O considered before as examples are explored: (1) under a single-fault scenario, O has a better performance than D*, and O is the best SFL technique at present; (2) O shows stable performance and the performance of D* fluctuates in a range under double-fault scenarios: O outperforms D* in three cases, and the performance of O is between the worst performance and the best performance of D* in the other three cases that are less likely to happen; and (3) sample programs are presented to explain such observations. The VTM-based method overcomes the limitations of existing empirical and systematic approaches, which enables a systematic evaluation for SFL techniques with fixed formulas under both single-fault and multiple-fault cases.

A program slicing-based method for effective detection of coincidentally correct test cases

Despite the proven applicability of the spectrum-based fault localization (SBFL) methods, their effectiveness may be degraded due to the presence of coincidental correctness, which occurs when faults fail to propagate, i.e., their execution does not result in failures. This article aims at improving SBFL effectiveness by mitigating the effect of coincidentally correct test cases. In this regard, given a test suite in which each test has been classified as failing or passing and each faulty program has a single-bug, we present a program slicing-based technique to identify a set of program entities that directly affect the program output when executed with failing test cases, called failure candidate causes (FCC). We then use FCC set to identify test cases that can be marked as being coincidentally correct. These tests are identified based on two heuristics: the average suspiciousness score of the statements that directly affect the program output and the coverage ratio of those statements. To evaluate our approach, we used several evaluation metrics and conducted extensive experiments on programs containing single and multiple bugs, including both real and seeded faults. The empirical results demonstrate that the proposed heuristics can alleviate the coincidental correctness problem and improve the accuracy of SBFL techniques.

An optimal mutation execution strategy for cost reduction of mutation-based fault localization

Identifying faulty program entities has been recognized as one of the most expensive, tedious and time-consuming processes in software debugging activity. Fault localization techniques are designed to assist developers in locating faults by giving a ranking of the probability that program entities incur failures. Mutation-based fault localization (MBFL) is a recently proposed fault localization approach via mutation analysis, which uses the location of mutants to identify the faulty statements. With improved effectiveness, the MBFL also brings huge execution cost. To reduce the execution cost of MBFL technique, this paper proposes a dynamic mutation execution strategy (DMES) to prioritize the execution on both mutants and test cases. As fewer mutants and test cases are executed with DMES, the whole process will become faster and the cost will be decreased. At the same time, it is proved that the fault localization accuracy of MBFL with DMES is the same as that of the original MBFL. Furthermore, this paper discusses the lowest mutation execution cost of MBFL in theory and gives a quick solution to compute the theoretical minimal mutation execution cost of MBFL in the case of keeping fault localization accuracy non-decreasing. The empirical studies show that MBFL with the dynamic strategy proposed in this paper can reduce mutant-test execution cost by 32.4–87% and is close to the theoretical optimal cost. Furthermore, the additional run time required by utilizing our strategy is minimum and can be ignored.

Lightweight fault localization combined with fault context to improve fault absolute rank

Lightweight fault localization (LFL), which outputs a list of suspicious program entities in descending order based on their likelihood to be a root fault, is a popular method used by programmers to assist them in debugging. However, owing to the nature of program structures, it is impossible for LFL to always rank the root faulty program entity at the top of a ranking list. Recently, Xia et al. noted in their study that programmers inspect the first top- K -ranked program entities outputted by an LFL tool in sequence. Therefore, it is of practical significance to further improve the absolute rank of those buggy programs by using LFL if the root fault is ranked higher. To solve this issue, we propose a new LFL combined with a fault context to improve the fault absolute rank. We conduct experiments in which we apply our proposed approach to seven Siemens benchmark programs. The results show that by combining the suspiciousness scores of program entities with their fault-context suspiciousness scores that are based on an LFL called Dstar, our approach can improve the fault absolute rank with an effectiveness rate of 35.7% for 129 faulty versions from the seven benchmark programs. It should be noted that our approach can obtain an average improvement of 65.18% for those improved programs to which LFL can be effectively applied, and that there were improvements to seven top-ranked root faults of buggy programs.

Noise in Mylyn interaction traces and its impact on developers and recommendation systems

Interaction traces (ITs) are developers' logs collected while developers maintain or evolve software systems. Researchers use ITs to study developers' editing styles and recommend relevant program entities when developers perform changes on source code. However, when using ITs, they make assumptions that may not necessarily be true. This article assesses the extent to which researchers' assumptions are true and examines noise in ITs. It also investigates the impact of noise on previous studies. This article describes a quasi-experiment collecting both Mylyn ITs and video-screen captures while 15 participants performed four realistic software maintenance tasks. It assesses the noise in ITs by comparing Mylyn ITs and the ITs obtained from the video captures. It proposes an approach to correct noise and uses this approach to revisit previous studies. The collected data show that Mylyn ITs can miss, on average, about 6% of the time spent by participants performing tasks and can contain, on average, about 85% of false edit events, which are not real changes to the source code. The approach to correct noise reveals about 45% of misclassification of ITs. It can improve the precision and recall of recommendation systems from the literature by up to 56% and 62%, respectively. Mylyn ITs include noise that biases subsequent studies and, thus, can prevent researchers from assisting developers effectively. They must be cleaned before use in studies and recommendation systems. The results on Mylyn ITs open new perspectives for the investigation of noise in ITs generated by other monitoring tools such as DFlow, FeedBag, and Mimec, and for future studies based on ITs.

A theoretical analysis on cloning the failed test cases to improve spectrum-based fault localization

Abstract Fault localization is the activity to locate faults in programs. Spectrum-based fault localization (SBFL) is a class of techniques for it. It contrasts the code coverage achieved by passed runs and that by failed runs, and estimates program entities responsible for the latter. Although previous work has empirically shown that the effectiveness of typical SBFL techniques can be improved by incorporating more failed runs, debugging often takes place when there are very few of them. In this paper, we report a comprehensive study to investigate the impact of cloning the failed test cases on the effectiveness of SBFL techniques. We include 33 popular such techniques, and examine the accuracy of their formulas on twelve benchmark programs, using four accuracy metrics and in three scenarios. The empirical results show that on 22, 21, and 23 of them the fault-localization accuracy can be significantly improved, when the failed test cases are cloned in the single-fault, double-fault, and triple-fault scenarios, respectively. We also analytically show that on 19 of them the improvements are provable for an arbitrary program and an arbitrary test suite, in the single-fault scenario; and moreover, for ten of the rest formulas, their accuracy are proved unaffected in all scenarios.