Information Retrieval Based Bug Localization Research Articles

COPS: An improved information retrieval-based bug localization technique using context-aware program simplification

Information Retrieval Based Bug Localization (IRBL) techniques are well suited for large-scale software debugging with fewer external dependencies and lower execution costs. However, existing IRBL techniques have several challenges, including localization granularity and applicability. First, existing IRBL techniques have not yet achieved statement-level bug localization. Second, almost all studies are limited to Java-based projects, while their effectiveness for other popular programming languages (e.g., Python) is unknown. The reason for these deficiencies is that existing IRBL techniques mainly rely on conventional NLP techniques to analyze the bug reports and have not yet fully utilized the stack traces attached to the bug reports. To improve the IRBL technique, we propose a context-aware program simplification technique – COPS – that can localize defective statements in suspicious files by analyzing the stack traces in bug reports, enabling statement-level bug localization for Python-based projects. Our experiment is based on 948 bug reports, and the results show that COPS can effectively localize buggy statements. First, compared to the original stack traces, Top@10 is improved by 102.6%, MAP@10 by 56.2%, and MRR@10 by 95.6%. We found that actual buggy code entities are more likely to appear in the first five frames of the stack trace. Second, COPS can achieve equally good localization performance compared to state-of-the-art statement-level bug localization techniques and achieve 92% buggy statement coverage with a full-scope search. Finally, experiments found that the stack trace’s first two-thirds of information is more conducive to localizing buggy statements.

BugRadar: Bug localization by knowledge graph link prediction

: Information Retrieval-based Bug Localization (IRBL) aims to design automatic systems that find buggy files according to bug reports, which can reduce the time consumption to fix bugs for programmers. There has been extensive research on IRBL techniques in recent years. However, these methods cannot make full use of the structure information in bug reports and source files. : In this paper, we propose a novel scheme BugRadar. It combines text features and structure features from bug reports and source files for bug localization. Especially, BugRadar leverages a knowledge graph to make use of structure features. : We originally propose a knowledge graph named TriGraph based on structure features and apply hyperbolic attention embedding to get the link prediction scores. For text features, we propose Partial Text Similarity which improves traditional Text Similarity and Method Level Text Similarity. We also propose Word Collaborative Filtering Score which leverages historical bug reports with more attention on important terms. Finally, we calculate the final suspicious scores based on the structure features, text features, and fixing time information from bug fixing history with a neural network. : We apply our scheme to four projects (Tomcat, SWT, JDT, and Birt) in a popular dataset and get approving results. BugRadar gets better results than other state-of-the-art methods on three projects out of the four. It achieves a relative improvement of 8.8% in SWT and 9.8% in JDT for Mean Average Precision compared to the previous best scheme KGBugLocator and 11.4% in Birt compared to Adaptive Regression. : BugRadar can achieve approving performance on large-scale projects with enough historical bug reports. It verifies that knowledge graphs are capable of representing the structure features for bug localization. The novel Partial Text Similarity and Word Collaborative Filtering Score are both effective improvements for using text features.

DreamLoc: A Deep Relevance Matching-Based Framework for bug Localization

To improve the software debugging efficiency, bug localization techniques have been developed to automatically locate buggy files based on bug reports. Traditional information retrieval-based bug localization cannot deal with the lexical mismatch, thus its performance is limited. In recent years, some deep learning models have been proposed to learn the semantics of bug reports and source files to bridge the lexical gap. However, their accuracy is still limited as building accurate semantic representations of bug reports and source files is very challenging. Recently, relevance matching was proposed to identify whether a document is relevant to a given query by considering both local matching and global matching. In this work, we propose a novel framework DreamLoc, which utilizes a relevance matching model to locate buggy files. Specifically, DreamLoc conducts the local matching by employing an attention-based mechanism to calculate the matching scores between bug report terms and code snippets. It also conducts the global matching by employing a gating mechanism to aggregate results of local matching and obtain the final matching score between a bug report and a source file. Since the local matching considers the relevance between each word and the global matching differentiates the importance of words, DreamLoc can effectively model the characteristics of bug reports and source files. Experimental results on five benchmark datasets show that DreamLoc outperforms five state-of-the-art models. For example, compared with DeepLoc, a recently proposed approach, the evaluation measures Accuracy@10, MAP, and MRR are improved by 6.4%, 7.4%, and 7.2%, respectively.

Information retrieval-based bug localization approach with adaptive attribute weighting

Software quality assurance is one of the crucial factors for the success of software projects. Bug fixing has an essential role in software quality assurance, and bug localization (BL) is the first step of this process. BL is difficult and time-consuming since the developers should understand the flow, coding structure, and the logic of the program. Information retrieval-based bug localization (IRBL) uses the information of bug reports and source code to locate the section of code in which the bug occurs. It is difficult to apply other tools because of the diversity of software development languages, design patterns, and development standards. The aim of this study is to build an adaptive IRBL tool and make it usable by more companies. BugSTAiR solves the aforementioned problem by means of the adaptive attribute weighting (AAW) algorithm and is evaluated on four open-source projects which are well-known benchmark datasets on BL. One of them is BLIA which is the state of the art in bug localization area and another is BLUIR which is a well-known BL tool. According to the promising results of experiments, Top1 rank of BugSTAiR is 2% and MAP is 10% better than BLIA's results on AspectJ and it has localized 4.6% of all bugs in Top1 and its precision is 6.1% better than BLIA on SWT, respectively. On the other side, it is 20% better in the Top1 metric and 30% in precision than BLUIR.

Pathidea: Improving Information Retrieval-Based Bug Localization by Re-Constructing Execution Paths Using Logs

To assist developers with debugging and analyzing bug reports, researchers have proposed information retrieval-based bug localization (IRBL) approaches. IRBL approaches leverage the textual information in bug reports as queries to generate a ranked list of potential buggy files that may need further investigation. Although IRBL approaches have shown promising results, most prior research only leverages the textual information that is “visible” in bug reports, such as bug description or title. However, in addition to the textual description of the bug, developers also often attach logs in bug reports. Logs provide important information that can be used to re-construct the system execution paths when an issue happens and assist developers with debugging. In this paper, we propose an IRBL approach, Pathidea, which leverages logs in bug reports to re-construct execution paths and helps improve the results of bug localization. Pathidea uses static analysis to create a file-level call graph, and re-constructs the call paths from the reported logs. We evaluate Pathidea on eight open source systems, with a total of 1,273 bug reports that contain logs. We find that Pathidea achieves a high recall (up to 51.9 percent for Top@5). On average, Pathidea achieves an improvement that varies from 8 to 21 and 5 to 21 percent over BRTracer in terms of Mean Average Precision (MAP) and Mean Reciprocal Rank (MRR) across studied systems, respectively. Moreover, we find that the re-constructed execution paths can also complement other IRBL approaches by providing a 10 and 8 percent improvement in terms of MAP and MRR, respectively. Finally, we conduct a parameter sensitivity analysis and provide recommendations on setting the parameter values when applying Pathidea.

Are datasets for information retrieval-based bug localization techniques trustworthy?

Various evaluation datasets are used to evaluate the performance of information retrieval-based bug localization (IRBL) techniques. To accurately evaluate the IRBL and furthermore improve the performance, it is strongly required to analyze the validity of these datasets in advance. To this end, we surveyed 50 previous studies, collected 41,754 bug reports, and found out critical problems that affect the validity of results of performance evaluation. They are in both the ground truth and the search space. These problems arise from using different bug types without clearly distinguishing them. We divided the bugs into production- and test-related bugs. Based on this distinction, we investigate and analyze the impact of the bug type on IRBL performance evaluation. Approximately 18.6% of the bug reports were linked to non-buggy files as the ground truth. Up to 58.5% of the source files in the search space introduced noise into the localization of a specific bug type. From the experiments, we validated that the average precision changed in approximately 90% of the bug reports linked with an incorrect ground truth; we determined that specifying a suitable search space changed the average precision in at least half of the bug reports. Further, we showed that these problems can alter the relative ranks of the IRBL techniques. Our large-scale analysis demonstrated that a significant amount of noise occurs, which can compromise the evaluation results. An important finding of this study is that it is essential to consider the bug types to improve the accuracy of the performance evaluation.

ManQ: Many-objective optimization-based automatic query reduction for IR-based bug localization

Abstract Context An information retrieval-based bug localization (IRBL) method is proposed to localize buggy files using a bug report as a query. The performance of this method strongly depends on the quality of the query. However, these queries contain noise terms that hinder their use for IRBL. To improve the quality of a query, an automatic query reduction (AQR) technique that removes noise words from the query is needed. Objective Our objective is to develop an AQR method for IRBL. Most existing AQR techniques are based on single objective optimization, which presents issues in terms of biased and limited performance. To solve these issues, it is necessary to find a subquery that comprehensively satisfies all of their objectives. Method We propose an AQR technique called ManQ, which is a many-objective optimization-based AQR method for IRBL. We design 15 objective functions to (1) maintain the query quality properties, (2) maintain the important terms, (3) maintain the initial information, and (4) minimize the query length. ManQ finds a final subquery that maximize the return values of these objective functions. Results The experimental results show that ManQ improves the quality of poor queries. We also show that if we select the best query among the candidates generated by ManQ, we can increase the number of improved queries by more than 53.4% of all queries. Conclusion ManQ improves the performance of IRBL by improving the quality of queries through a many-objective optimization approach.

정보검색기반 결함위치식별 기술의 성능 향상을 위한 버그리포트 품질 예측

정보검색기반 결함위치식별 기술의 성능 향상을 위한 버그리포트 품질 예측

Will this localization tool be effective for this bug? Mitigating the impact of unreliability of information retrieval based bug localization tools

Information retrieval (IR) based bug localization approaches process a textual bug report and a collection of source code files to find buggy files. They output a ranked list of files sorted by their likelihood to contain the bug. Recently, several IR-based bug localization tools have been proposed. However, there are no perfect tools that can successfully localize faults within a few number of most suspicious program elements for every single input bug report. Therefore, it is difficult for developers to decide which tool would be effective for a given bug report. Furthermore, for some bug reports, no bug localization tools would be useful. Even a state-of-the-art bug localization tool outputs many ranked lists where buggy files appear very low in the lists. This potentially causes developers to distrust bug localization tools. In this work, we build an oracle that can automatically predict whether a ranked list produced by an IR-based bug localization tool is likely to be effective or not. We consider a ranked list to be effective if a buggy file appears in the top-N position of the list. If a ranked list is unlikely to be effective, developers do not need to waste time in checking the recommended files one by one. In such cases, it is better for developers to use traditional debugging methods or request for further information to localize bugs. To build this oracle, our approach extracts features that can be divided into four categories: score features, textual features, topic model features, and metadata features. We build a separate prediction model for each category, and combine them to create a composite prediction model which is used as the oracle. We name this solution APRILE, which stands for Automated PRediction of IR-based Bug Localization's Effectiveness. We further integrate APRILE with two other components that are learned using our bagging-based ensemble classification (BEC) method. We refer to the extension of APRILE as APRILE +. We have evaluated APRILE + to predict the effectiveness of three state-of-the-art IR-based bug localization tools on more than three thousands bug reports from AspectJ, Eclipse, SWT, and Tomcat. APRILE + can achieve an average precision, recall, and F-measure of 77.61 %, 88.94 %, and 82.09 %, respectively. Furthermore, APRILE + outperforms a baseline approach by Le and Lo and APRILE by up to a 17.43 % and 10.51 % increase in F-measure respectively.

Improved bug localization based on code change histories and bug reports

ContextSeveral issues or defects in released software during the maintenance phase are reported to the development team. It is costly and time-consuming for developers to precisely localize bugs. Bug reports and the code change history are frequently used and provide information for identifying fault locations during the software maintenance phase. ObjectiveIt is difficult to standardize the style of bug reports written in natural languages to improve the accuracy of bug localization. The objective of this paper is to propose an effective information retrieval-based bug localization method to find suspicious files and methods for resolving bugs. MethodIn this paper, we propose a novel information retrieval-based bug localization approach, termed Bug Localization using Integrated Analysis (BLIA). Our proposed BLIA integrates analyzed data by utilizing texts, stack traces and comments in bug reports, structured information of source files, and the source code change history. We improved the granularity of bug localization from the file level to the method level by extending previous bug repository data. ResultsWe evaluated the effectiveness of our approach based on experiments using three open-source projects, namely AspectJ, SWT, and ZXing. In terms of the mean average precision, on average our approach improves the metric of BugLocator, BLUiR, BRTracer, AmaLgam and the preliminary version of BLIA by 54%, 42%, 30%, 25% and 15%, respectively, at the file level of bug localization. ConclusionCompared with prior tools, the results showed that BLIA outperforms these other methods. We analyzed the influence of each score of BLIA from various combinations based on the analyzed information. Our proposed enhancement significantly improved the accuracy. To improve the granularity level of bug localization, a new approach at the method level is proposed and its potential is evaluated.