Duplicate Bug Reports Research Articles

BushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports

BushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports

A Comparison of Summarization Methods for Duplicate Software Bug Reports

Bug reports vary in length, while some bug reports are lengthy, others are too brief to describe bugs in detail. In such a case, duplicate bug reports can serve as valuable resources for enriching bug descriptions. However, existing bug summarization methods mainly focused on summarizing a single bug report. In this paper, we focus on summarizing duplicate bug reports. By doing so, we aim to obtain an informative summary of bug reports while reducing redundant sentences in the summary. We apply several text summarization methods to duplicate bug reports. We then compare summarization results generated by different summarization methods and identify the most effective method for summarizing duplicate bug reports. Our comparative experiment reveals that the extractive multi-document method based on TF-IDF is the most effective in the summarization. This method successfully captures the relevant information from duplicate bug reports, resulting in comprehensive summaries. These results contribute to the advancement of bug summarization techniques, especially in summarizing duplicate bug reports.

Duplicate Bug Report Detection Using an Attention-Based Neural Language Model

<i>Context:</i> Users and developers use bug tracking systems to report errors that occur during the development and testing of software. The manual identification of duplicates is a tedious task especially with software that have large bug repositories. In this context, their automatic detection becomes a necessary task that can help prevent frequently fixing the same bug. <i>Objective:</i> In this article, we propose <i>BERT-MLP</i>, a novel pretrained language model using bidirectional encoder representations from ransformers (BERT) for duplicate bug report detection (DBRD) with the aim of improving the detection rate compared to existing works. <i>Method:</i> Our approach considers only unstructured data. These are fed into the BERT model in order to learn the contextual relationships between words. The output is fed into a multilayer perceptron (MLP) classifier, representing our base DBRD. <i>Results:</i> Our approach was evaluated on three projects: Mozilla Firefox, Eclipse Platform, and Thunderbird. It achieved an accuracy of 92.11, 94.08, and 89.03&#x0025;, respectively, for Mozilla, Eclipse, and Thunderbird. A comparison with a dual-channel convolutional neural network (DC-CNN) model and other pretrained models, including RoBERTa and Sentence-Bert has been conducted. Results showed that <i>BERT-MLP</i> outperformed, the second best performing models (DC-CNN and Sentence-BERT) by 12&#x0025; in accuracy for Eclipse and 9&#x0025; for both Mozilla and Thunderbird, respectively.

Duplicate Bug Report Detection: How Far Are We?

Many Duplicate Bug Report Detection (DBRD) techniques have been proposed in the research literature. The industry uses some other techniques. Unfortunately, there is insufficient comparison among them, and it is unclear how far we have been. This work fills this gap by comparing the aforementioned techniques. To compare them, we first need a benchmark that can estimate how a tool would perform if applied in a realistic setting today. Thus, we first investigated potential biases that affect the fair comparison of the accuracy of DBRD techniques. Our experiments suggest that data age and issue tracking system (ITS) choice cause a significant difference. Based on these findings, we prepared a new benchmark. We then used it to evaluate DBRD techniques to estimate better how far we have been. Surprisingly, a simpler technique outperforms recently proposed sophisticated techniques on most projects in our benchmark. In addition, we compared the DBRD techniques proposed in research with those used in Mozilla and VSCode . Surprisingly, we observe that a simple technique already adopted in practice can achieve comparable results as a recently proposed research tool. Our study gives reflections on the current state of DBRD, and we share our insights to benefit future DBRD research.

Sentence embedding and fine-tuning to automatically identify duplicate bugs

Industrial software maintenance is critical but burdensome. Activities such as detecting duplicate bug reports are often performed manually. Herein an automated duplicate bug report detection system improves maintenance efficiency using vectorization of the contents and deep learning–based sentence embedding to calculate the similarity of the whole report from vectors of individual elements. Specifically, sentence embedding is realized using Sentence-BERT fine tuning. Additionally, its performance is experimentally compared to baseline methods to validate the proposed system. The proposed system detects duplicate bug reports more effectively than existing methods.

Leveraging multi-level embeddings for knowledge-aware bug report reformulation

Software bug analysis based on the information retrieval (IR) technology is widely studied and used for bug understanding, localization and fixing. IR technology with various textual feature extraction methods formulates the textual information in a given new bug report (i.e., title and description) as an initial query. However, due to the low-quality content in the new bug report and improper representation to be used as a query, the retrieval results are usually not satisfactory.To alleviate these problems, we propose a novel knowledge-aware bug report reformulation approach (a.k.a, KABR) by leveraging multi-level embeddings from the bug data. First, we construct a bug-specific knowledge graph (KG) to manage and reuse prior knowledge extracted from historical bug reports. Then, we extract word embedding from the original bug data, entity embedding and context embedding from the bug-specific KG to enhance the initial query. Finally, a new query representation is generated by leveraging multi-level embeddings through Convolutional Neural Networks (CNN) with the self-attention mechanism. We evaluate KABR based on the duplicate bug report detection task, and the experimental results show that KABR achieves 6%–11% F1-measure improvement over the state-of-the-art approaches.

Does Deep Learning improve the performance of duplicate bug report detection? An empirical study

Do Deep Learning (DL) techniques actually help to improve the performance of duplicate bug report detection? Prior studies suggest that they do, if the duplicate bug report detection task is treated as a binary classification problem. However, in realistic scenarios, the task is often viewed as a ranking problem, which predicts potential duplicate bug reports by ranking based on similarities with existing historical bug reports. There is little empirical evidence to support that DL can be effectively applied to detect duplicate bug reports in the ranking scenario. Therefore, in this paper, we investigate whether well-known DL-based methods outperform classic information retrieval (IR) based methods on the duplicate bug report detection task. In addition, we argue that both IR- and DL-based methods suffer from incompletely evaluating the similarity between bug reports, resulting in the loss of important information. To address this problem, we propose a new method that combines IR and DL techniques to compute textual similarity more comprehensively. Our experimental results show that the DL-based method itself does not yield high performance compared to IR-based methods. However, our proposed combined method improves on the MAP metric of classic IR-based methods by a median of 7.09%–11.34% and a maximum of 17.228%–28.97%.

Adaptive Deep Learning Model for Software Bug Detection and Classification

Software is unavoidable in software development and maintenance. In literature, many methods are discussed which fails to achieve efficient software bug detection and classification. In this paper, efficient Adaptive Deep Learning Model (ADLM) is developed for automatic duplicate bug report detection and classification process. The proposed ADLM is a combination of Conditional Random Fields decoding with Long Short-Term Memory (CRF-LSTM) and Dingo Optimizer (DO). In the CRF, the DO can be consumed to choose the efficient weight value in network. The proposed automatic bug report detection is proceeding with three stages like pre-processing, feature extraction in addition bug detection with classification. Initially, the bug report input dataset is gathered from the online source system. In the pre-processing phase, the unwanted information from the input data are removed by using cleaning text, convert data types and null value replacement. The pre-processed data is sent into the feature extraction phase. In the feature extraction phase, the four types of feature extraction method are utilized such as contextual, categorical, temporal and textual. Finally, the features are sent to the proposed ADLM for automatic duplication bug report detection and classification. The proposed methodology is proceeding with two phases such as training and testing phases. Based on the working process, the bugs are detected and classified from the input data. The projected technique is assessed by analyzing performance metrics such as accuracy, precision, Recall, F_Measure and kappa.

Characterizing duplicate bugs: Perceptions of practitioners and an empirical analysis

AbstractBug handling is an essential part of the software development process. Ideally, in a bug‐tracking system, bugs are reported, fixed, verified, and closed. In some cases, bugs have to be reopened mostly due to an incorrect fix. However, instead of reopening the existing bug report, users may submit a new report for a previously reported bug, which results in duplicate bug reports. Users might report duplicate bugs if they miss the previous bug report or if the previous bug is unresolved (i.e., in progress) and therefore cannot be reopened. These duplicate bug reports may cost extra maintenance efforts in triaging and bug fixing. There have been several studies on characterizing reopened bugs and duplicate bug reports; however, to the best of our knowledge, there has been no prior work on understanding the dynamics of their interaction, which is missed reopen bugs. Our study is based on analyzing the difference between duplicate and nonduplicate bugs and further categorizing the duplicate bugs. In this regard, we categorize duplicate bugs according to the original bug's resolution status at the duplicate's creation time as Master‐Unresolved bugs and Master‐Resolved (Missed Reopen) bugs to distinguish their properties. We compare these two types of bugs in terms of their relationship to their master bug, bug surface time, bug fix time, bug's severity, and the number of users involved. We perform case studies using the Eclipse and Mozilla projects' bug repositories that include more than 165,500 and 394,000 bug reports, respectively. Additionally, we investigate the perceived importance, impact, and causes of duplicate bugs, as well as the difference between nonduplicate and duplicate bugs and its categories for practitioners in the software industry by conducting a survey.

Are duplicates really harmful? An empirical study on bug report summarization techniques

AbstractRecent research works have proven that duplicate bug reports can provide helpful information to assist developers in software tasks such as fault localization and program fixing, while thoroughly reading duplicate bug reports is time‐consuming and inefficient. Summarization is a possible solution for gaining essential information quickly. However, there are many challenges when applying existing summarizing techniques on duplicate bug reports. Duplicate bug reports describe the same problem from different views and vary in quality, content, and writing style. Moreover, the code snippet understanding and the semantic gap between natural and programming languages make the summary generation even more difficult. Thus, in this paper, we want to investigate whether the state‐of‐the‐art summarization approaches can overcome the resistance and generate an effective summary for duplicate bug reports. We collected more than 8,000 groups of duplicate reports from GitHub and labeled 60 groups with 149 reports manually for the evaluation. Results showed that although the existing summarization approaches can work on duplicate bug reports, there are significant differences between them when it comes to code snippet summarization. Moreover, several methods can be very sluggish for summarizing long bug reports. Our study provides insights and guidelines for choosing proper summarization approaches in different scenarios.

Bug Reports and Deep Learning Models

Deep Learning is one of the emerging and trending research area of machine learning in various domains. The paper describes the deep learning approaches applied to the domain of Bug Reports. The paper classifies the tasks being performed for mining of Bug Reports into Bug Report Classification, Bug Localization, Bug Report Summarization and Duplicate Bug Report Detection. The paper systematically discusses about the deep learning approaches being used for the mentioned tasks, and the future directions in this field of research.

A Systematic Study of Duplicate Bug Report Detection

Defects are an integral part of any software project. They can arise at any time, at any phase of the software development or the maintenance phase. In open source projects, open bug repositories are used to maintain the bug reports. When a new bug report arrives, a person called “Triager” analyzes the bug report and assign it to some responsible developer. But before assigning, has to look if it is duplicate or not. Duplicate Bug Report is one of the big problems in the maintenance of bug repositories. Lack of knowledge and vocabulary skills of reporters sometimes increases the effort required for this purpose. Bug Tracking Systems are usually used to maintain the bug reports and are the most consulted resource during the maintenance process. Because of the Uncoordinated nature of the submission of bug reports to the tracking system, many times the same bug report is reported by many users. Duplicate Bug Reports lead to the waste of resources and the economy. It creates problems for triagers and requires a lot of analysis and validation. Lot of work has been done in the field of duplicate bug report detection. In this paper, we present the researches systematically done in this field by classifying the works into three categories and listing down the methods being used for the classified researches. The paper considers the papers till January 2020 for the analysis purpose. The paper mentions the strengths, limitations, data set, and the major approach used by the popular papers of the research in this field. The paper also lists the challenges and future directions in this field of research.

SiameseQAT: A Semantic Context-Based Duplicate Bug Report Detection Using Replicated Cluster Information

In large-scale software development environments, defect reports are maintained through bug tracking systems (BTS) and analyzed by domain experts. Different users may create bug reports in a non-standard manner and may report a particular problem using a particular set of words due to stylistic choices and writing patterns. Therefore, the same defect can be reported with very different descriptions, generating non-trivial duplicates. To avoid redundant work for the development team, an expert needs to look at all new reports while trying to label possible duplicates. However, this approach is neither trivial nor scalable and directly impacts bug fix correction time. Recent efforts to find duplicate bug reports tend to focus on deep neural approaches that consider hybrid representations of bug reports, using both structured and unstructured information. Unfortunately, these approaches ignore that a single bug can have multiple previously identified duplicates and, therefore, multiple textual descriptions, titles, and categorical information. In this work, we propose SiameseQAT, a duplicate bug report detection method that considers information on individual bugs as well as information extracted from bug clusters. The SiameseQAT combines context and semantic learning on structured and unstructured features and corpus topic extraction-based features, with a novel loss function called Quintet Loss, which considers the centroid of duplicate clusters and their contextual information. We validated our approach on the well-known open-source software repositories Eclipse, NetBeans, and Open Office, comprised of more than 500 thousand bug reports. We evaluated both the retrieval and classification of duplicates, reporting a Recall@25 mean of 85% for retrieval and 84% AUROC for classification tasks, results that were significantly superior to previous works.

A Systematic Comparison of search-Based approaches for LDA hyperparameter tuning

Context: Latent Dirichlet Allocation (LDA) has been successfully used in the literature to extract topics from software documents and support developers in various software engineering tasks. While LDA has been mostly used with default settings, previous studies showed that default hyperparameter values generate sub-optimal topics from software documents. Objective: Recent studies applied meta-heuristic search (mostly evolutionary algorithms) to configure LDA in an unsupervised and automated fashion. However, previous work advocated for different meta-heuristics and surrogate metrics to optimize. The objective of this paper is to shed light on the influence of these two factors when tuning LDA for SE tasks. Method: We empirically evaluated and compared seven state-of-the-art meta-heuristics and three alternative surrogate metrics (i.e., fitness functions) to solve the problem of identifying duplicate bug reports with LDA. The benchmark consists of ten real-world and open-source projects from the Bench4BL dataset. Results: Our results indicate that (1) meta-heuristics are mostly comparable to one another (except for random search and CMA-ES), and (2) the choice of the surrogate metric impacts the quality of the generated topics and the tuning overhead. Furthermore, calibrating LDA helps identify twice as many duplicates than untuned LDA when inspecting the top five past similar reports. Conclusion: No meta-heuristic and/or fitness function outperforms all the others, as advocated in prior studies. However, we can make recommendations for some combinations of meta-heuristics and fitness functions over others for practical use. Future work should focus on improving the surrogate metrics used to calibrate/tune LDA in an unsupervised fashion.

Efficient feature extraction model for validation performance improvement of duplicate bug report detection in software bug triage systems

Abstract Context There are many duplicate bug reports in the semi-structured software repository of various software bug triage systems. The duplicate bug report detection (DBRD) process is a significant problem in software triage systems. Objective The DBRD problem has many issues, such as efficient feature extraction to calculate similarities between bug reports accurately, building a high-performance duplicate detector model, and handling continuous real-time queries. Feature extraction is a technique that converts unstructured data to structured data. The main objective of this study is to improve the validation performance of DBRD using a feature extraction model. Method This research focuses on feature extraction to build a new general model containing all types of features. Moreover, it introduces a new feature extractor method to describe a new viewpoint of similarity between texts. The proposed method introduces new textual features based on the aggregation of term frequency and inverse document frequency of text fields of bug reports in uni-gram and bi-gram forms. Further, a new hybrid measurement metric is proposed for detecting efficient features, whereby it is used to evaluate the efficiency of all features, including the proposed ones. Results The validation performance of DBRD was compared for the proposed features and state-of-the-art features. To show the effectiveness of our model, we applied it and other related studies to DBRD of the Android, Eclipse, Mozilla, and Open Office datasets and compared the results. The comparisons showed that our proposed model achieved (i) approximately 2% improvement for accuracy and precision and more than 4.5% and 5.9% improvement for recall and F1-measure, respectively, by applying the linear regression (LR) and decision tree (DT) classifiers and (ii) a performance of 91%−99% (average ~97%) for the four metrics, by applying the DT classifier as the best classifier. Conclusion Our proposed features improved the validation performance of DBRD concerning runtime performance. The pre-processing methods (primarily stemming) could improve the validation performance of DBRD slightly (up to 0.3%), but rule-based machine learning algorithms are more useful for the DBRD problem. The results showed that our proposed model is more effective both for the datasets for which state-of-the-art approaches were effective (i.e., Mozilla Firefox) and those for which state-of-the-art approaches were less effective (i.e., Android). The results also showed that the combination of all types of features could improve the validation performance of DBRD even for the LR classifier with less validation performance, which can be implemented easily for software bug triage systems. Without using the longest common subsequence (LCS) feature, which is effective but time-consuming, our proposed features could cover the effectiveness of LCS with lower time-complexity and runtime overhead. In addition, a statistical analysis shows that the results are reliable and can be generalized to other datasets or similar classifiers.

Duplicate Bug Report Detection and Classification System Based on Deep Learning Technique

Duplicate bug report detection is a process of finding a duplicate bug report in the bug tracking system. This process is essential to avoid unnecessary work and rediscovery. In typical bug tracking systems, more than thousands of duplicate bug reports are reported every day. In turn, human cost, effort and time are increased. This makes it an important problem in the software management process. The solution is to automate the duplicate bug report detection system for reducing the manual effort, thus the productivity of triager's and developer's is increased. It also speeds up the process of software management as a result software maintenance cost is also reduced. However, existing systems are not quite accurate yet, in spite of these systems used various machine learning approaches. In this work, an automatic bug report detection and classification model is proposed using deep learning technique. The proposed system has three modules i.e. Preprocessing, Deep Learning Model and Duplicate Bug report Detection and Classification. Further, the proposed model used Convolutional Neural Network based deep learning model to extract relevant feature. These relevant features are used to determine the similar features of bug reports. Hence, the bug reports similarity is computers through these similar features. The performance of the proposed system is evaluated on six publicly available datasets using six performance metrics. It is noticed that the proposed system outperforms the existing systems by achieving an accuracy rate in the range of 85% to 99 % and recall@k rate in between 79%-94%. Moreover, the effectiveness of the proposed system is also measured on the cross training datasets of same and different domain. The proposed system achieves a good high accuracy rate for same domain data sets and low accuracy rate for different domain datasets.

Static duplicate bug-report identification for compilers

Compiler bug reports are important for guaranteeing compiler quality; however, duplicate bug reports tend to incur extra costs. To identify duplicate bug reports for compilers, we propose a static approach (IdenDup) to identifying duplicate bug reports for compilers. This method effectively identifies duplicate bug reports for compilers in two scenarios (fuzz testing and the bug-management system) by utilizing static text and program information, including lexical features, syntax features, and proposed dataflow features that describe variable-usage path features (i.e., how variables are used and their order). We conducted empirical evaluations of the effectiveness of IdenDup based on the use of GCC and LLVM, with our results demonstrating that IdenDup effectively identified duplicate bug reports in the two scenarios for compilers and outperformed existing approaches.

IMPROVING DETECTION PERFORMANCE OF DUPLICATE BUG REPORTS USING EXTENDED CLASS CENTROID INFORMATION

In software maintenance, bug reports play an important role in the correctness of software packages. Unfortunately, the duplicatebug report problem arises because there are too many duplicate bug reports in various software projects. Handling with duplicate bug reports is thus time-consuming and has high cost of software maintenance. Therefore, this research introduces a detection scheme based on the extended class centroid information (ECCI) to enhance thedetection performance. This method is extended from the previous one, which used only centroid method without considering the effects of both inner and inter class. Besides, this method also improved the previous use of normalized cosine in identifying the similarity between two bug reports by denormalized cosine. The effectiveness of ECCI is proved through the empirical study with three open-source projects: SVN, Argo UML and Apache. The experimental results show thatECCI outperforms other detection schemes by about 10% in all cases.

MULTI-FEATURE MODEL FOR DUPLICATE BUG REPORTS DETECTION SOLUTION

For open source software such as Firefox, Eclipse, Subversion,. . . they usually have a system for bug management that sent by users. These bug reports help the system identify various software bugs which makes software maintenance better. However, a situation occurs that there are many error reports sent to the processing repository that these bug reports were previously reported by different users, this is called duplicate bug reports. In this paper, we introduce a multi-feature model combined with weighted improvements from CFC (ClassFeature-Centroid) to detect duplicate bug reportsautomatically. We have experimented on three projects of Mozilla, Eclipse and OpenOffice. The results show that our method can improve 8-12% better as compared to the compared methods.

An HMM-based approach for automatic detection and classification of duplicate bug reports

Abstract Context Software projects rely on their issue tracking systems to guide maintenance activities of software developers. Bug reports submitted to the issue tracking systems carry crucial information about the nature of the crash (such as texts from users or developers and execution information about the running functions before the occurrence of a crash). Typically, big software projects receive thousands of reports every day. Objective The aim is to reduce the time and effort required to fix bugs while improving software quality overall. Previous studies have shown that a large amount of bug reports are duplicates of previously reported ones. For example, as many as 30% of all reports in for Firefox are duplicates. Method While there exist a wide variety of approaches to automatically detect duplicate bug reports by natural language processing, only a few approaches have considered execution information (the so-called stack traces) inside bug reports. In this paper, we propose a novel approach that automatically detects duplicate bug reports using stack traces and Hidden Markov Models. Results When applying our approach to Firefox and GNOME datasets, we show that, for Firefox, the average recall for Rank k = 1 is 59%, for Rank k = 2 is 75.55%. We start reaching the 90% recall from k = 10. The Mean Average Precision (MAP) value is up to 76.5%. For GNOME, The recall at k = 1 is around 63%, while this value increases by about 10% for k = 2. The recall increases to 97% for k = 11. A MAP value of up to 73% is achieved. Conclusion We show that HMM and stack traces are a powerful combination for detecting and classifying duplicate bug reports in large bug repositories.