Duplicate Bug Report Detection Research Articles

Duplicate Bug Report Detection Using an Attention-Based Neural Language Model

<i>Context:</i> Users and developers use bug tracking systems to report errors that occur during the development and testing of software. The manual identification of duplicates is a tedious task especially with software that have large bug repositories. In this context, their automatic detection becomes a necessary task that can help prevent frequently fixing the same bug. <i>Objective:</i> In this article, we propose <i>BERT-MLP</i>, a novel pretrained language model using bidirectional encoder representations from ransformers (BERT) for duplicate bug report detection (DBRD) with the aim of improving the detection rate compared to existing works. <i>Method:</i> Our approach considers only unstructured data. These are fed into the BERT model in order to learn the contextual relationships between words. The output is fed into a multilayer perceptron (MLP) classifier, representing our base DBRD. <i>Results:</i> Our approach was evaluated on three projects: Mozilla Firefox, Eclipse Platform, and Thunderbird. It achieved an accuracy of 92.11, 94.08, and 89.03&#x0025;, respectively, for Mozilla, Eclipse, and Thunderbird. A comparison with a dual-channel convolutional neural network (DC-CNN) model and other pretrained models, including RoBERTa and Sentence-Bert has been conducted. Results showed that <i>BERT-MLP</i> outperformed, the second best performing models (DC-CNN and Sentence-BERT) by 12&#x0025; in accuracy for Eclipse and 9&#x0025; for both Mozilla and Thunderbird, respectively.

Duplicate Bug Report Detection: How Far Are We?

Many Duplicate Bug Report Detection (DBRD) techniques have been proposed in the research literature. The industry uses some other techniques. Unfortunately, there is insufficient comparison among them, and it is unclear how far we have been. This work fills this gap by comparing the aforementioned techniques. To compare them, we first need a benchmark that can estimate how a tool would perform if applied in a realistic setting today. Thus, we first investigated potential biases that affect the fair comparison of the accuracy of DBRD techniques. Our experiments suggest that data age and issue tracking system (ITS) choice cause a significant difference. Based on these findings, we prepared a new benchmark. We then used it to evaluate DBRD techniques to estimate better how far we have been. Surprisingly, a simpler technique outperforms recently proposed sophisticated techniques on most projects in our benchmark. In addition, we compared the DBRD techniques proposed in research with those used in Mozilla and VSCode . Surprisingly, we observe that a simple technique already adopted in practice can achieve comparable results as a recently proposed research tool. Our study gives reflections on the current state of DBRD, and we share our insights to benefit future DBRD research.

Sentence embedding and fine-tuning to automatically identify duplicate bugs

Industrial software maintenance is critical but burdensome. Activities such as detecting duplicate bug reports are often performed manually. Herein an automated duplicate bug report detection system improves maintenance efficiency using vectorization of the contents and deep learning–based sentence embedding to calculate the similarity of the whole report from vectors of individual elements. Specifically, sentence embedding is realized using Sentence-BERT fine tuning. Additionally, its performance is experimentally compared to baseline methods to validate the proposed system. The proposed system detects duplicate bug reports more effectively than existing methods.

Leveraging multi-level embeddings for knowledge-aware bug report reformulation

Software bug analysis based on the information retrieval (IR) technology is widely studied and used for bug understanding, localization and fixing. IR technology with various textual feature extraction methods formulates the textual information in a given new bug report (i.e., title and description) as an initial query. However, due to the low-quality content in the new bug report and improper representation to be used as a query, the retrieval results are usually not satisfactory.To alleviate these problems, we propose a novel knowledge-aware bug report reformulation approach (a.k.a, KABR) by leveraging multi-level embeddings from the bug data. First, we construct a bug-specific knowledge graph (KG) to manage and reuse prior knowledge extracted from historical bug reports. Then, we extract word embedding from the original bug data, entity embedding and context embedding from the bug-specific KG to enhance the initial query. Finally, a new query representation is generated by leveraging multi-level embeddings through Convolutional Neural Networks (CNN) with the self-attention mechanism. We evaluate KABR based on the duplicate bug report detection task, and the experimental results show that KABR achieves 6%–11% F1-measure improvement over the state-of-the-art approaches.

Does Deep Learning improve the performance of duplicate bug report detection? An empirical study

Do Deep Learning (DL) techniques actually help to improve the performance of duplicate bug report detection? Prior studies suggest that they do, if the duplicate bug report detection task is treated as a binary classification problem. However, in realistic scenarios, the task is often viewed as a ranking problem, which predicts potential duplicate bug reports by ranking based on similarities with existing historical bug reports. There is little empirical evidence to support that DL can be effectively applied to detect duplicate bug reports in the ranking scenario. Therefore, in this paper, we investigate whether well-known DL-based methods outperform classic information retrieval (IR) based methods on the duplicate bug report detection task. In addition, we argue that both IR- and DL-based methods suffer from incompletely evaluating the similarity between bug reports, resulting in the loss of important information. To address this problem, we propose a new method that combines IR and DL techniques to compute textual similarity more comprehensively. Our experimental results show that the DL-based method itself does not yield high performance compared to IR-based methods. However, our proposed combined method improves on the MAP metric of classic IR-based methods by a median of 7.09%–11.34% and a maximum of 17.228%–28.97%.

Adaptive Deep Learning Model for Software Bug Detection and Classification

Software is unavoidable in software development and maintenance. In literature, many methods are discussed which fails to achieve efficient software bug detection and classification. In this paper, efficient Adaptive Deep Learning Model (ADLM) is developed for automatic duplicate bug report detection and classification process. The proposed ADLM is a combination of Conditional Random Fields decoding with Long Short-Term Memory (CRF-LSTM) and Dingo Optimizer (DO). In the CRF, the DO can be consumed to choose the efficient weight value in network. The proposed automatic bug report detection is proceeding with three stages like pre-processing, feature extraction in addition bug detection with classification. Initially, the bug report input dataset is gathered from the online source system. In the pre-processing phase, the unwanted information from the input data are removed by using cleaning text, convert data types and null value replacement. The pre-processed data is sent into the feature extraction phase. In the feature extraction phase, the four types of feature extraction method are utilized such as contextual, categorical, temporal and textual. Finally, the features are sent to the proposed ADLM for automatic duplication bug report detection and classification. The proposed methodology is proceeding with two phases such as training and testing phases. Based on the working process, the bugs are detected and classified from the input data. The projected technique is assessed by analyzing performance metrics such as accuracy, precision, Recall, F_Measure and kappa.

Bug Reports and Deep Learning Models

Deep Learning is one of the emerging and trending research area of machine learning in various domains. The paper describes the deep learning approaches applied to the domain of Bug Reports. The paper classifies the tasks being performed for mining of Bug Reports into Bug Report Classification, Bug Localization, Bug Report Summarization and Duplicate Bug Report Detection. The paper systematically discusses about the deep learning approaches being used for the mentioned tasks, and the future directions in this field of research.

A Systematic Study of Duplicate Bug Report Detection

Defects are an integral part of any software project. They can arise at any time, at any phase of the software development or the maintenance phase. In open source projects, open bug repositories are used to maintain the bug reports. When a new bug report arrives, a person called “Triager” analyzes the bug report and assign it to some responsible developer. But before assigning, has to look if it is duplicate or not. Duplicate Bug Report is one of the big problems in the maintenance of bug repositories. Lack of knowledge and vocabulary skills of reporters sometimes increases the effort required for this purpose. Bug Tracking Systems are usually used to maintain the bug reports and are the most consulted resource during the maintenance process. Because of the Uncoordinated nature of the submission of bug reports to the tracking system, many times the same bug report is reported by many users. Duplicate Bug Reports lead to the waste of resources and the economy. It creates problems for triagers and requires a lot of analysis and validation. Lot of work has been done in the field of duplicate bug report detection. In this paper, we present the researches systematically done in this field by classifying the works into three categories and listing down the methods being used for the classified researches. The paper considers the papers till January 2020 for the analysis purpose. The paper mentions the strengths, limitations, data set, and the major approach used by the popular papers of the research in this field. The paper also lists the challenges and future directions in this field of research.

SiameseQAT: A Semantic Context-Based Duplicate Bug Report Detection Using Replicated Cluster Information

In large-scale software development environments, defect reports are maintained through bug tracking systems (BTS) and analyzed by domain experts. Different users may create bug reports in a non-standard manner and may report a particular problem using a particular set of words due to stylistic choices and writing patterns. Therefore, the same defect can be reported with very different descriptions, generating non-trivial duplicates. To avoid redundant work for the development team, an expert needs to look at all new reports while trying to label possible duplicates. However, this approach is neither trivial nor scalable and directly impacts bug fix correction time. Recent efforts to find duplicate bug reports tend to focus on deep neural approaches that consider hybrid representations of bug reports, using both structured and unstructured information. Unfortunately, these approaches ignore that a single bug can have multiple previously identified duplicates and, therefore, multiple textual descriptions, titles, and categorical information. In this work, we propose SiameseQAT, a duplicate bug report detection method that considers information on individual bugs as well as information extracted from bug clusters. The SiameseQAT combines context and semantic learning on structured and unstructured features and corpus topic extraction-based features, with a novel loss function called Quintet Loss, which considers the centroid of duplicate clusters and their contextual information. We validated our approach on the well-known open-source software repositories Eclipse, NetBeans, and Open Office, comprised of more than 500 thousand bug reports. We evaluated both the retrieval and classification of duplicates, reporting a Recall@25 mean of 85% for retrieval and 84% AUROC for classification tasks, results that were significantly superior to previous works.

Efficient feature extraction model for validation performance improvement of duplicate bug report detection in software bug triage systems

Abstract Context There are many duplicate bug reports in the semi-structured software repository of various software bug triage systems. The duplicate bug report detection (DBRD) process is a significant problem in software triage systems. Objective The DBRD problem has many issues, such as efficient feature extraction to calculate similarities between bug reports accurately, building a high-performance duplicate detector model, and handling continuous real-time queries. Feature extraction is a technique that converts unstructured data to structured data. The main objective of this study is to improve the validation performance of DBRD using a feature extraction model. Method This research focuses on feature extraction to build a new general model containing all types of features. Moreover, it introduces a new feature extractor method to describe a new viewpoint of similarity between texts. The proposed method introduces new textual features based on the aggregation of term frequency and inverse document frequency of text fields of bug reports in uni-gram and bi-gram forms. Further, a new hybrid measurement metric is proposed for detecting efficient features, whereby it is used to evaluate the efficiency of all features, including the proposed ones. Results The validation performance of DBRD was compared for the proposed features and state-of-the-art features. To show the effectiveness of our model, we applied it and other related studies to DBRD of the Android, Eclipse, Mozilla, and Open Office datasets and compared the results. The comparisons showed that our proposed model achieved (i) approximately 2% improvement for accuracy and precision and more than 4.5% and 5.9% improvement for recall and F1-measure, respectively, by applying the linear regression (LR) and decision tree (DT) classifiers and (ii) a performance of 91%−99% (average ~97%) for the four metrics, by applying the DT classifier as the best classifier. Conclusion Our proposed features improved the validation performance of DBRD concerning runtime performance. The pre-processing methods (primarily stemming) could improve the validation performance of DBRD slightly (up to 0.3%), but rule-based machine learning algorithms are more useful for the DBRD problem. The results showed that our proposed model is more effective both for the datasets for which state-of-the-art approaches were effective (i.e., Mozilla Firefox) and those for which state-of-the-art approaches were less effective (i.e., Android). The results also showed that the combination of all types of features could improve the validation performance of DBRD even for the LR classifier with less validation performance, which can be implemented easily for software bug triage systems. Without using the longest common subsequence (LCS) feature, which is effective but time-consuming, our proposed features could cover the effectiveness of LCS with lower time-complexity and runtime overhead. In addition, a statistical analysis shows that the results are reliable and can be generalized to other datasets or similar classifiers.

Duplicate Bug Report Detection and Classification System Based on Deep Learning Technique

Duplicate bug report detection is a process of finding a duplicate bug report in the bug tracking system. This process is essential to avoid unnecessary work and rediscovery. In typical bug tracking systems, more than thousands of duplicate bug reports are reported every day. In turn, human cost, effort and time are increased. This makes it an important problem in the software management process. The solution is to automate the duplicate bug report detection system for reducing the manual effort, thus the productivity of triager's and developer's is increased. It also speeds up the process of software management as a result software maintenance cost is also reduced. However, existing systems are not quite accurate yet, in spite of these systems used various machine learning approaches. In this work, an automatic bug report detection and classification model is proposed using deep learning technique. The proposed system has three modules i.e. Preprocessing, Deep Learning Model and Duplicate Bug report Detection and Classification. Further, the proposed model used Convolutional Neural Network based deep learning model to extract relevant feature. These relevant features are used to determine the similar features of bug reports. Hence, the bug reports similarity is computers through these similar features. The performance of the proposed system is evaluated on six publicly available datasets using six performance metrics. It is noticed that the proposed system outperforms the existing systems by achieving an accuracy rate in the range of 85% to 99 % and recall@k rate in between 79%-94%. Moreover, the effectiveness of the proposed system is also measured on the cross training datasets of same and different domain. The proposed system achieves a good high accuracy rate for same domain data sets and low accuracy rate for different domain datasets.

An HMM-based approach for automatic detection and classification of duplicate bug reports

Abstract Context Software projects rely on their issue tracking systems to guide maintenance activities of software developers. Bug reports submitted to the issue tracking systems carry crucial information about the nature of the crash (such as texts from users or developers and execution information about the running functions before the occurrence of a crash). Typically, big software projects receive thousands of reports every day. Objective The aim is to reduce the time and effort required to fix bugs while improving software quality overall. Previous studies have shown that a large amount of bug reports are duplicates of previously reported ones. For example, as many as 30% of all reports in for Firefox are duplicates. Method While there exist a wide variety of approaches to automatically detect duplicate bug reports by natural language processing, only a few approaches have considered execution information (the so-called stack traces) inside bug reports. In this paper, we propose a novel approach that automatically detects duplicate bug reports using stack traces and Hidden Markov Models. Results When applying our approach to Firefox and GNOME datasets, we show that, for Firefox, the average recall for Rank k = 1 is 59%, for Rank k = 2 is 75.55%. We start reaching the 90% recall from k = 10. The Mean Average Precision (MAP) value is up to 76.5%. For GNOME, The recall at k = 1 is around 63%, while this value increases by about 10% for k = 2. The recall increases to 97% for k = 11. A MAP value of up to 73% is achieved. Conclusion We show that HMM and stack traces are a powerful combination for detecting and classifying duplicate bug reports in large bug repositories.

Preventing duplicate bug reports by continuously querying bug reports

Bug deduplication or duplicate bug report detection is a hot topic in software engineering information retrieval research, but it is often not deployed. Typically to de-duplicate bug reports developers rely upon the search capabilities of the bug report software they employ, such as Bugzilla, Jira, or Github Issues. These search capabilities range from simple SQL string search to IR-based word indexing methods employed by search engines. Yet too often these searches do very little to stop the creation of duplicate bug reports. Some bug trackers have more than 10% of their bug reports marked as duplicate. Perhaps these bug tracker search engines are not enough? In this paper we propose a method of attempting to prevent duplicate bug reports before they start: continuously querying. That is as the bug reporter types in their bug report their text is used to query the bug database to find duplicate or related bug reports. This continuously querying bug reports allows the reporter to be alerted to duplicate bug reports as they report the bug, rather than formulating queries to find the duplicate bug report. Thus this work ushers in a new way of evaluating bug report deduplication techniques, as well as a new kind of bug deduplication task. We show that simple IR measures can address this problem but also that further research is needed to refine this novel process that is integrate-able into modern bug report systems.

A contextual approach towards more accurate duplicate bug report detection and ranking

The issue-tracking systems used by software projects contain issues, bugs, or tickets written by a wide variety of bug reporters, with different levels of training and knowledge about the system under development. Typically, reporters lack the skills and/or time to search the issue-tracking system for similar issues already reported. As a result, many reports end up referring to the same issue, which effectively makes the bug-report triaging process time consuming and error prone. Many researchers have approached the bug-deduplication problem using off-the-shelf information-retrieval (IR) tools. In this work, we extend the state of the art by investigating how contextual information about software-quality attributes, software-architecture terms, and system-development topics can be exploited to improve bug deduplication. We demonstrate the effectiveness of our contextual bug-deduplication method at ranking duplicates on the bug repositories of the Android, Eclipse, Mozilla, and OpenOffice software systems. Based on this experience, we conclude that taking into account domain-specific context can improve IR methods for bug deduplication.

An Empirical Comparison of Machine Learning Techniques in Predicting the Bug Severity of Open and Closed Source Projects

Bug severity is the degree of impact that a defect has on the development or operation of a component or system, and can be classified into different levels based on their impact on the system. Identification of severity level can be useful for bug triager in allocating the bug to the concerned bug fixer. Various researchers have attempted text mining techniques in predicting the severity of bugs, detection of duplicate bug reports and assignment of bugs to suitable fixer for its fix. In this paper, an attempt has been made to compare the performance of different machine learning techniques namely Support vector machine (SVM), probability based Naïve Bayes (NB), Decision Tree based J48 (A Java implementation of C4.5), rule based Repeated Incremental Pruning to Produce Error Reduction (RIPPER) and Random Forests (RF) learners in predicting the severity level (1 to 5) of a reported bug by analyzing the summary or short description of the bug reports. The bug report data has been taken from NASA’s PITS (Projects and Issue Tracking System) datasets as closed source and components of Eclipse, Mozilla &amp; GNOME datasets as open source projects. The analysis has been carried out in RapidMiner and STATISTICA data mining tools. The authors measured the performance of different machine learning techniques by considering (i) the value of accuracy and F-Measure for all severity level and (ii) number of best cases at different threshold level of accuracy and F-Measure.