Bug Report Classification Research Articles

An empirical evaluation of stacked generalization models for binary bug report classification

An empirical evaluation of stacked generalization models for binary bug report classification

LLM-BRC: A large language model-based bug report classification framework

LLM-BRC: A large language model-based bug report classification framework

Code quality control by bug report classification

Code quality control by bug report classification

An Empirical Study on the Classification of Bug Reports Using Machine Learning Algorithms - Supplementary Material

An Empirical Study on the Classification of Bug Reports Using Machine Learning Algorithms - Supplementary Material

Classification of open source software bug report based on transfer learning

AbstractCurrently, the feature richness of text encoding vectors in the bug report classification model based on deep learning is limited by the size of the domain dataset and the quality of the text. However, it is difficult to further enrich the features of text encoding vectors. At the same time, most existing bug report classification methods ignore the submitter's personal information. To solve these problems, we construct nine personal information characteristics of bug report submitters in GitHub by survey. Then, we propose a GitHub bug report classification method named personal information fine‐tuning network (PIFTNet) based on transfer learning and the submitter's personal information. PIFTNet transfers the general text feature vectors in bidirectional encoder representation from transformers (BERT) to the domain of bug report classification by fine‐tuning the pre‐training parameters in BERT. It also combines the text characteristics and the characteristics of the submitter's personal information to construct the classification model. In addition, we propose a two‐stage training method to alleviate the catastrophic changes in the pre‐training parameters and loss of the initially learned knowledge caused by direct training of PIFTNet. We verify the proposed PIFTNet on the dataset extracted from GitHub and empirical results prove the effectiveness of PIFTNet.

Temporal dynamics of requirements engineering from mobile app reviews.

Opinion mining for app reviews aims to analyze people’s comments from app stores to support data-driven requirements engineering activities, such as bug report classification, new feature requests, and usage experience. However, due to a large amount of textual data, manually analyzing these comments is challenging, and machine-learning-based methods have been used to automate opinion mining. Although recent methods have obtained promising results for extracting and categorizing requirements from users’ opinions, the main focus of existing studies is to help software engineers to explore historical user behavior regarding software requirements. Thus, existing models are used to support corrective maintenance from app reviews, while we argue that this valuable user knowledge can be used for preventive software maintenance. This paper introduces the temporal dynamics of requirements analysis to answer the following question: how to predict initial trends on defective requirements from users’ opinions before negatively impacting the overall app’s evaluation? We present the MAPP-Reviews (Monitoring App Reviews) method, which (i) extracts requirements with negative evaluation from app reviews, (ii) generates time series based on the frequency of negative evaluation, and (iii) trains predictive models to identify requirements with higher trends of negative evaluation. The experimental results from approximately 85,000 reviews show that opinions extracted from user reviews provide information about the future behavior of an app requirement, thereby allowing software engineers to anticipate the identification of requirements that may affect the future app’s ratings.

Bug Reports and Deep Learning Models

Deep Learning is one of the emerging and trending research area of machine learning in various domains. The paper describes the deep learning approaches applied to the domain of Bug Reports. The paper classifies the tasks being performed for mining of Bug Reports into Bug Report Classification, Bug Localization, Bug Report Summarization and Duplicate Bug Report Detection. The paper systematically discusses about the deep learning approaches being used for the mentioned tasks, and the future directions in this field of research.

Automated Classification of Unstructured Bilingual Software Bug Reports: An Industrial Case Study Research

Software bug report classification is a critical process to understand the nature, implications, and causes of software failures. Furthermore, classification enables a fast and appropriate reaction to software bugs. However, for large-scale projects, one must deal with a broad set of bugs from multiple types. In this context, manually classifying bugs becomes cumbersome and time-consuming. Although several studies have addressed automated bug classification using machine learning techniques, they have mainly focused on academic case studies, open-source software, and unilingual text input. This paper presents our automated bug classification approach applied and validated in an industrial case study. In contrast to earlier studies, our study is applied to a commercial software system based on unstructured bilingual bug reports written in English and Turkish. The presented approach adopts and integrates machine learning (ML), text mining, and natural language processing (NLP) techniques to support the classification of software bugs. The approach has been applied within an industrial case study. Compared to manual classification, our results show that bug classification can be automated and even performs better than manual bug classification. Our study shows that the presented approach and the corresponding tools effectively reduce the manual classification time and effort.

A Comparative Study of Class Rebalancing Methods for Security Bug Report Classification

Identifying security bug reports (SBRs) accurately from a bug repository can reduce a software product’s security risk. However, the class imbalance problem exists for SBR prediction since the number of SBRs is often limited, and this issue has not been thoroughly investigated in previous studies. In our study, we choose six real-world projects of different sizes with over 120 000 bug reports in total as our empirical subjects. We first analyze the impact of the class imbalance issue on SBR prediction and confirm its negative impact on prediction performance. Then we perform a comparative study of six state-of-the-art class rebalancing methods combined with five popular classification algorithms for SBR prediction. By comparing with the baseline method Farsec, using the class rebalancing methods can improve the performance in 78% of cases in the worst case. Moreover, the combination of the Rose and random forest classification algorithm can construct the model with the best performance, which increases the performance by 267% in the best case and 75% on average in terms of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F1-score</i> . Finally, we summarize eight main findings based on our empirical studies’ results, which can provide guidelines for choosing appropriate class rebalancing methods and classifiers for SBR prediction in practice.

On the classification of bug reports to improve bug localization

On the classification of bug reports to improve bug localization

Predicting Bug Priority Using Topic Modelling in Imbalanced Learning Environments

Manual classification of bug reports is time-consuming as the reports are received in large quantities. Alternatively, this project proposed automatic bug prediction models to classify the bug reports. The topics or the candidate keywords are mined from the developer description in bug reports using RAKE algorithm and converted into attributes. These attributes together with the target attribute—priority level—construct the training datasets. Naïve Bayes, logistic regression, and decision tree learner algorithms are trained, and the prediction quality was measured using area under recursive operative characteristics curves (AUC) as AUC does not consider the biasness in datasets. The logistics regression model outperforms the other two models providing the accuracy of 0.86 AUC whereas the naïve Bayes and the decision tree learner recorded 0.79 AUC and 0.81 AUC, respectively. The bugs can be classified without developer involvement and logistic regression is also a potential candidate as naïve Bayes for bug classification.

ProTeC: Production and Test Bug Report Classification Based on Deep Transfer Learning

ProTeC: Production and Test Bug Report Classification Based on Deep Transfer Learning

Invalid bug reports complicate the software aging situation

Symptoms of software aging include performance degradation and failure occurrence increasing when software systems run for a period of time. Therefore, software aging is closely related to system performance. Understanding and analyzing performance issues in the software system is critical to mastering software aging information. Instead of focusing on normal valid bug reports (VBRs), this paper advocates the usage of invalid bug reports (IBRs) to capture software aging signals. We use performance bugs that are highly related to software aging as an example to construct a binary classification model for bug report classification. We conduct a rigorous evaluation of the constructed models via different performance measures (i.e., recall, precision, F1-score, AUC). Then, the model is used to predict the performance bug reports (PBRs) in IBRs, and a manual analysis of the prediction results is conducted to identify aging-related bug reports (ABRs). The final results show that the ratio of PBRs in IBRs ranges from 4.9 to 42.18% for the two real open-source projects HDFS and HBase when considering five different classifiers. Among these five classifiers, Support Vector Machine (SVM) classifier can achieve the best performance. The ratios of PBRs in IBRs by using this classifier are 11.1% and 15.35% for these two datasets and the performances in terms of F1-score are 85% and 74%. Further analysis of the predicted PBRs of IBRs in the project HDFS is conducted through a manual user case study; some surprising findings revealing the relationship between IBRs, PBRs, and ABRs are presented: (1) Around 50% of the PBRs in IBRs are related to software aging; (2) components that undertake major tasks are more prone to aging problems; (3) more than 50% ARBs lead to timeout, 33% ARBs are caused by improper control of memory or threats, and 29% ARBs are caused by inappropriate management of file operation or disk usage; (4) hard to reproduce is the major reason that ARBs are usually closed as invalid because many aging-related bugs would temporarily disappear by restarting the system.

Identify Severity Bug Report with Distribution Imbalance by CR-SMOTE and ELM

Manually inspecting bugs to determine their severity is often an enormous but essential software development task, especially when many participants generate a large number of bug reports in a crowdsourced software testing context. Therefore, boosting the capabilities of methods of predicting bug report severity is critically important for determining the priority of fixing bugs. However, typical classification techniques may be adversely affected when the severity distribution of the bug reports is imbalanced, leading to performance degradation in a crowdsourcing environment. In this study, we propose an enhanced oversampling approach called CR-SMOTE to enhance the classification of bug reports with a realistically imbalanced severity distribution. The main idea is to interpolate new instances into the minority category that are near the center of existing samples in that category. Then, we use an extreme learning machine (ELM) — a feedforward neural network with a single layer of hidden nodes — to predict the bug severity. Several experiments were conducted on three datasets from real bug repositories, and the results statistically indicate that the presented approach is robust against real data imbalance when predicting the severity of bug reports. The average accuracies achieved by the ELM in predicting the severity of Eclipse, Mozilla, and GNOME bug reports were 0.780, 0.871, and 0.861, which are higher than those of classifiers by 4.36%, 6.73%, and 2.71%, respectively.

Bug Report Classification based on Vector Space Model

Bug Report Classification based on Vector Space Model

Ensemble Data Reduction Techniques and Multi-RSMOTE via Fuzzy Integral for Bug Report Classification

Due to the unavoidable bugs appearing in the most of the software systems, bug resolution has become one of the most important activities in software maintenance. To decrease the time cost in manual work, text classification techniques are applied to automatically identify severity of bug reports. In this paper, we address the problem of low-quality and class imbalance for identifying the severity of bug reports. First, we combine feature selection with instance selection to simultaneously reduce the bug report dimension and the word dimension, which could get small-scale and high-quality reduced data set. Then, an improve random oversampling technique, named, RSMOTE, which is presented to weaken the imbalancedness degree of class distribution. Finally, to avoid the random over-sampling uncertainty of RSMOTE, we develop an ensemble learning algorithm, which is based on Choquet fuzzy integral, to combine multiple RSMOTE. We empirically investigate the performance of data reduction on ten data sets of three large open source projects, namely, Eclipse, Mozilla, and GNOME. The results show that our approach can effectively reduce the data scale and improve the performance of identifying the severity of bug reports.

A Supervised Bug Report Classification with Incorporate and Textual field Knowledge

Performance of the bug prediction model is directly depends on the misclassification of bug reports. Misclassification issue surely scarifies the accuracy of the system. To resolve this issue the manual examination of bug reports are required, but it is very time consuming and tedious job for a developer and tester. In this paper the hybrid approach of merging text mining, natural language processing and machine learning techniques is used to identify bug report as bug or non-bug. The four incorporates fields with textual fields are added to bug reports to improve the performance of classifier. TF-IDF and Bigram feature extraction methods are used with feature selection and K-nearest neighbor (K-NN) classifier. The performance of the proposed system is evaluated by using Precision, Recall and F-measure by using five datasets. It is observed that the performance of K-NN classifier is changed according to the dataset and addition of bigram method improve the performance of classifier.

Learning to Rank and Classification of Bug Reports Using SVM and Feature Evaluation

Abstract When a new bug report is received, developers usually need to reproduce the bug and perform code reviews to find the cause, a process that can be tedious and time consuming. A tool for ranking all the source files with respect to how likely they are to contain the cause of the bug would enable developers to narrow down their search and improve productivity. This project introduces an adaptive ranking approach that leverages project knowledge through functional decomposition of source code, API descriptions of library components, the bugfixing history, the code change history, and the file dependency graph. Given a bug report, the ranking score of each source file is computed as a weighted combination of an array of features, where the weights are trained automatically on previously solved bug reports using a learning-to-rank technique. I applied SVM (Support Virtual Machine) to classify the bug reports to identify, which category the bug belongs to. It helps to fix the critical defects early. The ranking system evaluated on six large scale open source Java projects, using the beforefix version of the project for every bug report. The experimental results show that the learning-to-rank approach outperforms three recent state-of-the-art methods. In particular, proposed method makes correct recommendations within the top 10 ranked source files for over 70 percent of the bug reports in the Eclipse Platform and Tomcat projects.

Domain Specific Automated Triaging System for Bug Classification

The objective of this paper is to analyze and identify domain specific priority classification of bug reports. Different classification algorithms namely-Linear Discriminant Analysis (LDA), Naive Bayes (NB) to predict the performance measures are used. The performance of classification algorithms are compared using bug report instances of two open source software (OSS) namely NetBeans and Eclipse is downloaded from Bugzilla bug repository. Principal Component Analysis (PCA) for feature selection, Particle Swam Optimization (PSO) for instance selection to reduce the dimension of the bug reports. The results are compared based on four performance measures i.e. Accuracy, Precision, Recall and Processing Time. Feature selection to Instance selection gives better results than Instance selection to Feature selection is the result influenced from experiment performed. This approach shows that linear Discriminant analysis performs better than Naive Bayes classifier.

Combining text mining and data mining for bug report classification

Bug reports represent an important information source for software construction. Misclassification of these reports inevitably introduces bias. Manual examinations can help reduce the noise, but bring a heavy burden for developers instead. In this paper, we propose a multi-stage approach by combining both text mining and data mining techniques to automate the prediction process. The first stage leverages text mining techniques to analyze the summary parts of bug reports and classifies them into three levels of probability. The extracted features and some other structured features of bug reports are then fed into the machine learner in the second stage. Data grafting techniques are employed to bridge the two stages. Comparative experiments with previous studies on the same data-three large-scale open-source projects-consistently achieve a reasonable enhancement from 77.4% to 81.7%, 76.1% to 81.6%, and 87.4% to 93.7%, respectively over their best results in terms of overall performance. Additional comparative empirical experiments on other seven popular open-source systems confirm the findings. Moreover, based on the data obtained, we also empirically studied the impact relation between the underlying classifiers and various other properties of the combined model. A prototypical recommender system has been developed to demonstrate the applicability of our approach. Copyright © 2016 John Wiley & Sons, Ltd.