Bug Severity Research Articles

Crowdsourced test report prioritization considering bug severity

In crowdsourced testing, a large number of test reports will be generated in a short time. How to efficiently inspect these reports becomes one of the critical steps in the testing process. In recent years, many automated techniques like clustering, classification, and prioritization have emerged to provide an automated inspection order over test reports. Even though these methods have achieved good performance, they did not consider the priority to image and text information. Simultaneously, existing prioritization approaches only focus on the rate of detecting faults but ignore the severity of the faults. In fact, bug severity is a vital indicator that the users provide to flag the criticality of a bug, so developers can then use it to set their priority for the resolution process. For these reasons, this paper presents a novel prioritization approach for crowdsourcing test reports. It extracts features from text and screenshot information of the test reports, uses the hash technique to index test reports, and finally designs a prioritization algorithm. To validate our approach, we conducted experiments on six industrial projects. The results and the hypotheses analysis show that our approach can detect all faults faster in a limited time and can prioritize reports that have higher severity faults compared with the existing methods.

Classification Rule Discovery for Software Bug Severity using KNN with different distance metric

Classification Rule Discovery for Software Bug Severity using KNN with different distance metric

Bug Severity Prediction using Keywords in Imbalanced Learning Environment

Reported bugs of software systems are classified into different severity levels before fixing them. The number of bug reports may not be equally distributed according to the severity levels of bugs. However, most of the severity prediction models developed in the literature assumed that the underlying data distribution is evenly distributed, which may not correct at all instances and hence, the aim of this study is to develop bug classification models from unevenly distributed datasets and tested them accordingly. To that end first, the topics or keywords of developer descriptions of bug reports are extracted using Rapid Keyword Extraction (RAKE) algorithm and then transferred them into numerical attributes, which combined with severity levels constructs datasets. These datasets are used to build classification models; Naïve Bayes, Logistic Regression, and Decision Tree Learner algorithms. The models’ prediction quality is measured using Area Under Recursive Operative Characteristics Curves (AUC) as the models learnt from more skewed environments. According to the results, the prediction quality of the Logistics Regression model is 0.65 AUC whereas the other two models recorded maximum 0.60 AUC. Though the datasets contain comparatively less number of instances from the high severity classes; Blocking and High, the Logistic Regression models predict the two classes with a decent AUC value of 0.65 AUC. Hence, this projects shows that the models can be trained from highly skewed datasets so that the models prediction quality is equally well over all the classes regardless of number of instances representing the class. Further, this project emphasizes that the models should be evaluated using the appropriate metrics when the models are trained from imbalance learning environments. Also, this work uncovers that the Logistic Regression model is also capable of classifying documents as Naïve Bayes, which is well known for this task.

Developing bug severity prediction models using word2vec

Bug tracking systems use repositories to keep track of the bugs to improve software quality. A manual analysis of each bug and classifying it according to its severity is an unmanageable job. Therefore, it is imperative to correctly classify the severity of a bug, which otherwise might be misclassified by a laymen user. Text mining techniques have the potential to analyze such massive databases of the textual description of bug reports to classify bug severity levels adequately. Word embedding is a state-of-art text mining technique that captures the semantics of the text and group the words according to their relevance in a document. Words are embedded into real-valued vectors through word-embedding models. Word2vec is a word embedding model, proven to be effective in representing word meanings. However, the configuration of hyperparameters and feature selection affects the performance of word2vec. Tuning hyperparameter is a time-consuming process, and it is crucial to identify the correct set of parameters. The paper outlines the effectiveness of word2vec technique on the efficiency of classifiers to predict a bug severity from a bug report and examines the impact of different averaging methods, including the configuration of word embedding parameters on the classifiers’ efficiency. Results show that the bigger window size enhances the performance of classifiers; however, the influence of the minimum word count parameter was found to be mixed and depends on the selected data sets. Further, out of the classifiers used, Random Forest and Xgboost could classify the severity level for classes with few records or a rare occurrence of words specific to each class. Otherwise, Support Vector Machine and Naive Bayes classifiers performed better and worst, respectively.

Multiclass Software Bug Severity Classification using Decision Tree, Naive Bayes and Bagging

The software applications are experiencing the challenges of ever-growing complexity caused by the increase in the number of bugs. The software development process has been adversely affected due to the wastage of resources caused due to the bugs. It is imperative to identify and predict bugs to facilitate the software development process. Software bugs can be classified according to the severity of the bugs. In this paper a comparative analysis of Decision Tree, Naïve Bayes and Bagging approach is done for the bug severity classification. A comparative analysis of the Naïve Bayes, Decision Tree and Bagging approach is done for the accuracy, precision, recall and F-measure parameters

Mining Bug Report Repositories to Identify Significant Information for Software Bug Fixing

Most studies relating to bug reports aims to automatically identify necessary information from bug reports for software bug fixing. Unfortunately, the study of bug reports focuses only on one issue, but more complete and comprehensive software bug fixing would be facilitated by assessing multiple issues concurrently. This becomes a challenge in this study, where it aims to present a method of identifying bug reports at severe level from a bug report repository, together with assembling their related bug reports to visualize the overall picture of a software problem domain. The proposed method is called “mining bug report repositories”. Two techniques of text mining are applied as the main mechanisms in this method. First, classification is applied for identifying severe bug reports, called “bug severity classification”, while “threshold-based similarity analysis” is then applied to assemble bug reports that are related to a bug report at severe level. Our datasets are from three opensource namely SeaMonkey, Firefox, and Core:Layout downloaded from the Bugzilla. Finally, the best models from the proposed method are selected and compared with two baseline methods. For identifying severe bug reports using classification technique, the results show that our method improved accuracy, F1, and AUC scores over the baseline by 11.39, 11.63, and 19% respectively. Meanwhile, for assembling related bug reports using threshold-based similarity technique, the results show that our method improved precision, and likelihood scores over the other baseline by 15.76, and 9.14% respectively. This demonstrate that our proposed method may help increasing chance to fix bugs completely.

Is Bug Severity in Line with Bug Fixing Change Complexity?

Both complexity of code change for bug fixing and bug severity play an important role in release planning when considering which bugs should be fixed in a specific release under certain constraints. This work investigates whether there are significant differences between bugs of different severity levels regarding the complexity of code change for fixing the bugs. Code change complexity is measured by the number of modified lines of code, source files, and packages, as well as the entropy of code change. We performed a case study on 20 Apache open source software (OSS) projects using commit records and bug reports. The study results show that (1) for bugs of high severity levels (i.e. Blocker, Critical and Major in JIRA), there is no significant difference on the complexity of code change for fixing bugs of different severity levels for most projects, while (2) for bugs of low severity levels (i.e. Major, Minor and Trivial in JIRA), fixing bugs of a higher severity level needs significantly more complex code change than fixing bugs of a lower severity level for most projects. These findings provide useful and practical insights for effort estimation and release planning of OSS development.

Multiclass Severity Classification for Software Bugs Using Support Vector Machine, K-Nearest Neighbor, Decision Tree and Naïve Bayes

During the software development, all most 30–35 present cost is due to the testing. This means that if a bug travels from one phase to succeeding phases without detection, it will definitely increase the cost of the software development and due to this software quality may be compromised. So use of the data mining algorithm for the software bug classification is highly appreciable. Bug severity may be categorised into S1, S2, S3, S4 and S5 categories, depending on the impact of the severity. In this paper, multiclass of bug severity is done using SVM, KNN, Decision Tree and Naïve Bayes. Comparative analysis of these algorithms is done with respect to accuracy, precision, recall and execution time.

An efficient classification of secure and non-secure bug report material using machine learning method for cyber security

In the field of software development, the main problem is to recognize the security-oriented issues within the reported bugs due to its inacceptable rate to provide the satisfied reliability on customer and software dataset. The objective is to propose a novel machine learning approach for multiclass supervised classification named Bug Severity classification to overcome these challenges with the use of supervised Artificial Neural Network and stacking based Navies Bayes classifier. This proposed approach directly examines the latent and highly descriptive features. Primarily, using the natural language programming approaches bug report text is preprocessed. After then, N gram is employed for extracting features by overcoming data sparsity problems. Further, the supervised Artificial Neural Network extracts the salient feature patterns of the corresponding severity classes. Finally, the stacking-based Navies Bayes classifier is employed for classifying multiple bug severity classes.

Performance Gauging of Discrete Classifiers by Prognostication of a Reported Bug

Bugs are the frequently transpiring drawback in software’s, to prevent these problems; an in-depth study of bugs is required. Bug repositories are a significant supply of information. The bug repository helps the software team to have a better study about bugs and its related parameters. Often arising bugs helps the developers to get rid of them in upcoming releases. There is a huge variety of algorithms that facilitate in finding bugs. This paper intends to measure the performance of mining algorithms in predicting the bug severity by scrutinizing their capabilities. We propose a study of assorted algorithms in Lazy (IBK and KStar) and Tree (Random Forest) Classifiers with regard to different parameters. The case inspected here is Mozilla_Thunderbird, which is implemented in WEKA.

Multi-Feature Approach for Bug Severity Assignment

When bug reports are submitted through bug tracking systems, they are analysed manually to identify their severity levels. A severity level specifies the negative impact of a bug on a system. With the huge number of submitted reports, setting the severity class manually is tedious and time consuming. Moreover, some bug types are reported more often than other types, which leads to imbalanced bug repositories. This paper proposes a multi-feature approach for automatic severity assignment, which leverages lexical, semantic, and categorical properties of the bug reports. The proposed approach utilizes word embeddings, topic model, vector space model, and an adapted K-Nearest Neighbour technique. Moreover, the impact of utilizing two sampling techniques, namely SMOTE and cluster-based under-sampling (CBU), were investigated. Experiments over two open source repositories, Eclipse and Mozilla, demonstrated that the proposed approach is superior to two previous studies.

Does bug report summarization help in enhancing the accuracy of bug severity classification?

The programmer cannot write a program without any bug. A large numbers of bugs are deposited into the bug tracking system through bug reports. To find the root cause of a bug, a meaningful and huge conversation happens between the developer and reporter. The developer (triager) reads the whole bug report and then classified according to severity. The previous researchers observed that the bug report summaries provide the more resourcefully investigate information in the bug repository to the developer as part of the severity classification task. To further investigate the relationship between bug report summary and bug severity classification. A novel approach is proposed by using swarm intelligence and machine learning approaches. Firstly the n-gram technique is used to extract the semantic features score. These features are fed into the Summary Subset Selection Phase to select the optimal summary subset. The selected subset features are fed into the feature scoring phase to provide a relative score to each feature. These optimized features are used to train the proposed model. At last Naive Bayes approach is used to classify the multiclass severity classification. The results are analyzed by using 10-fold cross-validation on three benchmark datasets showed better performance in terms of Precision, Recall and F-measure. It is observed that the performance depend on the bug report contents. If the bug report has larger data for summarization than the summarization increase the classification accuracy otherwise decrease.

Mining Severe Priority Bugs in Software Maintenance

Maintenance of open source software is a hectic task as the number of bugs reported is huge. The number of projects, components and versions in an open source project also contribute to the number of bugs that are being reported. Classification of bugs based on priority and identification of the suitable engineers for assignment of bugs for such huge systems still remains a major challenge. Bugs that are misclassified or assigned to engineers who don’t have the component expertise, drastically affect the time taken towards bug resolution. In this paper we have explored the usage of data mining techniques on the classification of bugs and assignment of bugs to engineers.Our focus was on classifying bugs as either severe or non-severe and identification of engineers who have the right expertise to fix the bugs. The prediction of bug severity and identification of engineers were done by mining bug reports from JIRA, an open source software bug tracking tool. The mining process yielded positive results and will be a decision enhancer for severe bugs in the maintenance phase

Bug Severity Prediction using Class Imbalance Problem

Class imbalance problem is often observed when instances of major class exceed instances of minor class. The performance of machine learning techniques is immensely afflicted by imbalanced data in several fields. The skewed distribution either predicts the majority class with high error rate or will not foresee the minority class. To solve the problem of imbalanced data of software bugs, Synthetic minority oversampling technique (SMOTE) is used which balances the imbalanced datasets of Apache Projects. It is applied on summary of bugs to balance the dataset and predicts severity at system and component level. Several machine learning techniques are applied on imbalanced as well as balanced datasets to predict the severity of software bugs using textual description. Test outcomes and statistical analysis shows improved results on balanced datasets in respect to Gmean and balance metrics instead of machine learning techniques applied on imbalanced data. Evaluation metrics Gmean improves by 34% and balance by 11% at system level and by 42% and 62% at component level. Further, it was observed that solving class imbalance problem on textual data is helpful in augmenting the performance.

Automatic prediction of the severity of bugs using stack traces and categorical features

The severity of a bug is often used as an indicator of how a bug negatively affects system functionality. It is used by developers to prioritize bugs which need to be fixed. The problem is that, for various reasons, bug submitters often enter the incorrect severity level, delaying the bug resolution process. Techniques that can automatically predict the severity of a bug can significantly reduce the bug triaging overhead. In our previous work, we showed that the accuracy of description-based severity prediction techniques could be significantly improved by using stack traces as a source of information. In this study, we expand our previous work by exploring the effect of using categorical features, in addition to stack traces, to predict the severity of bugs. These categorical features include faulty product, faulty component, and operating system. We experimented with other features and observed that they do not improve the severity prediction accuracy. A Software system is composed of many products; each has a set of components. Components interact with each to provide the functionality of the product. The operating system field refers to the operating system on which the software was running on during the crash. The proposed approach uses a linear combination of stack trace and categorical features similarity to predict the severity. We adopted a cost sensitive K Nearest Neighbor approach to overcome the unbalance label distribution problem and improve the classifier accuracy. Our experiments on bug reports of Eclipse submitted between 2001 and 2015 and Gnome submitted between 1999 and 2015 show that the accuracy of our severity prediction approach can be improved from 5% to 20% by considering categorical features, in addition to stack traces. The accuracy of predicting the severity of bugs is higher when combining stack traces and three categorical features, product, component, and operating system.

Improved Framework for Bug Severity Classification using N-gram Features with Convolution Neural Network

Foreseeing the seriousness/severity of bugs has been established in former research study in order to recover triaging and the process of bug resolution. Therefore, numerous prediction/classification methodologies were developed throughout the years to give an automated reasoning over the seriousness classes. Seriousness or severity is a significant trait of a bug that chooses how rapidly it ought to be measured. It causes designers to comprehend significant bugs on schedule. Though, manual evaluation of severity is a dreary activity and could be off base. This paper comprises of using the text/content mining together along with the use feature selection and bi-grams to improve the order of bugs in six classes. In the proposed methodology the features are refined by the use of convolution layers. Here, the process of convolution-based refining indicates mapping of the features utilizing non-linear methods of all the classes as compared to the existing methodologies.

Multi-attribute dependent bug severity and fix time prediction modeling

A software bug is characterized by many features/attributes out of which some are entered during the time of bug reporting whereas others are entered during the bug fixing. Severity is an important bug attribute and critical factor in deciding how soon it needs to be fixed. During the initial period of bug reporting, its severity changes and get stabilizes over a period of time. Severity identification is a major task of triagers, whose success affects the bug fix time. The prediction of bug fix time will help in estimating the maintenance efforts and better software project management. We investigated the association among the bug attributes and built multi-attribute based classification and regression models for bug severity and fix time prediction. Bug severity and fix time prediction models have been built using the combinations of different independent bug attributes. We have used different classification and regression techniques, namely Support Vector Machine (SVM), Naive Bayes (NB), k-Nearest Neighbors (k-NN), Ordinal Regression (OR), Fuzzy Linear Regression (FLR), Fuzzy Multi Linear Regression (FMLR), Multiple Linear Regression (MLR), Support Vector Regression (SVR) and k-Nearest Neighbors Regression (k-NNR) to build the models. Our models are tested on the real world datasets from famous open source project: Mozilla. k-NN gives better performance than NB and SVM in terms of precision and f-measure for bug severity prediction. In terms of goodness of fit, SVR is better than MLR and k-NNR for bug fix time prediction. The proposed mechanism is able to predict severity and fix time for newly reported bugs. Empirical results reveal that the multi-attribute based classification and regression models work well for bug severity and fix time prediction. The two newly derived attributes Summary weight and Bug age are found to be good predictors of severity across all the used techniques. In case of bug fix time prediction, Bug age is found to be a good predictor.

Bug report severity level prediction in open source software: A survey and research opportunities

Context: The severity level attribute of a bug report is considered one of the most critical variables for planning evolution and maintenance in Free/Libre Open Source Software. This variable measures the impact the bug has on the successful execution of the software system and how soon a bug needs to be addressed by the development team. Both business and academic community have made an extensive investigation towards the proposal methods to automate the bug report severity prediction. Objective: This paper aims to provide a comprehensive mapping study review of recent research efforts on automatically bug report severity prediction. To the best of our knowledge, this is the first review to categorize quantitatively more than ten aspects of the experiments reported in several papers on bug report severity prediction. Method: The mapping study review was performed by searching four electronic databases. Studies published until December 2017 were considered. The initial resulting comprised of 54 papers. From this set, a total of 18 papers were selected. After performing snowballing, more nine papers were selected. Results: From the mapping study, we identified 27 studies addressing bug report severity prediction on Free/Libre Open Source Software. The gathered data confirm the relevance of this topic, reflects the scientific maturity of the research area, as well as, identify gaps, which can motivate new research initiatives. Conclusion: The message drawn from this review is that unstructured text features along with traditional machine learning algorithms and text mining methods have been playing a central role in the most proposed methods in literature to predict bug severity level. This scenario suggests that there is room for improving prediction results using state-of-the-art machine learning and text mining algorithms and techniques.

A Novel Deep-Learning-Based Bug Severity Classification Technique Using Convolutional Neural Networks and Random Forest with Boosting.

The accurate severity classification of a bug report is an important aspect of bug fixing. The bug reports are submitted into the bug tracking system with high speed, and owing to this, bug repository size has been increasing at an enormous rate. This increased bug repository size introduces biases in the bug triage process. Therefore, it is necessary to classify the severity of a bug report to balance the bug triaging process. Previously, many machine learning models were proposed for automation of bug severity classification. The accuracy of these models is not up to the mark because they do not extract the important feature patterns for learning the classifier. This paper proposes a novel deep learning model for multiclass severity classification called Bug Severity classification to address these challenges by using a Convolutional Neural Network and Random forest with Boosting (BCR). This model directly learns the latent and highly representative features. Initially, the natural language techniques preprocess the bug report text, and then n-gram is used to extract the features. Further, the Convolutional Neural Network extracts the important feature patterns of respective severity classes. Lastly, the random forest with boosting classifies the multiple bug severity classes. The average accuracy of the proposed model is 96.34% on multiclass severity of five open source projects. The average F-measures of the proposed BCR and the existing approach were 96.43% and 84.24%, respectively, on binary class severity classification. The results prove that the proposed BCR approach enhances the performance of bug severity classification over the state-of-the-art techniques.

An Analysis of Software Bug Reports Using Machine Learning Techniques

Bug tracking systems manage bug reports for assuring the quality of software products. A bug report (alsoreferred as trouble, problem, ticket or defect) contains several features for problem management and resolution purposes. Severity and priority are two essential features of a bug report that define the effect level and fixing order of the bug. Determining these features is challenging and depends heavily on human being, e.g., software developers or system operators, especially for assessing a large number of error and warning events occurring on software products or network services. This study first proposes a comparison of machine learning techniques for assessing severity and priority for software bug reports and then chooses an approach of using optimal decision trees, or random forest, for further investigation. This approach aims at constructing multiple decision trees based on the subsets of the existing bug dataset and features, and then selecting the best decision trees to assess the severity and priority of new bugs. The approach can be applied for detecting and forecasting faults in large, complex communication networks and distributed systems today. We have presented the applicability of random forest for bug report analysis and performed several experiments on software bug datasets obtained from open source bug tracking systems. Random forest yields an average accuracy score of 0.75 that can be sufficient for assisting system operators in determining these features. We have provided some analysis of the experimental results.