Bug Repositories Research Articles

A Novel Deep-Learning-Based Bug Severity Classification Technique Using Convolutional Neural Networks and Random Forest with Boosting.

The accurate severity classification of a bug report is an important aspect of bug fixing. The bug reports are submitted into the bug tracking system with high speed, and owing to this, bug repository size has been increasing at an enormous rate. This increased bug repository size introduces biases in the bug triage process. Therefore, it is necessary to classify the severity of a bug report to balance the bug triaging process. Previously, many machine learning models were proposed for automation of bug severity classification. The accuracy of these models is not up to the mark because they do not extract the important feature patterns for learning the classifier. This paper proposes a novel deep learning model for multiclass severity classification called Bug Severity classification to address these challenges by using a Convolutional Neural Network and Random forest with Boosting (BCR). This model directly learns the latent and highly representative features. Initially, the natural language techniques preprocess the bug report text, and then n-gram is used to extract the features. Further, the Convolutional Neural Network extracts the important feature patterns of respective severity classes. Lastly, the random forest with boosting classifies the multiple bug severity classes. The average accuracy of the proposed model is 96.34% on multiclass severity of five open source projects. The average F-measures of the proposed BCR and the existing approach were 96.43% and 84.24%, respectively, on binary class severity classification. The results prove that the proposed BCR approach enhances the performance of bug severity classification over the state-of-the-art techniques.

Reproducing performance bug reports in server applications: The researchers’ experiences

Performance is one of the key aspects of non-functional qualities as performance bugs can cause significant performance degradation and lead to poor user experiences. While bug reports are intended to help developers to understand and fix bugs, they are also extensively used by researchers for finding benchmarks to evaluate their testing and debugging approaches. Although researchers spend a considerable amount of time and effort in finding usable performance bugs from bug repositories, they often get only a few. Reproducing performance bugs is difficult even for performance bugs that are confirmed by developers with domain knowledge. The amount of information disclosed in a bug report may not always be sufficient to reproduce the performance bug for researchers, and thus hinders the usability of bug repository as the resource for finding benchmarks. In this paper, we study the characteristics of confirmed performance bugs by reproducing them using only informations available from the bug report to examine the challenges of bug reproduction from the perspective of researchers. We spent more than 800 h over the course of six months to study and to try to reproduce 93 confirmed performance bugs, which are randomly sampled from two large-scale open-source server applications. We (1) studied the characteristics of the reproduced performance bug reports; (2) summarized the causes of failed-to-reproduce performance bug reports from the perspective of researchers by reproducing bugs that have been solved in bug reports; (3) shared our experience on suggesting workarounds to improve the bug reproduction success rate; (4) delivered a virtual machine image that contains a set of 17 ready-to-execute performance bug benchmarks. The findings of our study provide guidance and a set of suggestions to help researchers to understand, evaluate, and successfully replicate performance bugs.

SMOTE and Feature Selection for More Effective Bug Severity Prediction

“Severity” is one of the essential features of software bug reports, which is a crucial factor for developers to decide which bug should be fixed immediately and which bug could be delayed to a next release. Severity assignment is a manual process and its accuracy depends on the experience of the assignee. Prior research proposed several models to automate this process. These models are based on textual preprocessing of historical bug reports and classification techniques. Although bug repositories suffer from severity class imbalance, none of the prior studies investigated the impact of implementing a class rebalancing technique on the accuracy of their models. In this paper, we propose a framework for predicting fine-grained severity levels which utilizes an over-sampling technique “SMOTE”, to balance the severity classes, and a feature selection scheme, to reduce the data scale and select the most informative features for training a [Formula: see text]-nearest neighbor (KNN) classifier. The KNN classifier utilizes a distance-weighted voting scheme to predict the proper severity level of a newly reported bug. We investigated the effectiveness of our proposed approach on two large bug repositories, namely Eclipse and Mozilla, and the experimental results showed that our approach outperforms cutting-edge studies in predicting the minority severity classes.

An HMM-based approach for automatic detection and classification of duplicate bug reports

Abstract Context Software projects rely on their issue tracking systems to guide maintenance activities of software developers. Bug reports submitted to the issue tracking systems carry crucial information about the nature of the crash (such as texts from users or developers and execution information about the running functions before the occurrence of a crash). Typically, big software projects receive thousands of reports every day. Objective The aim is to reduce the time and effort required to fix bugs while improving software quality overall. Previous studies have shown that a large amount of bug reports are duplicates of previously reported ones. For example, as many as 30% of all reports in for Firefox are duplicates. Method While there exist a wide variety of approaches to automatically detect duplicate bug reports by natural language processing, only a few approaches have considered execution information (the so-called stack traces) inside bug reports. In this paper, we propose a novel approach that automatically detects duplicate bug reports using stack traces and Hidden Markov Models. Results When applying our approach to Firefox and GNOME datasets, we show that, for Firefox, the average recall for Rank k = 1 is 59%, for Rank k = 2 is 75.55%. We start reaching the 90% recall from k = 10. The Mean Average Precision (MAP) value is up to 76.5%. For GNOME, The recall at k = 1 is around 63%, while this value increases by about 10% for k = 2. The recall increases to 97% for k = 11. A MAP value of up to 73% is achieved. Conclusion We show that HMM and stack traces are a powerful combination for detecting and classifying duplicate bug reports in large bug repositories.

Applying Topic Modeling and Similarity for Predicting Bug Severity in Cross Projects

Recently, software has increased in complexity and been applied in various industrial fields. As a result, the presence of software bugs cannot be avoided. Various bug severity prediction methodologies have been proposed, but their performance needs to be further improved. In this study, we propose a novel technique for bug severity prediction in cross projects such as Eclipse, Mozilla, WireShark, and Xamarin by using topic modeling and similarity (i.e., KL-divergence). First, we construct topic models from bug repositories in cross projects using Latent Dirichlet Allocation (LDA). Then, we find topics in each project that contain the most numerous similar bug reports by using a new bug report. Next, we extract the bug reports belonging to the selected topics and input them to a Naive Bayes Multinomial (NBM) algorithm. Finally, we predict the bug severity in the new bug report. In order to evaluate the performance of our approach and to verify the difference between cross projects and single project, we compare it with the Naive Bayes Multinomial approach; the Lamkanfi methodology, which is a well-known bug severity prediction approach; and an emotional similarity-based bug severity prediction approach. Our approach exhibits a better performance than the compared methods.

FineLocator: A novel approach to method-level fine-grained bug localization by query expansion

Abstract Context Bug localization, namely, to locate suspicious snippets from source code files for developers to fix the bug, is crucial for software quality assurance and software maintenance. Effective bug localization technique is desirable for software developers to reduce the effort involved in bug resolution. State-of-the-art bug localization techniques concentrate on file-level coarse-grained localization by lexical matching bug reports and source code files. However, this would bring about a heavy burden for developers to locate feasible code snippets to make change with the goal of fixing the bug. Objective This paper proposes a novel approach called FineLocator to method-level fine-grained bug localization by using semantic similarity, temporal proximity and call dependency for method expansion. Method Firstly, the bug reports and the methods of source code are represented by numeric vectors using word embedding (word2vec) and the TF-IDF method. Secondly, we propose three query expansion scores as semantic similarity score, temporal proximity score and call dependency score to address the representation sparseness problem caused by the short lengths of methods in the source code. Then, the representation of a method with short length is augmented by elements of its neighboring methods with query expansion. Thirdly, when a new bug report is incoming, FineLocator will retrieve the methods in source code by similarity ranking on the bug report and the augmented methods for bug localization. Results We collect bug repositories of ArgoUML, Maven, Kylin, Ant and AspectJ projects to investigate the performance of the proposed FineLocator approach. Experimental results demonstrate that the proposed FineLocator approach can improve the performances of method-level bug localization at average by 20%, 21% and 17% measured by Top-N indicator, MAP and MRR respectively, in comparison with state-of-the-art techniques. Conclusion This is the first paper to demonstrate how to make use of method expansion to address the representation sparseness problem for method-level fine-grained bug localization.

Identify Severity Bug Report with Distribution Imbalance by CR-SMOTE and ELM

Manually inspecting bugs to determine their severity is often an enormous but essential software development task, especially when many participants generate a large number of bug reports in a crowdsourced software testing context. Therefore, boosting the capabilities of methods of predicting bug report severity is critically important for determining the priority of fixing bugs. However, typical classification techniques may be adversely affected when the severity distribution of the bug reports is imbalanced, leading to performance degradation in a crowdsourcing environment. In this study, we propose an enhanced oversampling approach called CR-SMOTE to enhance the classification of bug reports with a realistically imbalanced severity distribution. The main idea is to interpolate new instances into the minority category that are near the center of existing samples in that category. Then, we use an extreme learning machine (ELM) — a feedforward neural network with a single layer of hidden nodes — to predict the bug severity. Several experiments were conducted on three datasets from real bug repositories, and the results statistically indicate that the presented approach is robust against real data imbalance when predicting the severity of bug reports. The average accuracies achieved by the ELM in predicting the severity of Eclipse, Mozilla, and GNOME bug reports were 0.780, 0.871, and 0.861, which are higher than those of classifiers by 4.36%, 6.73%, and 2.71%, respectively.

Bug Localization for Version Issues With Defect Patterns

Version issues are becoming more and more prominent with the continuous development of software. Bug localization for version issues is time-consuming and labor-intensive. Although some bug localization techniques, such as those based on information retrieval (IR), have been proposed, they cannot handle these bugs very well as the version-related bugs have their own defect patterns. However, few existing works have focused on revealing these defect patterns and utilizing them for localization of bugs. To fill this gap, we propose a new approach by leveraging the version-related defect patterns to localize the version-related issues integrated with the IR technique. First, we extract version-related bugs from bug repositories and build a version-related bug repository. Given a new version-related bug report, we identify the defect patterns of corresponding similar historical bug reports from the version-related bug repository. Then, we combine these defect patterns with the IR technique to rank the candidate code snippets as suspicious code for developers to fix. The evaluation demonstrates that our approach is more effective to identify the faulty code related to version issues than the existing IR-based bug localization technique.

Machine Learning Approaches for Predicting the Severity Level of Software Bug Reports in Closed Source Projects

In Software Development Life Cycle, fixing defect bugs is one of the essential activities of the software maintenance phase. Bug severity indicates how major or minor the bug impacts on the execution of the system and how rapidly the developer should fix it. Triaging a vast amount of new bugs submitted to the software bug repositories is a cumbersome and time-consuming process. Manual triage might lead to a mistake in assigning the appropriate severity level for each bug. As a consequence, a delay for fixing severe software bugs will take place. However, the whole process of assigning the severity level for bug reports should be automated. In this paper, we aim to build prediction models that will be utilized to determine the class of the severity (severe or non-severe) of the reported bug. To validate our approach, we have constructed a dataset from historical bug reports stored in JIRA bug tracking system. These bug reports are related to different closed-source projects developed by INTIX Company located in Amman, Jordan. We compare eight popular machine learning algorithms, namely Naive Bayes, Naive Bayes Multinomial, Support Vector Machine, Decision Tree (J48), Random Forest, Logistic Model Trees, Decision Rules (JRip) and K-Nearest Neighbor in terms of accuracy, F-measure and Area Under the Curve (AUC). According to the experimental results, a Decision Tree algorithm called Logistic Model Trees achieved better performance compared to other machine learning algorithms in terms of Accuracy, AUC and F-measure with values of 86.31, 0.90 and 0.91, respectively.

Characterizing and predicting blocking bugs in open source projects

Software engineering researchers have studied specific types of issues such reopened bugs, performance bugs, dormant bugs, etc. However, one special type of severe bugs is blocking bugs. Blocking bugs are software bugs that prevent other bugs from being fixed. These bugs may increase maintenance costs, reduce overall quality and delay the release of the software systems. In this paper, we study blocking bugs in eight open source projects and propose a model to predict them early on. We extract 14 different factors (from the bug repositories) that are made available within 24 hours after the initial submission of the bug reports. Then, we build decision trees to predict whether a bug will be a blocking bugs or not. Our results show that our prediction models achieve F-measures of 21%–54%, which is a two-fold improvement over the baseline predictors. We also analyze the fixes of these blocking bugs to understand their negative impact. We find that fixing blocking bugs requires more lines of code to be touched compared to non-blocking bugs. In addition, our file-level analysis shows that files affected by blocking bugs are more negatively impacted in terms of cohesion, coupling complexity and size than files affected by non-blocking bugs.

Prioritizing lingering bugs

As the software projects become more complex, the release deci-sion is made without resolving all the bugs in the issue tracking system. Accumulation of the bugs in the bug repository is similar to nancial obligation as we borrow time and resources to engage in another activity rather than resolving the bugs. Deferring the bug in the next release may have some consequences. Therefore, the decision whether to resolve the bug in the current release or postponing it to the next release is a crucial decision. In this proposal, we study the deferred bugs (lingering bugs) against the non-deferred bugs (regular bugs). Our aim is to develop the pre-dictive model which can predict whether the bug would linger or not. Additionally, we are interested in measuring of the linger-ing bug in terms of principal (standard time it takes to x them) and risk of liability (impact). We propose to use reinforcement learning for prioritization of lingering bugs with respect to their impact.

Effectiveness of exploring historical commits for developer recommendation: an empirical study

Developer recommendation is an essential task for resolving incoming issues in the evolution of software. Many developer recommendation techniques have been developed in the literature; among these studies, most techniques usually combined historical commits as supplementary information with bug repositories and/or source-code repositories to recommend developers. However, the question of whether the messages in historical commits are always useful has not yet been answered. This article aims at solving this problem by conducting an empirical study on four open-source projects. The results show that: (1) the number of meaningful words of the commit description has an impact on the quality of the commit, and a larger number of meaningful words in the description means that it can generally better reflect developers’ expertise; (2) using commit description to recommend the relevant developers is better than that using relevant files that are recorded in historical commits; (3) developers tend to change the relevant files that they have changed many times before; (4) developers generally tend to change the files that they have changed recently.

InvarNet-X: A Black-Box Invariant-Based Approach to Diagnosing Big Data Systems

As big data spreads rapidly, performance problems in these systems become common concerns. As the first line of defending these problems, performance diagnosis plays an essential role in big data systems. It is notoriously difficult to conduct performance diagnosis in large distributed systems. Previous work either pinpoint the root causes by instrumenting the applications or runtime systems in a white-box way, which leads to a considerable overhead, or just provide some hints to the hidden root causes in a black-box way. Very few works focus on pinpointing the real root causes in a black-box way. To address this problem, this paper proposes a black-box invariant-based diagnosing approach and implements a proof-of-concept system named InvarNet-X . In this paper, performance diagnosis is formalized as a pattern recognition problem, meaning that each performance problem is identified by a specific pattern. The rationale of InvarNet-X is that the unobservable root causes of performance problems always expose themselves through the violations of the associations among directly observable performance metrics. Such observable associations are called likely invariants calculated by the maximal information criterion, and each performance problem is signified by a sparse distributed representation. A problem signature database is constructed by training multiple real performance problems in advance. Once a performance anomaly is detected, the diagnosing procedure is triggered. The root cause is pinpointed by retrieving similar signatures in the signature database. The experimental evaluations in a controlled big data system show that InvarNet-X can achieve a high accuracy in diagnosing some real performance problems reported in software bug repositories, which is superior to several state-of-the-art approaches. Moreover, the light-weight property makes InvarNet-X easily facilitated in large-scale big data systems in real time.

Using Knowledge Transfer and Rough Set to Predict the Severity of Android Test Reports via Text Mining

Crowdsourcing is an appealing and economic solution to software application testing because of its ability to reach a large international audience. Meanwhile, crowdsourced testing could have brought a lot of bug reports. Thus, in crowdsourced software testing, the inspection of a large number of test reports is an enormous but essential software maintenance task. Therefore, automatic prediction of the severity of crowdsourced test reports is important because of their high numbers and large proportion of noise. Most existing approaches to this problem utilize supervised machine learning techniques, which often require users to manually label a large number of training data. However, Android test reports are not labeled with their severity level, and manual labeling is time-consuming and labor-intensive. To address the above problems, we propose a Knowledge Transfer Classification (KTC) approach based on text mining and machine learning methods to predict the severity of test reports. Our approach obtains training data from bug repositories and uses knowledge transfer to predict the severity of Android test reports. In addition, our approach uses an Importance Degree Reduction (IDR) strategy based on rough set to extract characteristic keywords to obtain more accurate reduction results. The results of several experiments indicate that our approach is beneficial for predicting the severity of android test reports.

En-LDA: An Novel Approach to Automatic Bug Report Assignment with Entropy Optimized Latent Dirichlet Allocation

With the increasing number of bug reports coming into the open bug repository, it is impossible to triage bug reports manually by software managers. This paper proposes a novel approach called En-LDA (Entropy optimized Latent Dirichlet Allocation (LDA)) for automatic bug report assignment. Specifically, we propose entropy to optimize the number of topics of the LDA model and further use the entropy optimized LDA to capture the expertise and interest of developers in bug resolution. A developer’s interest in a topic is modeled by the number of the developer’s comments on bug reports of the topic divided by the number of all the developer’s comments. A developer’s expertise in a topic is modeled by the number of the developer’s comments on bug reports of the topic divided by the number of all developers’ comments on the topic. Given a new bug report, En-LDA recommends a ranked list of developers who are potentially adequate to resolve the new bug. Experiments on Eclipse JDT and Mozilla Firefox projects show that En-LDA can achieve high recall up to 84% and 58%, and precision up to 28% and 41%, respectively, which indicates promising aspects of the proposed approach.

A Study of Causes and Consequences of Client-Side JavaScript Bugs

Client-side JavaScript is widely used in web applications to improve user-interactivity and minimize client-server communications. Unfortunately, JavaScript is known to be error-prone. While prior studies have demonstrated the prevalence of JavaScript faults, no attempts have been made to determine their causes and consequences. The goal of our study is to understand the root causes and impact of JavaScript faults and how the results can impact JavaScript programmers, testers and tool developers. We perform an empirical study of 502 bug reports from 19 bug repositories. The bug reports are thoroughly examined to classify and extract information about each bug' cause (the error ) and consequence (the failure and impact ). Our results show that the majority (68 percent) of JavaScript faults are DOM-related , meaning they are caused by faulty interactions of the JavaScript code with the Document Object Model (DOM) . Further, 80 percent of the highest impact JavaScript faults are DOM-related. Finally, most JavaScript faults originate from programmer mistakes committed in the JavaScript code itself, as opposed to other web application components. These results indicate that JavaScript programmers and testers need tools that can help them reason about the DOM. Additionally, developers can use the error patterns we found to design more powerful static analysis tools for JavaScript.

A fuzzy-based approach for bug report categorisation

Various studies conducted on bug repositories utilise issue reports labelled as 'bug'. Research conducted on a number of bug repositories have shown that not all issue reports labelled as 'bug' are actually bugs but can also be a request for additional feature, improvement or documentation. This not only threatens the validity of studies that have used mislabelled data but may also give wrong prediction results in future. This has necessitated need for correct labelling of issue reports. The proposed work using fuzzy logic classifier suggests improvement and also reduces the complexity. Validation of this work is done using five open source projects. Experimental results have shown that our approach gives better F-measure scores. The study also elaborated on use of issue reports from other similar projects for training a model; the impact of frequent terms from the training data and applicability of our approach to fine grained categorisation of issues.

Mining authorship characteristics in bug repositories

Bug reports are widely employed to facilitate software tasks in software maintenance. Since bug reports are contributed by people, the authorship characteristics of contributors may heavily impact the performance of resolving software tasks. Poorly written bug reports may delay developers when fixing bugs. However, no in-depth investigation has been conducted over the authorship characteristics. In this study, we first leverage byte-level $N$-grams to model the authorship characteristics and employ Normalized Simplified Profile Intersection (NSPI) to identify the similarity of the authorship characteristics. Then, we investigate a series of properties related to contributors authorship characteristics, including the evolvement over time and the variation among distinct products in open source projects. Moreover, we show how to leverage the authorship characteristics to facilitate a well-known task in software maintenance, namely Bug Report Summarization (BRS). Experiments on open source projects validate that incorporating the authorship characteristics can effectively improve a state-of-the-art method in BRS. Our findings suggest that contributors should retain stable authorship characteristics and the authorship characteristics can assist in resolving software tasks.

Automated triaging of very large bug repositories

Context: Bug tracking systems play an important role in software maintenance. They allow both developers and users to submit problem reports on observed failures. However, by allowing anyone to submit problem reports, it is likely that more than one reporter will report on the same issue. Research in open source repositories has focused on two broad areas: determining the original report associated with each known duplicate, and assigning a developer to fix a particular problem.Objective: Limited research has been done in developing a fully automated triager, one that can first ascertain if a problem report is original or duplicate, and then provide a list of 20 potential matches for a duplicate report. We address this limitation by developing an automated triaging system that can be used to assist human triagers in bug tracking systems.Method: Our automated triaging system automatically assigns a label of original or duplicate to each incoming problem report, and provides a list of 20 suggestions for reports classified as duplicate. The system uses 24 document similarity measures and associated summary statistics, along with a suite of document property and user metrics. We perform our research on a lifetime of problem reports from the Eclipse, Firefox and Open Office repositories. Results: Our system can be used as a filtration aide, with high original recall exceeding 95% and low duplicate recall, or as a triaging guide, with balanced recall of approximately 70% for both originals and duplicates. Furthermore, the system reduces the workload on the triager by over 90%.Conclusions: Our work represents the first full scale effort at automatically triaging problem reports in open source repositories. By utilizing multiple similarity measures, we reduce the potential of false matches caused by the diversity of human language.

Domain Specific Automated Triaging System for Bug Classification

The objective of this paper is to analyze and identify domain specific priority classification of bug reports. Different classification algorithms namely-Linear Discriminant Analysis (LDA), Naive Bayes (NB) to predict the performance measures are used. The performance of classification algorithms are compared using bug report instances of two open source software (OSS) namely NetBeans and Eclipse is downloaded from Bugzilla bug repository. Principal Component Analysis (PCA) for feature selection, Particle Swam Optimization (PSO) for instance selection to reduce the dimension of the bug reports. The results are compared based on four performance measures i.e. Accuracy, Precision, Recall and Processing Time. Feature selection to Instance selection gives better results than Instance selection to Feature selection is the result influenced from experiment performed. This approach shows that linear Discriminant analysis performs better than Naive Bayes classifier.