Bug Fixing Process Research Articles

Bug priority change: An empirical study on Apache projects

In issue tracking systems, each bug is assigned a priority level (e.g., Blocker, Critical, Major, Minor, or Trivial in JIRA from highest to lowest), which indicates the urgency level of the bug. In this sense, understanding bug priority changes helps to arrange the work schedule of participants reasonably, and facilitates a better analysis and resolution of bugs. According to the data extracted from JIRA deployed by Apache, a proportion of bugs in each project underwent priority changes after such bugs were reported, which brings uncertainty to the bug fixing process. However, there is a lack of in-depth investigation on the phenomenon of bug priority changes, which may negatively impact the bug fixing process. Thus, we conducted a quantitative empirical study on bugs with priority changes through analyzing 32 non-trivial Apache open source software projects. The results show that: (1) 8.3% of the bugs in the selected projects underwent priority changes; (2) the median priority change time interval is merely a few days for most (28 out of 32) projects, and half (50. 7%) of bug priority changes occurred before bugs were handled; (3) for all selected projects, 87.9% of the bugs with priority changes underwent only one priority change, most priority changes tend to shift the priority to its adjacent priority, and a higher priority has a greater probability to undergo priority change; (4) bugs that require bug-fixing changes of higher complexity or that have more comments are likely to undergo priority changes; and (5) priorities of bugs reported or allocated by a few specific participants are more likely to be modified, and maximally only one participant in each project tends to modify priorities.

SBugLocater: Bug Localization Based on Deep Matching and Information Retrieval

Bug localization is a technology that locates buggy source files using bug reports reported by users. Automatic localization of buggy files can speed up the process of bug fixing to improve the efficiency and productivity of software quality assurance teams. Nowadays, some research studies have investigated the natural language information retrieval technology, but few of them have applied the matching technology in deep learning to bug localization. Therefore, we propose a bug localization model SBugLocater based on deep matching and IR. The model composes of three layers: semantic matching layer, relevance matching layer, and IR layer. In particular, the relevance matching layer captures fine-grained local matching signals, while coarse-grained semantic similarity signals come from the semantic matching layer. Further, based on collaborative filtering in different directions, the IR layer works to find whether bug reports and source files are related, which indirectly transforms the matching task of different grammatical structures between bug reports and source files into the same structure and solves the mismatching problem of the first two matching models when the query is short. In our work, four benchmark data sets are used as experimental data sets and Accuracy@k, MAP, and MRR as evaluation metrics, which are used to compare and analyze the performance of bug localization with the four state-of-the-art methods. Experimental results show that SBugLocater outperforms the four models. For example, compared with the best of the four models, the evaluation metrics of Accuracy@10, MAP, and MRR are improved by 6.9%, 13.9%, and 17%, respectively.

Text Mining Approaches for Dependent Bug Report Assembly and Severity Prediction

In general, most existing bug report studies focus only on solving a single specific issue. Considering of multiple issues at one is required for a more complete and comprehensive process of bug fixing. We took up this challenge and proposed a method to analyze two issues of bug reports based on text mining techniques. Firstly, dependent bug reports are assembled into an individual cluster and then the bug reports in each cluster are analyzed for their severity. The method of dependent bug report assembly is experimented with threshold-based similarity analysis. Cosine similarity and BM25 are compared with term frequency (tf) weighting to obtain the most appropriate method. Meanwhile, four classification algorithms namely Random Forest (RF), Support Vector Machines (SVM) with the RBF kernel function, Multinomial Naïve Bayes (MNB), and k-Nearest Neighbor (k-NN) are utilized to model the bug severity predictor with four term weighting schemes, i.e., tf, term frequency-inverse document frequency (tf-idf), term frequency-inverse class frequency (tf-icf), and term frequency-inverse gravity moment (tf-igm). After the experimentation process, BM25 was found to be the most appropriate for dependent bug report assemblage, while for severity prediction using tf-icf weighting on the RF method yielded the best performance value.

Demystifying the challenges and benefits of analyzing user-reported logs in bug reports

Logs in bug reports provide important debugging information for developers. During the debugging process, developers need to study the bug report and examine user-provided logs to understand the system executions that lead to the problem. Intuitively, user-provided logs illustrate the problems that users encounter and may help developers with the debugging process. However, some logs may be incomplete or inaccurate, which can cause difficulty for developers to diagnose the bug, and thus, delay the bug fixing process. In this paper, we conduct an empirical study on the challenges that developers may encounter when analyzing the user-provided logs and their benefits. In particular, we study both log snippets and exception stack traces in bug reports. We conduct our study on 10 large-scale open-source systems with a total of 1,561 bug reports with logs (BRWL) and 7,287 bug reports without logs (BRNL). Our findings show that: 1) BRWL takes longer time (median ranges from 3 to 91 days) to resolve compared to BRNL (median ranges from 1 to 25 days). We also find that reporters may not attach accurate or sufficient logs (i.e., developers often ask for additional logs in the Comments section of a bug report), which extends the bug resolution time. 2) Logs often provide a good indication of where a bug is located. Most bug reports (73%) have overlaps between the classes that generate the logs and their corresponding fixed classes. However, there is still a large number of bug reports where there is no overlap between the logged and fixed classes. 3) Our manual study finds that there is often missing system execution information in the logs. Many logs only show the point of failure (e.g., exception) and do not provide a direct hint on the actual root cause. In fact, through call graph analysis, we find that 28% of the studied bug reports have the fixed classes reachable from the logged classes, while they are not visible in the logs attached in bug reports. In addition, some logging statements are removed in the source code as the system evolves, which may cause further challenges in analyzing the logs. In short, our findings highlight possible future research directions to better help practitioners attach or analyze logs in bug reports.

How are issue reports discussed in Gitter chat rooms?

Informal communication channels like mailing lists, IRC and instant messaging play a vital role in open source software development by facilitating communication within geographically diverse project teams e.g., to discuss issue reports to facilitate the bug-fixing process. More recently, chat systems like Slack and Gitter have gained a lot of popularity and developers are rapidly adopting them. Gitter is a chat system that is specifically designed to address the needs of GitHub users. Gitter hosts project-based asynchronous chats which foster frequent project discussions among participants. Developer discussions contain a wealth of information such as the rationale behind decisions made during the evolution of a project. In this study, we explore 24 open source project chat rooms that are hosted on Gitter, containing a total of 3,133,106 messages and 14,096 issue references. We manually analyze the contents of chat room discussions around 457 issue reports. The results of our study show the prevalence of issue discussions on Gitter, and that the discussed issue reports have a longer resolution time than the issue reports that are never brought on Gitter.

Laprob: A Label propagation-Based software bug localization method

Bug localization, which locates suspicious snippets related to the bugs mentioned in the bug reports, is time-consuming and laborious. Many automatic bug localization methods have been proposed to speed up the process of bug fixing and reduce the burden on developers. However, these methods have not fully utilized the intra-relations and inter-relations among the bug reports and the source files (i.e., call relationships between the source files).

Predicting the bug fixing time using word embedding and deep long short term memories

In bug fixing process, estimating the ‘Time to Fix Bug’ is one of the factors that helps the triager to allocate jobs in a better way. Due to the limitation of resources for bug fixing, the bugs with long fixing time must be identified, as soon as possible, after receiving the report. This helps the prioritisation and fixing process of the bug reports. In the process of bug fixing, a temporal sequence of activities is done. Each activity is represented by a term. Useful semantic information and long-term dependency are available between terms in the sequence, but it is usually underutilised by existing bug fixing time predictor approaches. This work presents a novel deep learning-based model (called DeepLSTMPred) that (i) converts constituent terms to a vector of real numbers by considering their semantic meaning, (ii) finds the long-term dependencies between terms by deep long short term memory (LSTM) and (iii) classifies sequences to short fixing time or long fixing time. DeepLSTMPred is evaluated on bug reports extracted from the Mozilla project. The results show that the proposed method has better performance in comparison with a state-of-the-art approach (that is the hidden Markov-based model). The experimental results show that DeepLSTMPred achieves 15–20% improvement in terms of accuracy, precision, f-score, and recall.

Analyzing bug fix for automatic bug cause classification

During the bug fixing process, developers usually need to analyze the source code to induce the bug cause, which is useful for bug understanding and localization. The bug fixes of historical bugs usually reflects the bug causes when fixing them. This paper aims at exploiting the corresponding relationship between bug causes and bug fixes to automatically classify bugs into their cause categories. First, we define the code-related bug classification criterion from the perspective of the cause of bugs. Then, we propose a new model to exploit the knowledge in the bug fix by constructing fix trees from the diff source code at Abstract Syntax Tree (AST) level, and representing each fix tree based on the encoding method of Tree-based Convolutional Neural Network (TBCNN). Finally, the corresponding relationship between bug causes and bug fixes is analyzed by automatically classifying bugs into their cause categories. We collected 2000 real-world bugs from two open source projects Mozilla and Radare2 to evaluate our approach. The experimental results show the existence of observational correlation between the bug fix and the cause of the historical bugs, and the proposed fix tree can effectively express the characteristics of the historical bugs for bug cause classification.

Performance Assessment of Bug Fixing Process in Open Source Repositories

Software bugs are inevitable. Large software projects often use bug tracking systems in order to collect, organize and keep track of reported bugs. The primary objective of using these bug tracking systems is to expedite the bug fix process which further improves the overall quality of software product. However, managing bug tracking systems incurs additional cost and efforts. Hence, it is important to monitor the performance of the bug fix process which is followed in bug tracking systems. The aim of this paper is to identify the performance characteristics of the bug fixing process in open source bug repositories. Various performance metrics have been investigated which measures the bug fix process quality. For experimental evaluation, bug reports are collected from Bugzilla repository of Mozilla Firefox, Eclipse, NetBeans and Open Office projects. It has been found that overall 45.67% of bug reports are closed with fixed resolution. Rest 54.33% bug reports keep lying in bug repository unresolved thereby creating an extra backlog of work for software developers. Further, the root causes have been determined which keeps these large portions of bug reports lying in bug repositories opened and found that Non-Reproducible bugs are the most deteriorating factor in bug fix process and affects the fix process performance negatively.

An empirical study of non-reproducible bugs

A large number of bug reports received in bug tracking system causes a negative impact on software’s growth and maintenance. Non-Reproducible bugs are an additional overhead on bug fixing process as certain percentage of Non-Reproducible bugs get fixed and these are difficult to detect. Their fixation can enhance the efficiency of bug fixing process. Past studies had ignored Non-Reproducible bugs while investigating bugs with varied perspectives. In this paper, we investigate characteristics of Non-Reproducible bug reports to understand their behaviour so that overall bug fixing process can be improved. We have also build machine learning models to predict fixable bugs from the set of Non-Reproducible marked bugs. We mine four large, open-source projects of Bugzilla repository and analyze Non-Reproducible bugs with different dimensions: frequency distribution, time-series analysis, active fix time, developer discussions, severity, priority and component. This study shows that there exists a significant number of Non-Reproducible bugs in each project and their occurrence follows a normal distribution with project longevity. They possess higher severity and lower priority as compared to reproducible bugs. Also, there exist specific components which are more prone to have non-reproducible bugs. We found that Non-Reproducible bugs consume more fixation time thereby affecting bug fixing processes’ efficiency. The overall results suggested that efforts in Non-Reproducible bugs could be minimized by developing a fixability prediction model that can build confidence among developers. Hence, we learned 52 classifiers for prediction of fixability among NR bugs. Our results show that prediction models achieved F1-Score up to 89.2%, 93.7%, 93.8% and 98.4% for NetBeans, Eclipse, Open Office and Mozilla Firefox projects respectively.

How Practitioners Perceive Automated Bug Report Management Techniques

Bug reports play an important role in the process of debugging and fixing bugs. To reduce the burden of bug report managers and facilitate the process of bug fixing, a great amount of software engineering research has been invested toward automated bug report management techniques. However, the verdict is still open whether such techniques are actually required and applicable outside the domain of theoretical research. To fill this gap, we conducted a survey among 327 practitioners to gain their insights into various categories of automated bug report management techniques. Specifically, we asked the respondents to rate the importance of such techniques and provide the rationale. To get deeper insights into practitioners’ perspective, we conducted follow-up interviews with 25 interviewees selected from the survey respondents. Through the survey and the interviews, we gained a better understanding of the perceived usefulness (or its lack) of different categories of automated bug report management techniques. Based on our findings, we summarized some potential research directions in developing techniques to help developers better manage bug reports.

Quantitative Quality Evaluation of Software Products by Considering Summary and Comments Entropy of a Reported Bug.

A software bug is characterized by its attributes. Various prediction models have been developed using these attributes to enhance the quality of software products. The reporting of bugs leads to high irregular patterns. The repository size is also increasing with enormous rate, resulting in uncertainty and irregularities. These uncertainty and irregularities are termed as veracity in the context of big data. In order to quantify these irregular and uncertain patterns, the authors have appliedentropy-based measures of the terms reported in the summary and the comments submitted by the users. Both uncertainties and irregular patterns have been taken care of byentropy-based measures. In this paper, the authors considered that the bug fixing process does not only depend upon the calendar time, testing effort and testing coverage, but it also depends on the bug summary description and comments. The paper proposed bug dependency-based mathematical models by considering the summary description of bugs and comments submitted by users in terms of the entropy-based measures. The models were validated on different Eclipse project products. The models proposed in the literature have different types of growth curves. The models mainly follow exponential, S-shaped or mixtures of both types of curves. In this paper, the proposed models were compared with the modelsfollowingexponential, S-shaped and mixtures of both types of curves.

Improving bug localization with word embedding and enhanced convolutional neural networks

Context: Automatic localization of buggy files can speed up the process of bug fixing to improve the efficiency and productivity of software quality assurance teams. Useful semantic information is available in bug reports and source code, but it is usually underutilized by existing bug localization approaches.Objective: To improve the performance of bug localization, we propose DeepLoc, a novel deep learning-based model that makes full use of semantic information.Method: DeepLoc is composed of an enhanced convolutional neural network (CNN) that considers bug-fixing recency and frequency, together with word-embedding and feature-detecting techniques. DeepLoc uses word embeddings to represent the words in bug reports and source files that retain their semantic information, and different CNNs to detect features from them. DeepLoc is evaluated on over 18,500 bug reports extracted from AspectJ, Eclipse, JDT, SWT, and Tomcat projects.Results: The experimental results show that DeepLoc achieves 10.87%–13.4% higher MAP (mean average precision) than conventional CNN. DeepLoc outperforms four current state-of-the-art approaches (DeepLocator, HyLoc, LR+WE, and BugLocator) in terms of Accuracy@k (the percentage of bug reports for which at least one real buggy file is located within the top k rank), MAP, and MRR (mean reciprocal rank) using less computation time.Conclusion: DeepLoc is capable of automatically connecting bug reports to the corresponding buggy files and achieves better performance than four state-of-the-art approaches based on a deep understanding of semantics in bug reports and source code.

Are tweets useful in the bug fixing process? An empirical study on Firefox and Chrome

When encountering an issue, technical users (e.g., developers) usually file the issue report to the issue tracking systems. But non-technical end-users are more likely to express their opinions on social network platforms, such as Twitter. For software systems (e.g., Firefox and Chrome) that have a high exposure to millions of non-technical end-users, it is important to monitor and solve issues observed by a large user base. The widely used micro-blogging site (i.e., Twitter) has millions of active users. Therefore, it can provide instant feedback on products to the developers. In this paper, we investigate whether social networks (i.e., Twitter) can improve the bug fixing process by analyzing the short messages posted by end-users on Twitter (i.e., tweets). We propose an approach to remove noisy tweets, and map the remaining tweets to bug reports. We conduct an empirical study to investigate the usefulness of Twitter in the bug fixing process. We choose two widely adopted browsers (i.e., Firefox and Chrome) that are also large and rapidly released software systems. We find that issue reports are not treated differently regardless whether users tweet about the issue or not, except that Firefox developers tend to label an issue as more severe if users tweet about it. The feedback from Firefox contributors confirms that the tweets are not currently leveraged in the bug fixing process, due to the challenges associated to discovering bugs through Twitter. Moreover, we observe that many issues are posted on Twitter earlier than on issue tracking systems. More specifically, at least one third of issues could have been reported to developers 8.2 days and 7.6 days earlier in Firefox and Chrome, respectively. In conclusion, tweets are useful in providing earlier acknowledgment of issues, which developers can potentially use to focus their efforts on the issues impacting a large user-base.

Automated Bug Report Field Reassignment and Refinement Prediction

Bug fixing is one of the most important activities in software development and maintenance. Bugs are reported, recorded, and managed in bug tracking systems such as Bugzilla. In general, a bug report contains many fields, such as product, component, severity, priority, fixer, operating system (OS), and platform, which provide important information for the bug triaging and fixing process. Our previous study finds that approximately 80% of bug reports have their fields reassigned and refined at least once, and bugs with reassigned and refined fields take more time to fix than bugs with no reassigned and refined fields. Thus, automatically predicting which bug report fields get reassigned and refined could help developers to save bug fixing time. Considering that a bug report could have multiple field reassignments and refinements (e.g., the product, component, fixer, and other fields of a bug report can get reassigned and refined), in this paper, we propose a multi-label learning algorithm to predict which bug report fields might be reassigned and refined. We note that the number of bug reports with some types of reassignment and refinement (e.g., bugs whose severity fields gets reassigned and refined) is a small proportion of the whole bug report collection, indicating the class imbalance problem. Thus, we propose imbalanced ML.KNN (Im-ML.KNN), which extends ML.KNN, one of the state-of-the-art multi-label learning algorithms, to achieve better performance. Im-ML.KNN is a composite model that combines 3 multi-label classifiers built using different types of features (i.e., meta, textual, and mixed features). We evaluate our solution on 4 large bug report datasets including OpenOffice, Netbeans, Eclipse, and Mozilla containing a total of 190,558 bug reports. We show that Im-ML.KNN can achieve an average F-measure score of 0.56–0.62. We also compare Im-ML.KNN with other state-of-art methods, such as the method proposed by Lamkanfi , ML.KNN, and HOMER-NB. The results show that Im-ML.KNN, on average, improves the average F-measure scores of Lamkanfi 's method, ML.KNN, and HOMER-NB by 119.69%, 9.11%, and 161.08%, respectively.

Guiding Bug Triage through Developer Analysis in Bug Reports

An important part of software maintenance is bug report analysis during bug-fixing, especially for large-scale software projects. Since bugs reported to the bug repository need to be fixed, triagers are responsible to identify appropriate developers to execute the fix. Previous research focused on optimizing this process, such as by duplicate detection and use of developer recommendations for reducing the workload of triagers. However, there were scant studies that analyzed developer roles (e.g. reporter and assignee) in the bug-fixing process. Therefore, in this paper, we perform an in-depth empirical study of the different roles that developers perform in bug resolution. By extracting the factors that affect bug resolution from the analysis results, we propose a novel bug triage algorithm to recommend the appropriate developers to fix a given bug. We implement the proposed recommendations on the Eclipse and Mozilla Firefox projects, with the results showing that the new bug triage algorithm can effectively recommend which experts should fix given bugs.

토픽 모델과 버그 리포트 메타 필드를 이용한 버그 심각도 예측 방법

Recently developed software systems have many components, and their complexity is thus increasing. Last year, about 375 bug reports in one day were reported to a software repository in Eclipse and Mozilla open source projects. With so many bug reports submitted, developers' time and efforts have increased unnecessarily. Since the bug severity is manually determined by quality assurance, project manager or other developers in the general bug fixing process, it is biased to them. They might also make a mistake on the manual decision because of the large number of bug reports. Therefore, in this study, we propose an approach of bug severity prediction to solve these problems. First, we find similar topics within a new bug report and reduce the candidate reports of the topic by using the meta field of the bug report. Next, we train the reduced reports by applying Naive Bayes Multinomial. Finally, we predict the severity of the new bug report. We compare our approach with other prediction algorithms by using bug reports in open source projects. The results show that our approach better predicts bug severity than other algorithms.

Understanding the triaging and fixing processes of long lived bugs

ContextBug fixing is an integral part of software development and maintenance. A large number of bugs often indicate poor software quality, since buggy behavior not only causes failures that may be costly but also has a detrimental effect on the user’s overall experience with the software product. The impact of long lived bugs can be even more critical since experiencing the same bug version after version can be particularly frustrating for user. While there are many studies that investigate factors affecting bug fixing time for entire bug repositories, to the best of our knowledge, none of these studies investigates the extent and reasons of long lived bugs. ObjectiveIn this paper, we investigate the triaging and fixing processes of long lived bugs so that we can identify the reasons for delay and improve the overall bug fixing process. MethodologyWe mine the bug repositories of popular open source projects, and analyze long lived bugs from five different perspectives: their proportion, severity, assignment, reasons, as well as the nature of fixes. ResultsOur study on seven open-source projects shows that there are a considerable number of long lived bugs in each system and over 90% of them adversely affect the user’s experience. The reasons for these long lived bugs are diverse including long assignment time, not understanding their importance in advance, etc. However, many bug-fixes were delayed without any specific reasons. Furthermore, 40% of long lived bugs need only small fixes. ConclusionOur overall results suggest that a significant number of long lived bugs may be minimized through careful triaging and prioritization if developers could predict their severity, change effort, and change impact in advance. We believe our results will help both developers and researchers better to understand factors behind delays, improve the overall bug fixing process, and investigate analytical approaches for prioritizing bugs based on bug severity as well as expected bug fixing effort.

A methodology for quantitatively managing the bug fixing process using Mahalanobis Taguchi system

Article history: Received March 5, 2015 Received in revised format August 16 2015 Accepted October 13 2015 Available online October 15 2015 The controlling of bug fixing process during the system testing phase of software development life cycle is very important for fixing all the detected bugs within the scheduled time. The presence of open bugs often delays the release of the software or result in releasing the software with compromised functionalities. These can lead to customer dissatisfaction, cost overrun and eventually the loss of market share. In this paper, the authors propose a methodology to quantitatively manage the bug fixing process during system testing. The proposed methodology identifies the critical milestones in the system testing phase which differentiates the successful projects from the unsuccessful ones using Mahalanobis Taguchi system. Then a model is developed to predict whether a project is successful or not with the bug fix progress at critical milestones as control factors. Finally the model is used to control the bug fixing process. It is found that the performance of the proposed methodology using Mahalanobis Taguchi system is superior to the models developed using other multi-dimensional pattern recognition techniques. The proposed methodology also reduces the number of control points providing the managers with more options and flexibility to utilize the bug fixing resources across system testing phase. Moreover the methodology allows the mangers to carry out midcourse corrections to bring the bug fixing process back on track so that all the detected bugs can be fixed on time. The methodology is validated with eight new projects and the results are very encouraging. Growing Science Ltd. All rights reserved. 5 © 201

UNDERSTANDING THE DEVELOPER PARTICIPATION IN BUG FIX PROCESS

Prediction of the bug fix time in open source softwares is a challenging job. A software bug consists of many attributes that define the characteristics of the bug. Some of the attributes get filled at the time of reporting and some are at the time of bug fixing. In this paper, 836 bug reports of two products namely Thunderbird and Webtools of Mozilla open source project have been considered. In bug report, we see that there is no linear relationship among the bug attributes namely bug fix time, developers, cc count and severity. This paper has analyzed the interdependence among these attributes through graphical representation.The results conclude that :Case 1. 73% of bugs reported for Webtools are fixed by 17% developers and 61% of bugs are fixed by 14% developers for Thundebird.Case 2. We tried to find a relationship between the time taken by a developer in fixing a bug and the corresponding developer. We also observed that there is a significant variation in bug fixing process, bugs may take 1 day to 4 years in fixing.Case 3. There is no linear relationship between cc count i.e. manpower involved in bug fixing process and bug fix time.Case 4. Maximum number of developers are involved in fixing bugs for major severity class.