Bug Prediction Research Articles

Bert-Based Feature Extraction for Long-Lived Bug Prediction in Floss: A Comparative Study

Bert-Based Feature Extraction for Long-Lived Bug Prediction in Floss: A Comparative Study

Comments on "Using k-core Decomposition on Class Dependency Networks to Improve Bug Prediction Model's Practical Performance"

In a very recent paper by Qu et al. (IEEE Transactions on Software Engineering, vol. 47 no. 2, pp. 348-366, Feb. 1 2021, doi:10.1109/TSE.2019.2892959), the authors propose an effective equation, top-core, to improve the performance of effort-aware bug prediction models. A distinctive feature of top-core is that it takes into account the coreness of a class in a Class Dependency Network (CDN) when calculating the relative risk of a class to be buggy. In this comment, we show that Qu et al.'s paper contains three shortcomings that may influence the performance of top-core or even have the potential to lead to erroneous results. First, we show that the CDN that they use to calculate the coreness of classes is not very accurate, neglecting many important types of dependency relations between classes such as method call relation, access relation, and instantiates relation. Second, they trained a Logistic Regression model using the scikit-learn framework to predict the probability of a specific class to be buggy. It is actually an L2 regularized Logistic Regression model, which is dependent on the scale of the features. But they neglected to normalize the features, making the obtained results erroneous. Finally, the number of execution times (viz. 10 times in the paper of Qu et al.) they used to reduce the bias caused by the randomness (viz. random split of instances and the process to handle class-imbalance problem) in the experiments is too small to ensure that the obtained results converge to stable values; but they failed to signify the precision level of their results for comparison. In this comment, we provide solutions to the problems by using i) an improved CDN (ICDN) to represent the structure of software systems, ii) the z-score method to normalize the features, and iii) an adaptive mechanism to determine the number of execution times. In the experiments, we find that Qu et al.'s approach based on the Logistic Regression model does not perform significantly better than the state-of-the-art approach Ree, which is inconsistent with the conclusion in Qu et al.'s work. We also observe that replacing CDN with ICDN does improve the performance of Qu et al.'s approach.

Copula-based software metrics aggregation

A quality model is a conceptual decomposition of an abstract notion of quality into relevant, possibly conflicting characteristics and further into measurable metrics. For quality assessment and decision making, metrics values are aggregated to characteristics and ultimately to quality scores. Aggregation has often been problematic as quality models do not provide the semantics of aggregation. This makes it hard to formally reason about metrics, characteristics, and quality. We argue that aggregation needs to be interpretable and mathematically well defined in order to assess, to compare, and to improve quality. To address this challenge, we propose a probabilistic approach to aggregation and define quality scores based on joint distributions of absolute metrics values. To evaluate the proposed approach and its implementation under realistic conditions, we conduct empirical studies on bug prediction of ca. 5000 software classes, maintainability of ca. 15000 open-source software systems, and on the information quality of ca. 100000 real-world technical documents. We found that our approach is feasible, accurate, and scalable in performance.

Comparing fine-grained source code changes and code churn for bug prediction - A retrospective

More than two decades ago, researchers started to mine the data stored in software repositories to help software developers in making informed decisions for developing and testing software systems. Bug prediction was one of the most promising and popular research directions that uses the data stored in software repositories to predict the bug-proneness or number of bugs in source files. On that topic and as part of Emanuel's PhD studies, we submitted a paper with the title Comparing fine-grained source code changes and code churn for bug prediction [8] to the 8th Working Conference on Mining Software Engineering, held 2011 in beautiful Honolulu, Hawaii. Ten years later, it got selected as one of the finalists to receive the MSR 2021 Most Influential Paper Award. In the following, we provide a retrospective on our work, describing the road to publishing this paper, its impact in the field of bug prediction, and the road ahead.

Hybrid Optimization of Multiple Intelligent Recommendation Engines for Software Development Cycles

The primary purpose is to create a hybrid recommendation system approach to improve the performance of such systems. This recommendation system would typically be used to assign or suggest a small number of developers suitable for troubleshooting a bug report. For example, managing collections inside bug repositories is software developers' task to fix any bugs that have been identified. Unfortunately, bugs are often created, so the number of developers needed is high, so it's hard to decide how many to assign to specific tasks.This better aims better to understand the outcomes of the latestscientific methods. We also addressed developer prioritization and how it can be used to determine the assignment of a problem to a developer. We have studied two aspects: first, selecting bug reports using hybrid machine learning methods, modeling prioritization in the bug repository, and supporting developer assignment tasks with our model. Second, we modeled the relevant objectives suggested by the developers' backgrounds based on proven knowledge and experience. The study focuses on two topers' experience with fixing bugs and developer rankings in the App Store. We've tried to take better assignments using developer prioritization in bug repositories, e.g., bug triage, severity identification, and re-opened bug prediction. We examine the output of the model in a representative sample of bug repositories. The results show that the prioritization of developers' prioritization triage worker and allow the program to solve the bugs more effectively in support of the software support has been clarified. The introduction article, section 2 on the literature and context, section 3 on the work contribution that will be made, section 4 on the methodology analysis and the expected outcomes will be explained, section 5 on the conclusion, and finally, on the potentialaspects of this work

Leveraging developer information for efficient effort-aware bug prediction

Software bug prediction techniques can provide informative guidance in software engineering practices. Over the past 15 years, developer information has been intensively used in bug prediction as features or basic data source to construct other useful models. Further leverage developer information from a new and straightforward perspective to improve effort-aware bug prediction. We propose to investigate the direct relations between the number of developers and the probability for a file to be buggy. Based on an empirical study on nine open-source Java systems with 32 versions, we observe a widely-existed and interesting tendency: when there are more developers working on a source file, there will be a stronger possibility for this file to be buggy. Based on the observed tendency, we propose an unsupervised algorithm and a supervised equation both called top-dev to improve effort-aware bug prediction. The key idea is to prioritize the ranking of files, whose number of developers is large, in the suspicious file list generated by effort-aware models. Experimental results show that the proposed top-dev algorithm and equation significantly outperform the unsupervised and supervised baseline models (ManualUp, Rad, Rdd, Ree, CBS+, and top-core). Moreover, the unsupervised top-dev algorithm is comparable or superior to existing supervised baseline models. The proposed approaches are very useful in effort-aware bug prediction practices. Practitioners can use the top-dev algorithm to generate a high-quality and informative suspicious file list without training complex machine learning classifiers. On the other hand, when building supervised bug prediction model, the best practice is to combine existing models with the top-dev equation.

Evaluating network embedding techniques’ performances in software bug prediction

Software bug prediction techniques can be very helpful in testing and code inspection. Over the past decade, network measures have been successfully used in bug prediction. Following the same intuition, recently, researchers started using network embedding techniques in bug prediction. However, existing studies only evaluated the Skip-gram and CBOW models with random walk. Considering network embedding is a fast-developing research direction, it is important to evaluate other network embedding techniques’ performances in bug prediction. Moreover, existing studies have not investigated the application and performance of network embedding in effort-aware bug prediction, which is thought to be a more realistic scenario that evaluates the cost effectiveness of bug prediction models. In this paper, we conduct an extensive empirical study to evaluate network embedding algorithms in bug prediction by utilizing and extending node2defect, a newly proposed bug prediction model that combines the embedded vectors with traditional software engineering metrics through concatenation. Experiments are conducted based on seven network embedding algorithms, two effort-aware models, and 13 open-source Java systems. Experimental results show that node2defect outperforms traditional metrics by + 14.64% in terms of MCC score, and by + 7.51% to + 16.57% in effort-aware bug prediction. More interestingly, when combined with CBS + , the embedded vectors alone can achieve the best performance. Among different network embedding algorithms, the newly proposed algorithm ProNE has the best performance.

Effort-Aware Just-in-Time Bug Prediction for Mobile Apps Via Cross-Triplet Deep Feature Embedding

Just-in-time (JIT) bug prediction is an effective quality assurance activity that identifies whether a code commit will introduce bugs into the mobile app, aiming to provide prompt feedback to practitioners for priority review. Since collecting sufficient labeled bug data is not always feasible for some mobile apps, one possible approach is to leverage cross-app models. In this work, we propose a new cross-triplet deep feature embedding method, called CDFE, for cross-app JIT bug prediction task. The CDFE method incorporates a state-of-the-art cross-triplet loss function into a deep neural network to learn high-level feature representation for the cross-app data. This loss function adapts to the cross-app feature learning task and aims to learn a new feature space to shorten the distance of commit instances with the same label and enlarge the distance of commit instances with different labels. In addition, this loss function assigns higher weights to losses caused by cross-app instance pairs than that by intra-app instance pairs, aiming to narrow the discrepancy of cross-app bug data. We evaluate our CDFE method on a benchmark bug dataset from 19 mobile apps with two effort-aware indicators. The experimental results on 342 cross-app pairs show that our proposed CDFE method performs better than 14 baseline methods.

Watch Out for Extrinsic Bugs! A Case Study of Their Impact in Just-In-Time Bug Prediction Models on the OpenStack Project

Intrinsic bugs are bugs for which a bug-introducing change can be identified in the version control system of a software. In contrast, extrinsic bugs are caused by external changes to a software, such as errors in external APIs; thereby they do not have an explicit bug-introducing change in the version control system. Although most previous research literature has assumed that all bugs are of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">intrinsic</i> nature, in a previous study, we show that not all bugs are intrinsic. This paper shows an example of how considering extrinsic bugs can affect software engineering research. Specifically, we study the impact of extrinsic bugs in Just-In-Time bug prediction by partially replicating a recent study by McIntosh and Kamei on JIT models. These models are trained using properties of earlier bug-introducing changes. Since extrinsic bugs do not have bug-introducing changes in the version control system, we manually curate McIntosh and Kamei’s dataset to distinguish between intrinsic and extrinsic bugs. Then, we address their original research questions, this time removing extrinsic bugs, to study whether bug-introducing changes are a moving target in Just-In-Time bug prediction. Finally, we study whether characteristics of intrinsic and extrinsic bugs are different. Our results show that intrinsic and extrinsic bugs are of different nature. When removing extrinsic bugs the performance is different up to 16 percent Area Under the Curve points. This indicates that our JIT models obtain a more accurate representation of the real world. We conclude that extrinsic bugs negatively impact Just-In-Time models. Furthermore, we offer evidence that extrinsic bugs should be further investigated, as they can significantly impact how software engineers understand bugs.

An empirical study on the effect of community smells on bug prediction

Community-aware metrics through socio-technical developer networks or organizational structures have already been studied in the software bug prediction field. Community smells are also proposed to identify communication and collaboration patterns in developer communities. Prior work reports a statistical association between community smells and code smells identified in software modules. We investigate the contribution of community smells on predicting bug-prone classes and compare their contribution with that of code smell-related information and state-of-the-art process metrics. We conduct our empirical analysis on ten open-source projects with varying sizes, buggy and smelly class ratios. We build seven different bug prediction models to answer three RQs: a baseline model including a state-of-the-art metric set used, three models incorporating a particular metric set, namely community smells, code smells, code smell intensity, into the baseline, and three models incorporating a combination of smell-related metrics into the baseline. The performance of these models is reported in terms of recall, false positive rates, F-measure and AUC and statistically compared using Scott–Knott ESD tests. Community smells improve the prediction performance of a baseline model by up to 3% in terms of AUC, while code smell intensity improves the baseline models by up to 40% in terms of F-measure and up to 17% in terms of AUC. The conclusions are significantly influenced by the validation strategies used, algorithms and the selected projects’ data characteristics. While the code smell intensity metric captures the most information about technical flaws in predicting bug-prone classes, the community smells also contribute to bug prediction models by revealing communication and collaboration flaws in software development teams. Future research is needed to capture the communication patterns through multiple channels and to understand whether socio-technical flaws could be used in a cross-project bug prediction setting.

Generative Adversarial Networks-Based Imbalance Learning in Software Aging-Related Bug Prediction

Software aging refers to a problem of performance decay in the software systems, which are running for a long period. The primary cause of this phenomenon is the accumulation of run-time errors in the software, which are also known as aging-related bugs (ARBs). Many efforts have been reported earlier to predict the origin of ARBs in the software so that these bugs can be identified and fixed during testing. Imbalanced dataset, where the representation of ARBs patterns is very less as compared to the representation of the non-ARBs pattern significantly hinders the performance of the ARBs prediction models. Therefore, in this article, we present an oversampling approach, generative adversarial networks-based synthetic data generation-based ARBs prediction models. The approach uses generative adversarial networks to generate synthetic samples for the ARBs patterns in the given datasets implicitly and build the prediction models on the processed datasets. To validate the performance of the presented approach, we perform an experimental study for the seven ARBs datasets collected from the public repository and use various performance measures to evaluate the results. The experimental results showed that the presented approach led to the improved performance of prediction models for the ARBs prediction as compared to the other state-of-the-art models.

Using K-core Decomposition on Class Dependency Networks to Improve Bug Prediction Model's Practical Performance

In recent years, Complex Network theory and graph algorithms have been proved to be effective in predicting software bugs. On the other hand, as a widely-used algorithm in Complex Network theory, k-core decomposition has been used in software engineering domain to identify key classes. Intuitively, key classes are more likely to be buggy since they participate in more functions or have more interactions and dependencies. However, there is no existing research uses k -core decomposition to analyze software bugs. To fill this gap, we first use k -core decomposition on Class Dependency Networks to analyze software bug distribution from a new perspective. An interesting and widely existed tendency is observed: for classes in k -cores with larger k values, there is a stronger possibility for them to be buggy. Based on this observation, we then propose a simple but effective equation named as top-core which improves the order of classes in the suspicious class list produced by effort-aware bug prediction models. Based on an empirical study on 18 open-source Java systems, we show that the bug prediction models’ performances are significantly improved in 85.2 percent experiments in the cross-validation scenario and in 80.95 percent experiments in the forward-release scenario, after using top-core . The models’ average performances are improved by 11.5 and 12.6 percent, respectively. It is concluded that the proposed top-core equation can help the testers or code reviewers locate the real bugs more quickly and easily in software bug prediction practices.

Adaptive selection of classifiers for bug prediction: A large-scale empirical analysis of its performances and a benchmark study

Bug prediction aims at locating defective source code components relying on machine learning models. Although some previous work showed that selecting the machine-learning classifier is crucial, the results are contrasting. Therefore, several ensemble techniques, i.e., approaches able to mix the output of different classifiers, have been proposed. In this paper, we present a benchmark study in which we compare the performance of seven ensemble techniques on 21 open-source software projects. Our aim is twofold. On the one hand, we aim at bridging the limitations of previous empirical studies that compared the accuracy of ensemble approaches in bug prediction. On the other hand, our goal is to verify how ensemble techniques perform in different settings such as cross- and local-project defect prediction. Our empirical experimentation results show that ensemble techniques are not a silver bullet for bug prediction. In within-project bug prediction, using ensemble techniques improves the prediction performance with respect to the best stand-alone classifier. We confirm that the models based on Validation and Voting achieve slightly better results. However, they are similar to those obtained by other ensemble techniques. Identifying buggy classes using external sources of information is still an open problem. In this setting, the use of ensemble techniques does not provide evident benefits with respect to stand-alone classifiers. The statistical analysis highlights that local and global models are mostly equivalent in terms of performance. Only one ensemble technique ( i.e., ASCI) slightly exploits local learning to improve performance.

Comprehensive Study on Machine Learning Techniques for Software Bug Prediction

Software bugs are defects or faults in computer programs or systems that cause incorrect or unexpected operations. These negatively affect software quality, reliability, and maintenance cost; therefore many researchers have already built and developed several models for software bug prediction. Till now, a few works have been done which used machine learning techniques for software bug prediction. The aim of this paper is to present comprehensive study on machine learning techniques that were successfully used to predict software bug. Paper also presents a software bug prediction model based on supervised machine learning algorithms are Decision Tree (DT), Naïve Bayes (NB), Random Forest (RF) and Logistic Regression (LR) on four datasets. We compared the results of our proposed models with those of the other studies. The results of this study demonstrated that our proposed models performed better than other models that used the same data sets. The evaluation process and the results of the study show that machine learning algorithms can be used effectively for prediction of bugs.

Predicting Bug Priority Using Topic Modelling in Imbalanced Learning Environments

Manual classification of bug reports is time-consuming as the reports are received in large quantities. Alternatively, this project proposed automatic bug prediction models to classify the bug reports. The topics or the candidate keywords are mined from the developer description in bug reports using RAKE algorithm and converted into attributes. These attributes together with the target attribute—priority level—construct the training datasets. Naïve Bayes, logistic regression, and decision tree learner algorithms are trained, and the prediction quality was measured using area under recursive operative characteristics curves (AUC) as AUC does not consider the biasness in datasets. The logistics regression model outperforms the other two models providing the accuracy of 0.86 AUC whereas the naïve Bayes and the decision tree learner recorded 0.79 AUC and 0.81 AUC, respectively. The bugs can be classified without developer involvement and logistic regression is also a potential candidate as naïve Bayes for bug classification.

Enhanced Bug Prediction in JavaScript Programs with Hybrid Call-Graph Based Invocation Metrics

Bug prediction aims at finding source code elements in a software system that are likely to contain defects. Being aware of the most error-prone parts of the program, one can efficiently allocate the limited amount of testing and code review resources. Therefore, bug prediction can support software maintenance and evolution to a great extent. In this paper, we propose a function level JavaScript bug prediction model based on static source code metrics with the addition of a hybrid (static and dynamic) code analysis based metric of the number of incoming and outgoing function calls (HNII and HNOI). Our motivation for this is that JavaScript is a highly dynamic scripting language for which static code analysis might be very imprecise; therefore, using a purely static source code features for bug prediction might not be enough. Based on a study where we extracted 824 buggy and 1943 non-buggy functions from the publicly available BugsJS dataset for the ESLint JavaScript project, we can confirm the positive impact of hybrid code metrics on the prediction performance of the ML models. Depending on the ML algorithm, applied hyper-parameters, and target measures we consider, hybrid invocation metrics bring a 2–10% increase in model performances (i.e., precision, recall, F-measure). Interestingly, replacing static NOI and NII metrics with their hybrid counterparts HNOI and HNII in itself improves model performances; however, using them all together yields the best results.

On the prediction of long-lived bugs: An analysis and comparative study using FLOSS projects

Software evolution and maintenance activities in today’s Free/Libre Open Source Software (FLOSS) rely primarily on information extracted from bug reports registered in bug tracking systems. Many studies point out that most bugs that adversely affect the user’s experience across versions of FLOSS projects are long-lived bugs. However, proposed approaches that support bug fixing procedures do not consider the real-world lifecycle of a bug, in which bugs are often fixed very fast. This may lead to useless efforts to automate the bug management process. This study aims to confirm whether the number of long-lived bugs is significantly high in popular open-source projects and to characterize the population of long-lived bugs by considering the attributes of bug reports. We also aim to conduct a comparative study evaluating the prediction accuracy of five well-known machine learning algorithms and text mining techniques in the task of predicting long-lived bugs. We collected bug reports from six popular open-source projects repositories (Eclipse, Freedesktop, Gnome, GCC, Mozilla, and WineHQ) and used the following machine learning algorithms to predict long-lived bugs: K-Nearest Neighbor, Naïve Bayes, Neural Networks, Random Forest, and Support Vector Machines. Our results show that long-lived bugs are relatively frequent (varying from 7.2% to 40.7%) and have unique characteristics, confirming the need to study solutions to support bug fixing management. We found that the Neural Network classifier yielded the best results in comparison to the other algorithms evaluated. Research efforts regarding long-lived bugs are needed and our results demonstrate that it is possible to predict long-lived bugs with a high accuracy (around 70.7%) despite the use of simple prediction algorithms and text mining methods.

Artificial Intelligence for Software Engineering: An Initial Review on Software Bug Detection and Prediction

The need for speed and quality in delivering all software engineering artifacts has inevitably remained the biggest challenge in today’s software development environment. While everyone caters to complex software engineering processes, new releases are expected by the market on almost a daily basis. Thus, several Artificial Intelligence (AI) techniques have been introduced that are intensively used in the modern software engineering industry to fulfill market needs. This paper presents the initial results of our review work on software bug detection and prediction studies using AI techniques. Our focus is to (i) identify factors affecting the effectiveness of current software bug detection and prediction techniques and (ii) identify the effectiveness of AI techniques in improving current software bug detection and prediction techniques. The evidence showed that the software engineering domain has utilized artificial intelligence approaches and techniques to facilitate the complex tasks of software bug detection and bug prediction. It mainly demonstrates the significance of merging artificial intelligence with the software engineering domain in terms of reduced overhead and efficient results to enhance the quality of software products.

WITHDRAWN: Software bug prediction and classification by global pooling of different activation of convolution layers

Abstract To enhance the reliability of the software, the prediction of software defects is being used to identify software bugs and prioritize testing efforts. Some authors have successfully adopted deep learning models, like the convolutional neural network (CNN), and Deep Belief Network (DBN), and used automated features extracted from deep learning models and abstract syntax trees (ASTs) to boost efficiency in the prediction of defects. However, due to its insufficient repeated experiments, limited dataset size, and obsolete baseline choice, investigation mostly on the CNN model failed to present conclusive results. To fix these issues, we built the PROMISE Source Code (PSC) dataset to broaden the original data set throughout the CNN studies, that designers labelled the Simplified PROMISE Source Code (SPSC) dataset. All the variations have been used in the dataset. The proposed methodology uses CNN hybridization with distinct activation function with global pooling and then learning by additional tree classification.

Development of software defect prediction system using artificial neural network

&lt;span&gt;Software testing is an activity to enable a system is bug free during execution process. The software bug prediction is one of the most encouraging exercises of the testing phase of the software improvement life cycle. In any case, in this paper, a framework was created to anticipate the modules that deformity inclined in order to be utilized to all the more likely organize software quality affirmation exertion. Genetic Algorithm was used to extract relevant features from the acquired datasets to eliminate the possibility of overfitting and the relevant features were classified to defective or otherwise modules using the Artificial Neural Network. The system was executed in MATLAB (R2018a) Runtime environment utilizing a statistical toolkit and the performance of the system was assessed dependent on the accuracy, precision, recall, and the f-score to check the effectiveness of the system. In the finish of the led explores, the outcome indicated that ECLIPSE JDT CORE, ECLIPSE PDE UI, EQUINOX FRAMEWORK and LUCENE has the accuracy, precision, recall and the f-score of 86.93, 53.49, 79.31 and 63.89% respectively, 83.28, 31.91, 45.45 and 37.50% respectively, 83.43, 57.69, 45.45 and 50.84% respectively and 91.30, 33.33, 50.00 and 40.00% respectively. This paper presents an improved software predictive system for the software defect detections.&lt;/span&gt;