Software Defect Prediction Research Articles

Analyzing the Role of Class Rebalancing Techniques in Software Defect Prediction

Predicting software defects is an important task during software testing phase, especially for allocating appropriate resources and prioritizing testing tasks. Typically, classification algorithms are used to accomplish this task by using previously collected datasets. However, these datasets suffer from imbalanced label distribution where clean modules outnumber defective modules. Traditional classification algorithms cannot handle this nature in defect datasets because they assume the datasets are balanced. Failing to address this problem, the classification algorithm will produce a prediction biased towards the majority label. In the literature, there are several techniques designed to address this problem and most of them focus on data re-balancing. Recently, ensemble class imbalance techniques have emerged as an opposing approach to data rebalancing approaches. Regarding the software defect prediction, there are no studies examining the performance of ensemble class imbalance learning against data re-balancing approaches. This paper investigates the efficiency of ensemble class imbalance learning for software defect prediction. We conducted a comprehensive experiment that involved 12 datasets, six classifiers, nine class imbalance techniques, and 10 evaluation metrics. The experiments showed that ensemble approaches, particularly the Under Bagging technique, outperform traditional data re-balancing approaches, particularly when dealing with datasets that have high defect ratios.

Studying the effectiveness of deep active learning in software defect prediction

Accurate prediction of defective software modules is of great importance for prioritizing quality assurance efforts, reasonably allocating testing resources, reducing costs and improving software quality. Several studies have used machine learning to predict software defects. However, complex structures and imbalanced class distributions in software defect data make learning an effective defect prediction model challenging. In this article, two deep learning-based defect prediction models using static code metrics are proposed. In order to enhance the learning process and improve the performance of the proposed models, pool-based active learning is employed. In this regard, the possibility of using active learning to mitigate the need for a large amount of labeled data in the process of building deep learning models is investigated. To deal with imbalanced distribution of software modules between defective and non-defective classes, Near-Miss under-sampling and KNN, with different number of neighbors, are used. The reason for choosing them is their good performance in binary classification problems. Experiments are performed on two well-known, publicly available datasets, GitHub Bug Dataset and public Unified Bug Dataset for java projects. The evaluation results reveal the effectiveness of our proposed models in comparison to the traditional machine learning algorithms. In the conducted investigations on the Unified Bug Dataset, at the file level, the value of F-measure and AUC criteria have improved by 13 and 11 percent, respectively and at the class level, the values have improved by 14 and 11 percent, respectively.

Reliable prediction of software defects using Shapley interpretable machine learning models

Predicting defect-prone software components can play a significant role in allocating relevant testing resources to fault-prone modules and hence increasing the business value of software projects. Most of the current software defect prediction studies utilize traditional supervised machine learning algorithms to predict defects in software applications. The software datasets utilized in such studies are imbalanced and therefore the reported results cannot be reliably used to judge their performance. Moreover, it is important to explain the output of machine learning models employed in fault-predication techniques to determine the contribution of each utilized feature to the model output. In this paper, we propose a new framework for predicting software defects utilizing eleven machine learning classifiers over twelve different datasets. For feature selection, we employ four different nature-inspired search algorithms, namely, particle swarm optimization, genetic algorithm, harmony algorithm, and ant colony optimization. Moreover, we make use of the synthetic minority oversampling technique (SMOTE) to address the problem of data imbalance. Furthermore, we utilize the Shapley additive explanation model for highlighting the highest determinative features. The obtained results demonstrate that gradient boosting, stochastic gradient boosting, decision trees, and categorical boosting outperform others tested model with over 90% accuracy and ROC-AUC. Additionally, we found that the ant colony optimization technique outperforms the other tested feature extraction techniques.

Software defects prediction by metaheuristics tuned extreme gradient boosting and analysis based on Shapley Additive Explanations

Software testing represents a crucial component of software development, and it is usually making the difference between successful and failed projects. Although it is extremely important, due to the fast pace and short deadlines of contemporary projects it is often neglected or not detailed enough due to the lack of available time, leading to the potential loss of reputation, private users’ data, money, and even lives in some circumstances. In such situations, it would be vital to have the option to predict what modules are error-prone according to the collection of software metrics, and to focus testing on them, and that task is a typical classification task. Machine learning models have been frequently employed within a wide range of classification problems with significant success, and this paper proposes eXtreme gradient boosting (XGBoost) model to execute the defect prediction task. A modified variant of the well-known reptile search optimization algorithm has been suggested to carry out the calibrating of the XGBoost hyperparameters. The enhanced algorithm was named HARSA and evaluated on the collection of challenging CEC2019 benchmark functions, where it exhibited excellent performance. Later, the introduced XGBoost model that uses the proposed algorithm has been evaluated on two benchmark software testing datasets, and the simulation outcomes have been compared to other powerful swarm intelligence metaheuristics that were used in the identical experimental environment, where the proposed approach attained superior classification accuracy on both datasets. Finally, Shapley Additive Explanations analysis was conducted to discover the impact of various software metrics on the classification results.

Diversity based multi-cluster over sampling approach to alleviate the class imbalance problem in software defect prediction

Diversity based multi-cluster over sampling approach to alleviate the class imbalance problem in software defect prediction

A Software Defect Prediction Approach Based on Hybrid Feature Dimensionality Reduction

Software defect prediction (SDP) is designed to assist software testing, which can reasonably allocate test resources to reduce costs and improve development efficiency. In order to improve the prediction performance, researchers have designed many defect-related features for SDP. However, feature redundancy (FR) and irrelevance caused by the increasing dimensions of data will greatly degrade the performance of defect prediction. In order to solve the problems, researchers have proposed various data dimensionality reduction methods. These methods can be simply divided into two categories of methods: feature selection and feature extraction. However, the two categories of methods have their own advantages and limitation. In this paper, we propose a Hybrid Feature Dimensionality Reduction Approach (HFDRA) for SDP, which combines the two different kinds of methods, to improve the performance of SDP. HFDRA approach can be divided into two stages: feature selection and feature extraction. First, HFDRA divides the original features into several feature subsets through a clustering algorithm in the feature selection stage. Then, in the feature extraction stage, kernel principal component analysis (KPCA) is used to reduce the dimensionality of each feature subset. Finally, the reduced-dimensional data is used to build the prediction model. In the empirical study, we use 22 projects from AEEEM, SOFTLAB, MORP, and ReLink as experiment object. In this paper, we first compare our approach with seven baseline methods and three state-of-the-art methods. Then, we analyze the relationship between FR and prediction performance. Experiment results show that our approach outperforms the state-of-the-art data dimensionality reduction methods for defect prediction.

Study on Software Defect Prediction Based on SVM and Decision Tree Algorithm

Software Defect Prediction is a process of identifying the potential defects in software systems before they occur. In this approach, the dataset containing information about software attributes is used as input, and the output is the prediction of whether the software is defective or not. The input dataset is generally in the form of a CSV file, which contains various software attributes such as cyclomatic complexity, essential complexity, Design Complexity, etc. The output of the defect prediction is binary classification. It is done by using SVM (Support Vector Machine) and a decision tree algorithm. This approach can help software developers identify their systems' defects before they cause any harm or affect the system’s performance.

Hybrid Defect Prediction Model Based on Counterfactual Feature Optimization

Software defect prediction is critical to ensuring software quality. Researchers have worked on building various defect prediction models to improve the performance of defect prediction. Existing defect prediction models are mainly divided into two categories: models constructed based on artificial statistical features and models constructed based on semantic features. DP-CNN [Li J, He P, Zhu J, et al. Software defect prediction via convolutional neural network. In: 2017 IEEE international conference on software quality, reliability and security (QRS). IEEE, 2017; 318–328.] is one of the best defect prediction models, because it combines both artificial statistical features and semantic features, so its performance is greatly improved compared to traditional defect prediction models. This paper is based on the DP-CNN model and makes the following two improvements: first, using a new Struc2vec network representation technique to mine existing information between software modules, which specializes in learning node representations from structural identity and can further extract structural features associated with defects. Let the DP-CNN model once again incorporate the newly mined structural features. Then, this paper proposes a feature selection method based on counterfactual explanations, which can determine the importance score of each feature by the feature change rate of counterfactual samples. The origin of these feature importance scores is interpretable. Under the guidance of these interpretable feature importance scores, better feature subsets can be obtained and used to optimize artificial statistical features within the DP-CNN model. Based on the above methods, this paper proposes a new hybrid defect prediction model DPS-CNN-STR. Evaluating our model on six open source projects in terms of F1 score in defect prediction. Experimental results show that DPS-CNN-STR improves the state-of-the-art method by an average of 3.3%.

Industrial applications of software defect prediction using machine learning: A business-driven systematic literature review

Industrial applications of software defect prediction using machine learning: A business-driven systematic literature review

Software Defect Prediction Based on Optimized Machine Learning Models: A Comparative Study

Software defect prediction is crucial used for detecting possible defects in software before they manifest. While machine learning models have become more prevalent in software defect prediction, their effectiveness may vary based on the dataset and hyperparameters of the model. Difficulties arise in determining the most suitable hyperparameters for the model, as well as identifying the prominent features that serve as input to the classifier. This research aims to evaluate various traditional machine learning models that are optimized for software defect prediction on NASA MDP (Metrics Data Program) datasets. The datasets were classified using k-nearest neighbors (k-NN), decision trees, logistic regression, linear discriminant analysis (LDA), single hidden layer multilayer perceptron (SHL-MLP), and Support Vector Machine (SVM). The hyperparameters of the models were fine-tuned using random search, and the feature dimensionality was decreased by utilizing principal component analysis (PCA). The synthetic minority oversampling technique (SMOTE) was implemented to oversample the minority class in order to correct the class imbalance. k-NN was found to be the most suitable for software defect prediction on several datasets, while SHL-MLP and SVM were also effective on certain datasets. It is noteworthy that logistic regression and LDA did not perform as well as the other models. Moreover, the optimized models outperform the baseline models in terms of classification accuracy. The choice of model for software defect prediction should be based on the specific characteristics of the dataset. Furthermore, hyperparameter tuning can improve the accuracy of machine learning models in predicting software defects.

A Study of Software Defect Prediction Using Artificial Intelligent and Business Risk Management Approach

Abstract: The usage of several artificial intelligence algorithms in the area of software defect prediction is reviewed in the literature in this paper, it review the use of various techniques for artificial intelligence in the field of software defect prediction. Software system flaws remain a significant issue, and modern software development now faces many difficulties. Identification, estimation, and evaluation of the risks are crucial steps in order to avoid, minimize, and monitor the risks and their effects.This literature review is quite helpful because it improves understanding of the studied area. A software fault is a flaw in an executable product that results in software failure. Saving money, time, and effort may be accomplished by identifying general software process areas that may require attention from the start of a project and understanding the potential causes of problems. Planning, managing, and carrying out software development activities could be aided by the ability for early fault estimation. In order to study how many prediction approaches have been developed over this time to address the problems linked to software defect, this research conducts a literature review of works from the previous many years.In the field of software development, the study also discusses software risk management. This study makes a significant contribution by proposing a method for analyzing, organizing, and utilizing information technology initiatives in the context of software defect prediction and risk management.

Learning based Software Defect Prediction

Learning based Software Defect Prediction

Pre-trained Model-based Software Defect Prediction for Edge-cloud Systems

Edge-cloud computing is a distributed computing infrastructure that brings computation and data storage with low latency closer to clients. As interest in edge-cloud systems grows, research on testing the systems has also been actively studied. However, as with traditional systems, the amount of resources for testing is always limited. Thus, we suggest a function-level just-in-time (JIT) software defect prediction (SDP) model based on a pre-trained model to address the limitation by prioritizing the limited testing resources for the defect-prone functions. The pre-trained model is a transformer-based deep learning model trained on a large corpus of code snippets, and the fine-tuned pre-trained model can provide the defect proneness for the changed functions at a commit level. We evaluate the performance of the three popular pre-trained models (i.e., CodeBERT, GraphCodeBERT, UniXCoder) on edge-cloud systems in within-project and cross-project environments. To the best of our knowledge, it is the first attempt to analyse the performance of the three pre-trained model-based SDP models for edge-cloud systems. As a result, we can confirm that UniXCoder showed the best performance among the three in the WPDP environment. However, we also confirm that additional research is necessary to apply the SDP models to the CPDP environment.

Implicit and explicit mixture of experts models for software defect prediction

Implicit and explicit mixture of experts models for software defect prediction

Implement Classification Approach for Software Defect Prediction

Implement Classification Approach for Software Defect Prediction

A Hybrid Software Defects Prediction Model for Imbalance Datasets Using Machine Learning Techniques: (S-SVM Model)

Software defect prediction (SDP) is an essential task for developing quality software, and various models have been developed for this purpose. However, the imbalanced nature of software defect datasets has challenged these models, resulting in decreased performance. To address this challenge, the author has proposed a hybrid machine learning model that combines Synthetic Minority Oversampling Technique (SMOTE) with Support Vector Machine (SVM)—SMOTE-SVM (S-SVM) model. The author has empirically examined SDP using multiple datasets (CM1, PC1, JM1, PC3, KC1, EQ and JDT) from the PROMISE and AEEEM repositories. The experimental study indicates that the S-SVM model involved training and compared with previously developed balanced and imbalanced test datasets using four evaluation metrics: Precision, Recall, F1 score, and Accuracy. For the balanced dataset, the S-SVM model achieved precision values ranging from 70 to 96, recall values ranging from 52 to 94, F1-score values ranging from 67 to 90, and accuracy values ranging from 69 to 98. For the imbalanced dataset, the S-SVM model achieved precision values ranging from 60 to 93, recall values ranging from 64 to 97, F1-score values ranging from 69 to 91, and accuracy values ranging from 67 to 87. The proposed S-SVM model outperforms other models’ ability to classify and predict software defects. Therefore, the hybridisation of SMOTE and SVM improved the model’s ability to categories and predict balanced and imbalanced datasets when sufficient defective and non-defective data is provided.

Just‐in‐time defect prediction enhanced by the joint method of line label fusion and file filtering

AbstractJust‐In‐Time (JIT) defect prediction aims to predict the defect proneness of software changes when they are initially submitted. It has become a hot topic in software defect prediction due to its timely manner and traceability. Researchers have proposed many JIT defect prediction approaches. However, these approaches cannot effectively utilise line labels representing added or removed lines and ignore the noise caused by defect‐irrelevant files. Therefore, a JIT defect prediction model enhanced by the joint method of line label Fusion and file Filtering (JIT‐FF) is proposed. Firstly, to distinguish added and removed lines while preserving the original software changes information, the authors represent the code changes as original, added, and removed codes according to line labels. Secondly, to obtain semantics‐enhanced code representation, a cross‐attention‐based line label fusion method to perform complementary feature enhancement is proposed. Thirdly, to generate code changes containing fewer defect‐irrelevant files, the authors formalise the file filtering as a sequential decision problem and propose a reinforcement learning‐based file filtering method. Finally, based on generated code changes, CodeBERT‐based commit representation and multi‐layer perceptron‐based defect prediction are performed to identify the defective software changes. The experiments demonstrate that JIT‐FF can predict defective software changes more effectively.

Software Defect Prediction Using Dagging Meta-Learner-Based Classifiers

To guarantee that software does not fail, software quality assurance (SQA) teams play a critical part in the software development procedure. As a result, prioritizing SQA activities is a crucial stage in SQA. Software defect prediction (SDP) is a procedure for recognizing high-risk software components and determining the influence of software measurements on the likelihood of software modules failure. There is a continuous need for sophisticated and better SDP models. Therefore, this study proposed the use of dagging-based and baseline classifiers to predict software defects. The efficacy of the dagging-based SDP model for forecasting software defects was examined in this study. The models employed were naïve Bayes (NB), decision tree (DT), and k-nearest neighbor (kNN), and these models were used on nine NASA datasets. Findings from the experimental results indicated the superiority of SDP models based on dagging meta-learner. Dagging-based models significantly outperformed experimented baseline classifiers built on accuracy, the area under the curve (AUC), F-measure, and precision-recall curve (PRC) values. Specifically, dagging-based NB, DT, and kNN models had +6.62%, +3.26%, and +4.14% increments in average accuracy value over baseline NB, DT, and kNN models. Therefore, it can be concluded that the dagging meta-learner can advance the recognition performances of SDP methods and should be considered for SDP processes.

Class Imbalance Reduction and Centroid based Relevant Project Selection for Cross Project Defect Prediction

Cross-Project Defect Prediction (CPDP) is the process of predicting defects in a target project using information from other projects. This can assist developers in prioritizing their testing efforts and finding flaws. Transfer Learning (TL) has been frequently used at CPDP to improve prediction performance by reducing the disparity in data distribution between the source and target projects. Software Defect Prediction (SDP) is a common study topic in software engineering that plays a critical role in software quality assurance. To address the cross-project class imbalance problem, Centroid-based PF-SMOTE for Imbalanced data is used. In this paper, we used a Centroid-based PF-SMOTE to balance the datasets and Centroid based relevant data selection for Cross Project Defect Prediction. These methods use the mean of all attributes in a dataset and calculating the difference between mean of all datasets. For experimentation, the open source software defect datasets namely, AEEM, Re-Link, and NASA, are considered.

Just-in-time defect prediction for mobile applications: using shallow or deep learning?

Just-in-time defect prediction (JITDP) research is increasingly focused on program changes instead of complete program modules within the context of continuous integration and continuous testing paradigm. Traditional machine learning-based defect prediction models have been built since the early 2000s, and recently, deep learning-based models have been designed and implemented. While deep learning (DL) algorithms can provide state-of-the-art performance in many application domains, they should be carefully selected and designed for a software engineering problem. In this research, we evaluate the performance of traditional machine learning algorithms and data sampling techniques for JITDP problems and compare the model performance with the performance of a DL-based prediction model. Experimental results demonstrated that DL algorithms leveraging sampling methods perform significantly worse than the decision tree-based ensemble method. The XGBoost-based model appears to be 116 times faster than the multilayer perceptron-based (MLP) prediction model. This study indicates that DL-based models are not always the optimal solution for software defect prediction, and thus, shallow, traditional machine learning can be preferred because of better performance in terms of accuracy and time parameters.