Software Defect Prediction Model Research Articles

Enhancing software defect prediction models using metaheuristics with a learning to rank approach

Enhancing software defect prediction models using metaheuristics with a learning to rank approach

Improving Software Defect Prediction With a Combination of Feature Selection Based On Ant Colony Optimization and Ensemble Technique

Software defect prediction plays a vital role in enhancing software quality and minimizing maintenance costs. This study aims to improve software defect prediction by employing a combination of Ant Colony Optimization (ACO) for feature selection and ensemble techniques, particularly Gradient Boosting. The research utilizes three NASA MDP datasets: MC1, KC1, and PC2, to evaluate the performance of four machine learning algorithms: Random Forest, Support Vector Machine (SVM), Decision Tree, and Naïve Bayes. Data preprocessing involved handling class imbalances using the SMOTE technique and transforming categorical data into numerical representations. The results indicate that the integration of ACO and Gradient Boosting significantly enhances the accuracy of all four algorithms. Notably, the Random Forest algorithm achieved the highest accuracy of 99% on the MC1 dataset. The findings suggest that combining ACO-based feature selection with ensemble techniques can effectively boost the performance of software defect prediction models, offering a robust approach for early detection of potential softwaredefects and contributing to improved software reliability and efficiency.

WSO-KELM: War Strategy Optimization-Based Kernel Extreme Learning Machine for Automatic Software Fault Prediction Model

WSO-KELM: War Strategy Optimization-Based Kernel Extreme Learning Machine for Automatic Software Fault Prediction Model

Towards a framework for reliable performance evaluation in defect prediction

Enhancing software reliability, dependability, and security requires effective identification and mitigation of defects during early development stages. Software defect prediction (SDP) models have emerged as valuable tools for this purpose. However, there is currently a lack of consensus in evaluating the predictive performance of newly proposed models, which hinders accurate measurement of progress and can lead to misleading conclusions. To tackle this challenge, we present MATTER (a fraMework towArd a consisTenT pErformance compaRison), which aims to provide reliable and consistent performance comparisons for SDP models. MATTER incorporates three key considerations. First, it establishes a global reference point, ONE (glObal baseliNe modEl), which possesses the 3S properties (Simplicity in implementation, Strong predictive ability, and Stable prediction performance), to serve as the baseline for evaluating other models. Second, it proposes using the SQA-effort-aligned threshold setting to ensure fair performance comparisons. Third, it advocates for consistent performance evaluation by adopting a set of core performance indicators that reflect the practical value of prediction models in achieving tangible progress. Through the application of MATTER to the same benchmark data sets, researchers and practitioners can obtain more accurate and meaningful insights into the performance of defect prediction models, thereby facilitating informed decision-making and improving software quality. When evaluating representative SDP models from recent years using MATTER, we surprisingly observed that: none of these models demonstrated a notable enhancement in prediction performance compared to the simple baseline model ONE. In future studies, we strongly recommend the adoption of MATTER to assess the actual usefulness of newly proposed models, promoting reliable scientific progress in defect prediction.

A novel software defect prediction model using two-phase grey wolf optimisation for feature selection

A novel software defect prediction model using two-phase grey wolf optimisation for feature selection

Development of optimised software fault prediction model using machine learning

Software fault prediction is a crucial task, especially with the rapid improvements in software technology and increasing complexity of software. As identifying and addressing bugs early in the development process can significantly minimize the costs and enhance the software quality. Software fault prediction using machine learning algorithms has gained significant attention due to its potential to improve software quality and save time in the testing phase. This research paper investigates the impact of classification models on bug prediction performance and explores the use of bio-inspired optimization techniques to enhance model results. Through experiments, it is demonstrated that applying bio-inspired algorithms improves the accuracy of fault prediction models. The evaluation is based on multiple performance metrics and the results show that KNN with BACO (Binary Ant Colony Optimization) generally outperform the other models in terms of accuracy. The BACO-KNN fault prediction model attains the accuracy of 96.39% surpassing the previous work.

Optimizing Software Defect Prediction Models: Integrating Hybrid Grey Wolf and Particle Swarm Optimization for Enhanced Feature Selection with Popular Gradient Boosting Algorithm

Software defects, also referred to as software bugs, are anomalies or flaws in computer program that cause software to behave unexpectedly or produce incorrect results. These defects can manifest in various forms, including coding errors, design flaws, and logic mistakes, this defect have the potential to emerge at any stage of the software development lifecycle. Traditional prediction models usually have lower prediction performance. To address this issue, this paper proposes a novel prediction model using Hybrid Grey Wolf Optimizer and Particle Swarm Optimization (HGWOPSO). This research aims to determine whether the Hybrid Grey Wolf and Particle Swarm Optimization model could potentially improve the effectiveness of software defect prediction compared to base PSO and GWO algorithms without hybridization. Furthermore, this study aims to determine the effectiveness of different Gradient Boosting Algorithm classification algorithms when combined with HGWOPSO feature selection in predicting software defects. The study utilizes 13 NASA MDP dataset. These dataset are divided into testing and training data using 10-fold cross-validation. After data is divided, SMOTE technique is employed in training data. This technique generates synthetic samples to balance the dataset, ensuring better performance of the predictive model. Subsequently feature selection is conducted using HGWOPSO Algorithm. Each subset of the NASA MDP dataset will be processed by three boosting classification algorithms namely XGBoost, LightGBM, and CatBoost. Performance evaluation is based on the Area under the ROC Curve (AUC) value. Average AUC values yielded by HGWOPSO XGBoost, HGWOPSO LightGBM, and HGWOPSO CatBoost are 0.891, 0.881, and 0.894, respectively. Results of this study indicated that utilizing the HGWOPSO algorithm improved AUC performance compared to the base GWO and PSO algorithms. Specifically, HGWOPSO CatBoost achieved the highest AUC of 0.894. This represents a 6.5% increase in AUC with a significance value of 0.00552 compared to PSO CatBoost, and a 6.3% AUC increase with a significance value of 0.00148 compared to GWO CatBoost. This study demonstrated that HGWOPSO significantly improves the performance of software defect prediction. The implication of this research is to enhance software defect prediction models by incorporating hybrid optimization techniques and combining them with gradient boosting algorithms, which can potentially identify and address defects more accurately

Interpretable Software Defect Prediction from Project Effort and Static Code Metrics

Software defect prediction models enable test managers to predict defect-prone modules and assist with delivering quality products. A test manager would be willing to identify the attributes that can influence defect prediction and should be able to trust the model outcomes. The objective of this research is to create software defect prediction models with a focus on interpretability. Additionally, it aims to investigate the impact of size, complexity, and other source code metrics on the prediction of software defects. This research also assesses the reliability of cross-project defect prediction. Well-known machine learning techniques, such as support vector machines, k-nearest neighbors, random forest classifiers, and artificial neural networks, were applied to publicly available PROMISE datasets. The interpretability of this approach was demonstrated by SHapley Additive exPlanations (SHAP) and local interpretable model-agnostic explanations (LIME) techniques. The developed interpretable software defect prediction models showed reliability on independent and cross-project data. Finally, the results demonstrate that static code metrics can contribute to the defect prediction models, and the inclusion of explainability assists in establishing trust in the developed models.

Adjusted Trust Score: A Novel Approach for Estimating the Trustworthiness of Software Defect Prediction Models

Adjusted Trust Score: A Novel Approach for Estimating the Trustworthiness of Software Defect Prediction Models

Software Defect Prediction Using Non-Dominated Sorting Genetic Algorithm and $k$-Nearest Neighbour Classifier

Background: Software Defect Prediction (SDP) is a vital step in software development. SDP aims to identify the most likely defect-prone modules before starting the testing phase, and it helps assign resources and reduces the cost of testing. Aim: Although many machine learning algorithms have been used to classify software modules based on static code metrics, the k-Nearest Neighbors (kNN) method does not greatly improve defect prediction because it requires careful set-up of multiple configuration parameters before it can be used. To address this issue, we used the Non-dominated Sorting Genetic Algorithm (NSGA-II) to optimize the parameters in the kNN classifier with favor to improve SDP accuracy. We used NSGA-II because the existing accuracy metrics often behave differently, making an opposite judgment in evaluating SDP models. This means that changing one parameter might improve one accuracy measure while it decreases the others. Method: The proposed NSGAII-kNN model was evaluated against the classical kNN model and state-of-the-art machine learning algorithms such as Support Vector Machine (SVM), Naïve Bayes (NB), and Random Forest (RF) classifiers. Results: Results indicate that the GA-optimized kNN model yields a higher Matthews Coefficient Correlation and higher balanced accuracy based on ten datasets. Conclusion: The paper concludes that integrating GA with kNN improved defect prediction when applied to large or small or large datasets.

A trustworthy hybrid model for transparent software defect prediction: SPAM-XAI.

Maintaining quality in software development projects is becoming very difficult because the complexity of modules in the software is growing exponentially. Software defects are the primary concern, and software defect prediction (SDP) plays a crucial role in detecting faulty modules early and planning effective testing to reduce maintenance costs. However, SDP faces challenges like imbalanced data, high-dimensional features, model overfitting, and outliers. Moreover, traditional SDP models lack transparency and interpretability, which impacts stakeholder confidence in the Software Development Life Cycle (SDLC). We propose SPAM-XAI, a hybrid model integrating novel sampling, feature selection, and eXplainable-AI (XAI) algorithms to address these challenges. The SPAM-XAI model reduces features, optimizes the model, and reduces time and space complexity, enhancing its robustness. The SPAM-XAI model exhibited improved performance after experimenting with the NASA PROMISE repository's datasets. It achieved an accuracy of 98.13% on CM1, 96.00% on PC1, and 98.65% on PC2, surpassing previous state-of-the-art and baseline models with other evaluation matrices enhancement compared to existing methods. The SPAM-XAI model increases transparency and facilitates understanding of the interaction between features and error status, enabling coherent and comprehensible predictions. This enhancement optimizes the decision-making process and enhances the model's trustworthiness in the SDLC.

Boosting Software Fault Prediction: Addressing Class Imbalance With Enhanced Ensemble Learning

Software fault prediction (SFP) is a crucial aspect of software engineering, aiding in the early identification of potential defects. This proactive approach significantly contributes to enhancing software quality and reliability. However, a common challenge in SFP is class imbalance (CI). Ensemble learning (EL) is a powerful strategy for refining SFP models in object‐oriented systems with imbalanced data and improving sensitivity to minority classes. This study aimed to improve the effectiveness of ensemble classes in SFP within object‐oriented systems, tackling the challenges associated with imbalanced data. It focuses on enhancing the performance of three ensemble classifiers, BalancedBagging, RUSBoost, and EasyEnsemble, explicitly designed for imbalanced datasets. In Enhanced_BalancedBagging (E_BB) and ROSBoost, random undersampling (RUS) is substituted with random oversampling (ROS). Meanwhile, Enhanced_EasyEnsemble (E_EE) replaces RUS with ROS and AdaBoost with XGBoost. The experimental results demonstrate the superior performance of E_BB, ROSBoost, and E_EE over their base models, achieving the highest F‐measure, balanced accuracy, and AUC. Statistical tests, such as the Wilcoxon signed‐rank test, provide robust support for the enhanced models, highlighting their practical significance through substantial improvements in F‐measure and AUC, as indicated by low negative rank sums and large effect sizes.

HYBRID BINARY WHALE OPTIMIZATION ALGORITHM BASED ON TAPER SHAPED TRANSFER FUNCTION FOR SOFTWARE DEFECT PREDICTION

Reliability is one of the key factors used to gauge software quality. Software defect prediction (SDP) is one of the most important factors which affects measuring software's reliability. Additionally, the high dimensionality of the features has a direct effect on the accuracy of SDP models. The objective of this paper is to propose a hybrid binary whale optimization algorithm (BWOA) based on taper-shape transfer functions for solving feature selection problems and dimension reduction with a KNN classifier as a new software defect prediction method. In this paper, the values of a real vector that represents the individual encoding have been converted to binary vector by using the four types of Taper-shaped transfer functions to enhance the performance of BWOA to reduce the dimension of the search space. The performance of the suggested method (T-BWOA-KNN) was evaluated using eleven standard software defect prediction datasets from the PROMISE and NASA repositories depending on the K-Nearest Neighbor (KNN) classifier. Seven evaluation metrics have been used to assess the effectiveness of the suggested method. The experimental results have shown that the performance of T-BWOA-KNN produced promising results compared to other methods including ten methods from the literature, four types of T-BWOA with the KNN classifier. In addition, the obtained results are compared and analyzed with other methods from the literature in terms of the average number of selected features (SF) and accuracy rate (ACC) using the Kendall W test. In this paper, a new hybrid software defect prediction method called T-BWOA-KNN has been proposed which is concerned with the feature selection problem. The experimental results have proved that T-BWOA-KNN produced promising performance compared with other methods for most datasets.

Software Defect Prediction For Quality Evaluation Using Learning Techniques Ensemble Stacking

This research aims to improve the software quality and effectiveness of zakat management by the National Amil Zakat Agency (BAZNAS) through the development of a software defect prediction model (SDPM). We used machine learning techniques and ensemble stacking approach on the "Masjid Tower" dataset containing 228 records and 34 attributes. The preprocessing process involved label encoding, feature selection with Pearson correlation, standard normalization, and the use of SMOTE to handle data imbalance. We performed hyperparameter tuning with grid search CV on Machine Learning algorithms such as Ada Boost and Gradient Boosting. The results showed that the ensemble stacking approach with a combination of Gradient Boosting, Ada Boost, Decision Tree, Bayesian Ridge, and LightGBM meta learner algorithms provided high accuracy with R2 score reaching 0.97, MAE of 0.037, and MSE of 0.006. This finding proves that the ensemble stacking approach is able to overcome the problem of software defects with accurate prediction results, provide useful guidance in the management of zakat and other software applications, and has the potential to improve software quality and the effectiveness of BAZNAS in managing zakat.

MCDM-EFS: A novel ensemble feature selection method for software defect prediction using multi-criteria decision making

Software defect prediction models are used for predicting high risk software components. Feature selection has significant impact on the prediction performance of the software defect prediction models since redundant and unimportant features make the prediction model more difficult to learn. Ensemble feature selection has recently emerged as a new methodology for enhancing feature selection performance. This paper proposes a new multi-criteria-decision-making (MCDM) based ensemble feature selection (EFS) method. This new method is termed as MCDM-EFS. The proposed method, MCDM-EFS, first generates the decision matrix signifying the feature’s importance score with respect to various existing feature selection methods. Next, the decision matrix is used as the input to well-known MCDM method TOPSIS for assigning a final rank to each feature. The proposed approach is validated by an experimental study for predicting software defects using two classifiers K-nearest neighbor (KNN) and naïve bayes (NB) over five open-source datasets. The predictive performance of the proposed approach is compared with existing feature selection algorithms. Two evaluation metrics – nMCC and G-measure are used to compare predictive performance. The experimental results show that the MCDM-EFS significantly improves the predictive performance of software defect prediction models against other feature selection methods in terms of nMCC as well as G-measure.

Evaluating the Impact of Data Transformation Techniques on the Performance and Interpretability of Software Defect Prediction Models

The performance of software defect prediction (SDP) models determines the priority of test resource allocation. Researchers also use interpretability techniques to gain empirical knowledge about software quality from SDP models. However, SDP methods designed in the past research rarely consider the impact of data transformation methods, simple but commonly used preprocessing techniques, on the performance and interpretability of SDP models. Therefore, in this paper, we investigate the impact of three data transformation methods (Log, Minmax, and Z-score) on the performance and interpretability of SDP models. Through empirical research on (i) six classification techniques (random forest, decision tree, logistic regression, Naive Bayes, K-nearest neighbors, and multilayer perceptron), (ii) six performance evaluation indicators (Accuracy, Precision, Recall, F1, MCC, and AUC), (iii) two interpretable methods (permutation and SHAP), (iv) two feature importance measures (Top-k feature rank overlap and difference), and (v) three datasets (Promise, Relink, and AEEEM), our results show that the data transformation methods can significantly improve the performance of the SDP models and greatly affect the variation of the most important features. Specifically, the impact of data transformation methods on the performance and interpretability of SDP models depends on the classification techniques and evaluation indicators. We observe that log transformation improves NB model performance by 7%–61% on the other five indicators with a 5% drop in Precision. Minmax and Z-score transformation improves NB model performance by 2%–9% across all indicators. However, all three transformation methods lead to substantial changes in the Top-5 important feature ranks, with differences exceeding 2 in 40%–80% of cases (detailed results available in the main content). Based on our findings, we recommend that (1) considering the impact of data transformation methods on model performance and interpretability when designing SDP approaches as transformations can improve model accuracy, and potentially obscure important features, which lead to challenges in interpretation, (2) conducting comparative experiments with and without the transformations to validate the effectiveness of proposed methods which are designed to improve the prediction performance, and (3) tracking changes in the most important features before and after applying data transformation methods to ensure precise and traceable interpretability conclusions to gain insights. Our study reminds researchers and practitioners of the need for comprehensive considerations even when using other similar simple data processing methods.

An Integrated Semi-supervised Software Defect Prediction Model

<p>A novel semi-supervised software defect prediction model FFeSSTri (Filtered Feature Selecting, Sample and Tri-training) is proposed to address the problem that class imbalance and too many irrelevant or redundant features in labelled samples lower the accuracy of semi-supervised software defect prediction. Its innovation lies in that the construction of FFeSSTri integrates an oversampling technique, a new feature selection method, and a Tri-training algorithm, thus it can effectively improve the accuracy. Firstly, the oversampling technique is applied to expand the class of inadequate samples, thus it solves the unbalanced classification of the labelled samples. Secondly, a new filtered feature selection method based on relevance and redundancy is proposed, which can exclude those irrelevant or redundant features from labelled samples. Finally, the Tri-training algorithm is used to learn the labelled training samples to build the defect prediction model FFeSSTri. The experiments conducted on the NASA software defect prediction dataset show that FFeSSTri outperforms the existing four supervised learning methods and one semi-supervised learning method in terms of F-Measure values and AUC values.</p> <p> </p>

Transfer Learning based Low Shot Classifier for Software Defect Prediction

Background: The rapid growth and increasing complexity of software applications are causing challenges in maintaining software quality within constraints of time and resources. This challenge led to the emergence of a new field of study known as Software Defect Prediction (SDP), which focuses on predicting future defect in advance, thereby reducing costs and improving productivity in software industry. Objective: This study aimed to address data distribution disparities when applying transfer learning in multi-project scenarios, and to mitigate performance issues resulting from data scarcity in SDP. Methods: The proposed approach, namely Transfer Learning based Low Shot Classifier (TLLSC), combined transfer learning and low shot learning approaches to create an SDP model. This model was designed for application in both new projects and those with minimal historical defect data. Results: Experiments were conducted using standard datasets from projects within the National Aeronautics and Space Administration (NASA) and Software Research Laboratory (SOFTLAB) repository. TLLSC showed an average increase in F1-Measure of 31.22%, 27.66%, and 27.54% for project AR3, AR4, and AR5, respectively. These results surpassed those from Transfer Component Analysis (TCA+), Canonical Correlation Analysis (CCA+), and Kernel Canonical Correlation Analysis plus (KCCA+). Conclusion: The results of the comparison between TLLSC and state-of-the-art algorithms, namely TCA+, CCA+, and KCCA+ from the existing literature consistently showed that TLLSC performed better in terms of F1-Measure. Keywords: Just-in-time, Defect Prediction, Deep Learning, Transfer Learning, Low Shot Learning

Software defect prediction using a bidirectional LSTM network combined with oversampling techniques

AbstractSoftware defects are a critical issue in software development that can lead to system failures and cause significant financial losses. Predicting software defects is a vital aspect of ensuring software quality. This can significantly impact both saving time and reducing the overall cost of software testing. During the software defect prediction (SDP) process, automated tools attempt to predict defects in the source codes based on software metrics. Several SDP models have been proposed to identify and prevent defects before they occur. In recent years, recurrent neural network (RNN) techniques have gained attention for their ability to handle sequential data and learn complex patterns. Still, these techniques are not always suitable for predicting software defects due to the problem of imbalanced data. To deal with this problem, this study aims to combine a bidirectional long short-term memory (Bi-LSTM) network with oversampling techniques. To establish the effectiveness and efficiency of the proposed model, the experiments have been conducted on benchmark datasets obtained from the PROMISE repository. The experimental results have been compared and evaluated in terms of accuracy, precision, recall, f-measure, Matthew’s correlation coefficient (MCC), the area under the ROC curve (AUC), the area under the precision-recall curve (AUCPR) and mean square error (MSE). The average accuracy of the proposed model on the original and balanced datasets (using random oversampling and SMOTE) was 88%, 94%, And 92%, respectively. The results showed that the proposed Bi-LSTM on the balanced datasets (using random oversampling and SMOTE) improves the average accuracy by 6 and 4% compared to the original datasets. The average F-measure of the proposed model on the original and balanced datasets (using random oversampling and SMOTE) were 51%, 94%, And 92%, respectively. The results showed that the proposed Bi-LSTM on the balanced datasets (using random oversampling and SMOTE) improves the average F-measure by 43 and 41% compared to the original datasets. The experimental results demonstrated that combining the Bi-LSTM network with oversampling techniques positively affects defect prediction performance in datasets with imbalanced class distributions.

Improved Software Fault Prediction Model Based on Optimal Features Set and Threshold Values Using Metaheuristic Approach

Improved Software Fault Prediction Model Based on Optimal Features Set and Threshold Values Using Metaheuristic Approach