Cross-project Defect Prediction Research Articles

A novel defect prediction method based on semantic feature enhancement

SummaryAlthough cross‐project defect prediction (CPDP) techniques that use traditional manual features to build defect prediction model have been well‐developed, they usually ignore the semantic and structural information inside the program and fail to capture the hidden features that are critical for program category prediction, resulting in poor defect prediction results. Researchers have proposed using deep learning to automatically extract the semantic features of programs and fuse them with traditional features as training data. However, in practice, it is important to explore the effective representation of the semantic features in the programs and how the fusion of a reasonable ratio between the two types of features can maximize the effectiveness of the model. In this paper, we propose a semantic feature enhancement‐based defect prediction framework (SFE‐DP), which augments the semantic feature set extracted from the program code with data. We also introduce a layer of self‐attentive mechanism and a matching layer to filter low‐efficiency and non‐critical semantic features in the model structure. Finally, we combine the idea of hybrid loss function to iteratively optimize the model parameters. Extensive experiments validate that SFE‐DP can outperform the baseline approaches on 90 pairs of CPDP tasks formed by 10 open‐source projects.

Improving transfer learning for software cross-project defect prediction

AbstractSoftware cross-project defect prediction (CPDP) makes use of cross-project (CP) data to overcome the lack of data necessary to train well-performing software defect prediction (SDP) classifiers in the early stage of new software projects. Since the CP data (known as the source) may be different from the new project’s data (known as the target), this makes it difficult for CPDP classifiers to perform well. In particular, it is a mismatch of data distributions between source and target that creates this difficulty. Transfer learning-based CPDP classifiers are designed to minimize these distribution differences. The first Transfer learning-based CPDP classifiers treated these differences equally, thereby degrading prediction performance. To this end, recent research has the Weighted Balanced Distribution Adaptation (W-BDA) method to leverage the importance of both distribution differences to improve classification performance. Although W-BDA has been shown to improve model performance in CPDP and tackle the class imbalance by balancing the class proportion of each domain, research to date has failed to consider model performance in light of increasing target data. We provide the first investigation studying the effects of increasing the target data when leveraging the importance of both distribution differences. We extend the initial W-BDA method and call this extension the W-BDA$$\mathbf {^{+}}$$ + method. To evaluate the effectiveness of W-BDA$$\mathbf {^{+}}$$ + for improving CPDP performance, we conduct eight experiments on 18 projects from four datasets, where data sampling was performed with different sampling methods. Data sampling was only performed on the baseline methods and not on our proposed W-BDA$$\mathbf {^{+}}$$ + and the original W-BDA because data sampling issues do not exist for these two methods. We evaluate our method using four complementary indicators (i.e., Balanced Accuracy, AUC, F-measure and G-Measure). Our findings reveal an average improvement of 6%, 7.5%, 10% and 12% for these four indicators when W-BDA$$\mathbf {^{+}}$$ + is compared to the original W-BDA and five other baseline methods (for all four of the sampling methods used). Also, as the target to source ratio is increased with different sampling methods, we observe a decrease in performance for the original W-BDA, with our W-BDA$$\mathbf {^{+}}$$ + approach outperforming the original W-BDA in most cases. Our results highlight the importance of having an awareness of the effect of the increasing availability of target data in CPDP scenarios when using a method that can handle the class imbalance problem.

Cross-Project Defect Prediction Based on Domain Adaptation and LSTM Optimization

Cross-project defect prediction (CPDP) aims to predict software defects in a target project domain by leveraging information from different source project domains, allowing testers to identify defective modules quickly. However, CPDP models often underperform due to different data distributions between source and target domains, class imbalances, and the presence of noisy and irrelevant instances in both source and target projects. Additionally, standard features often fail to capture sufficient semantic and contextual information from the source project, leading to poor prediction performance in the target project. To address these challenges, this research proposes Smote Correlation and Attention Gated recurrent unit based Long Short-Term Memory optimization (SCAG-LSTM), which first employs a novel hybrid technique that extends the synthetic minority over-sampling technique (SMOTE) with edited nearest neighbors (ENN) to rebalance class distributions and mitigate the issues caused by noisy and irrelevant instances in both source and target domains. Furthermore, correlation-based feature selection (CFS) with best-first search (BFS) is utilized to identify and select the most important features, aiming to reduce the differences in data distribution among projects. Additionally, SCAG-LSTM integrates bidirectional gated recurrent unit (Bi-GRU) and bidirectional long short-term memory (Bi-LSTM) networks to enhance the effectiveness of the long short-term memory (LSTM) model. These components efficiently capture semantic and contextual information as well as dependencies within the data, leading to more accurate predictions. Moreover, an attention mechanism is incorporated into the model to focus on key features, further improving prediction performance. Experiments are conducted on apache_lucene, equinox, eclipse_jdt_core, eclipse_pde_ui, and mylyn (AEEEM) and predictor models in software engineering (PROMISE) datasets and compared with active learning-based method (ALTRA), multi-source-based cross-project defect prediction method (MSCPDP), the two-phase feature importance amplification method (TFIA) on AEEEM and the two-phase transfer learning method (TPTL), domain adaptive kernel twin support vector machines method (DA-KTSVMO), and generative adversarial long-short term memory neural networks method (GB-CPDP) on PROMISE datasets. The results demonstrate that the proposed SCAG-LSTM model enhances the baseline models by 33.03%, 29.15% and 1.48% in terms of F1-measure and by 16.32%, 34.41% and 3.59% in terms of Area Under the Curve (AUC) on the AEEEM dataset, while on the PROMISE dataset it enhances the baseline models’ F1-measure by 42.60%, 32.00% and 25.10% and AUC by 34.90%, 27.80% and 12.96%. These findings suggest that the proposed model exhibits strong predictive performance.

Cross-Project Defect Prediction Using Transfer Learning with Long Short-Term Memory Networks

With the increasing number of software projects, within-project defect prediction (WPDP) has already been unable to meet the demand, and cross-project defect prediction (CPDP) is playing an increasingly significant role in the area of software engineering. The classic CPDP methods mainly concentrated on applying metric features to predict defects. However, these approaches failed to consider the rich semantic information, which usually contains the relationship between software defects and context. Since traditional methods are unable to exploit this characteristic, their performance is often unsatisfactory. In this paper, a transfer long short-term memory (TLSTM) network model is first proposed. Transfer semantic features are extracted by adding a transfer learning algorithm to the long short-term memory (LSTM) network. Then, the traditional metric features and semantic features are combined for CPDP. First, the abstract syntax trees (AST) are generated based on the source codes. Second, the AST node contents are converted into integer vectors as inputs to the TLSTM model. Then, the semantic features of the program can be extracted by TLSTM. On the other hand, transferable metric features are extracted by transfer component analysis (TCA). Finally, the semantic features and metric features are combined and input into the logical regression (LR) classifier for training. The presented TLSTM model performs better on the f-measure indicator than other machine and deep learning models, according to the outcomes of several open-source projects of the PROMISE repository. The TLSTM model built with a single feature achieves 0.7% and 2.1% improvement on Log4j-1.2 and Xalan-2.7, respectively. When using combined features to train the prediction model, we call this model a transfer long short-term memory for defect prediction (DPTLSTM). DPTLSTM achieves a 2.9% and 5% improvement on Synapse-1.2 and Xerces-1.4.4, respectively. Both prove the superiority of the proposed model on the CPDP task. This is because LSTM capture long-term dependencies in sequence data and extract features that contain source code structure and context information. It can be concluded that: (1) the TLSTM model has the advantage of preserving information, which can better retain the semantic features related to software defects; (2) compared with the CPDP model trained with traditional metric features, the performance of the model can validly enhance by combining semantic features and metric features.

Interpretable Software Defect Prediction from Project Effort and Static Code Metrics

Software defect prediction models enable test managers to predict defect-prone modules and assist with delivering quality products. A test manager would be willing to identify the attributes that can influence defect prediction and should be able to trust the model outcomes. The objective of this research is to create software defect prediction models with a focus on interpretability. Additionally, it aims to investigate the impact of size, complexity, and other source code metrics on the prediction of software defects. This research also assesses the reliability of cross-project defect prediction. Well-known machine learning techniques, such as support vector machines, k-nearest neighbors, random forest classifiers, and artificial neural networks, were applied to publicly available PROMISE datasets. The interpretability of this approach was demonstrated by SHapley Additive exPlanations (SHAP) and local interpretable model-agnostic explanations (LIME) techniques. The developed interpretable software defect prediction models showed reliability on independent and cross-project data. Finally, the results demonstrate that static code metrics can contribute to the defect prediction models, and the inclusion of explainability assists in establishing trust in the developed models.

Landmark-Based Domain Adaptation and Selective Pseudo-Labeling for Heterogeneous Defect Prediction

Cross -project defect prediction (CPDP) is a promising technical means to solve the problem of insufficient training data in software defect prediction. As a special case of CPDP, heterogeneous defect prediction (HDP) has received increasing attention in recent years due to its ability to cope with different metric sets in projects. Existing studies have proven that using mixed-project data is a potential way to improve HDP performance, but there remain several challenges, including the negative impact of noise modules and the insufficient utilization of unlabeled modules. To this end, we propose a landmark-based domain adaptation and selective pseudo-labeling (LDASP) approach for mixed-project HDP. Specifically, we propose a novel landmark-based domain adaptation algorithm considering marginal and conditional distribution alignment and a class-wise locality structure to reduce the heterogeneity between both projects while reweighting modules to alleviate the negative impact brought by noise ones. Moreover, we design a progressive pseudo-label selection strategy exploring the underlying discriminative information of unlabeled target data to further improve the prediction effect. Extensive experiments are conducted based on 530 heterogeneous prediction combinations that are built from 27 projects using four datasets. The experimental results show that (1) our approach improves the F1-score and AUC over the baselines by 9.8–20.2% and 4.8–14.4%, respectively and (2) each component of LDASP (i.e., the landmark weights and selective pseudo-labeling strategy) can promote the HDP performance effectively.

Heterogeneous Cross-Project Defect Prediction Using Encoder Networks and Transfer Learning

Heterogeneous Cross-Project Defect Prediction Using Encoder Networks and Transfer Learning

능동 학습 기반 교차 프로젝트 결함 예측 시스템에 대한 개선 연구

능동 학습 기반 교차 프로젝트 결함 예측 시스템에 대한 개선 연구

Adversarial domain adaptation for cross-project defect prediction

Adversarial domain adaptation for cross-project defect prediction

Empirical validation of feature selection techniques for cross-project defect prediction

Empirical validation of feature selection techniques for cross-project defect prediction

SMOTE-Based Homogeneous Prediction for Aging-Related Bugs in Cloud-Oriented Software

Software aging is the process caused by Aging-Related Bugs (ARBs) which leads to the depletion of resources and degradation of performance in the long run. ARBs are difficult to find and replicate in future studies as they are less in number, thus prediction of ARB is necessary to save cost and time in the testing phase. ARBs are present in low proportion as compared to non-ARBs known as the class Imbalance problem resulting in insufficient training dataset for prediction models. In this study, Synthetic Minority Oversampling Technique (SMOTE) is applied along with homogeneous cross-project ARB prediction to reduce the effect of imbalance problem in software. SMOTE is oversampling of the minority instances synthetically to balance the dataset and improve the capability of defect prediction models. Homogeneous cross-project prediction is implemented where the datasets are different but the distribution of metric sets of both training and testing datasets is similar. The experiment is conducted on five cloud-oriented software such as Cassandra, Hive, Storm, Hadoop HDFS and Hadoop Mapreduce. The novelty of this study is the combination of SMOTE and homogeneous cross-project defect prediction for ARBs in cloud-oriented software. The comparative analysis is also conducted to understand the difference between SMOTE and non-SMOTE results with the help of machine learning classifiers. The result conveys that SMOTE is an efficient method to address class imbalance problem in ARB prediction.

Class Imbalance Reduction and Centroid based Relevant Project Selection for Cross Project Defect Prediction

Cross-Project Defect Prediction (CPDP) is the process of predicting defects in a target project using information from other projects. This can assist developers in prioritizing their testing efforts and finding flaws. Transfer Learning (TL) has been frequently used at CPDP to improve prediction performance by reducing the disparity in data distribution between the source and target projects. Software Defect Prediction (SDP) is a common study topic in software engineering that plays a critical role in software quality assurance. To address the cross-project class imbalance problem, Centroid-based PF-SMOTE for Imbalanced data is used. In this paper, we used a Centroid-based PF-SMOTE to balance the datasets and Centroid based relevant data selection for Cross Project Defect Prediction. These methods use the mean of all attributes in a dataset and calculating the difference between mean of all datasets. For experimentation, the open source software defect datasets namely, AEEM, Re-Link, and NASA, are considered.

Efficient Random Forest Algorithm for Multi-objective Optimization in Software Defect Prediction

The Software Defect Prediction (SDP) process provides reliable software by identifying defect-prone modules before the testing stage. It efficiently and effectively utilizes quality assurance resources. Most predictive models are trained on historical data which belong to the same project or comparable project. These models show satisfactory performance as they utilize similar settings to the considered projects. But the limitation of these models is that they are effective only if there are adequate historical data to train a predictive model. In reality, however, such historical data are minimal for some projects and absent for new projects. The defect prediction in such projects which lack historical data can be accomplished by training prediction models on different project data. This process is known as Cross-Project Defect Prediction (CPDP). Software defect datasets also suffered from class imbalance issues which further degrades the model’s performance. In this research work, the authors have proposed a Multi-Objective Random Forest (MO-RF) algorithm with a data resampling technique to minimize the probability of false alarms, to maximize the probability of detection and to overcome the class imbalance problem. The study also evaluates the performance of other prediction models. The proposed method has shown percentage improvement (in terms of AUC) of 2.78 and 3.46 over MONB and MONBNN, respectively.

Multi-Source Heterogeneous Kernel Mapping in Software Defect Prediction

Heterogeneous defect prediction (HDP) is a significant research topic in cross-project defect prediction (CPDP), due to the inconsistency of metrics used between source and target projects. While most HDP methods aim to improve the performance of models trained on data from one source project, few studies have investigated how the number of source projects affects predictive performance. In this paper, we propose a new multi-source heterogeneous kernel mapping (MSHKM) algorithm to analyze the effects of different numbers of source projects on prediction results. First, we introduce two strategies based on MSHKM for multi-source HDP. To determine the impact of the number of source projects on the predictive performance of the model, we regularly vary the number of source projects in each strategy. Then, we compare the proposed MSHKM with state-of-the-art HDP methods and within-project defect prediction (WPDP) methods, in terms of three common performance measures, using 28 data sets from five widely used projects. Our results demonstrate that, (1) in the multi-source HDP scenario, strategy 2 outperforms strategy 1; (2) for MSHKM, a lower number of source projects leads to better results and performance under strategy 1, while n = 4 is the optimal number under strategy 2; (3) MSHKM performs better than related state-of-the-art HDP methods; and (4) MSHKM outperforms WPDP. In summary, our proposed MSHKM algorithm provides a promising solution for heterogeneous cross-project defect prediction, and our findings suggest that the number of source projects should be carefully selected to achieve optimal predictive performance.

ARRAY: Adaptive triple feature-weighted transfer Naive Bayes for cross-project defect prediction

Context:Cross-project defect prediction (CPDP) aims to predict defects of target data by using prediction models trained on the source dataset. However, owing to the huge distribution difference, it is still a challenge to build high-performance CPDP models. Objective:We propose a novel high-performance CPDP method named adaptive triple feature-weighted transfer naive Bayes (ARRAY). Methods:ARRAY is characterized by feature weighted similarity, feature weighted instance weight, and the model adaptive adjustment. Experiments are performed on 34 defect datasets. We compare ARRAY with seven state-of-the-art CPDP methods in terms of area under ROC curve (AUC), F1, and Matthews correlation coefficient (MCC) with statistical testing methods. Results:Experimental results show that: (1) on average, ARRAY separately improves MCC, AUC, and F1 over the baselines by at least 18.4%, 6.5%, and 4.5%; (2) ARRAY significantly performs better than each baseline on most datasets; (3) ARRAY significantly outperforms all baselines with non-negligible effect size according to post-hoc test. Conclusion:It can be concluded that: (1) the proposed feature weighted similarity, feature weighted instance weight, and the model adaptive adjustment are very helpful for improving the performance of CPDP models; (2) ARRAY is a more promising alternative for CPDP with common metrics.

Cross-project clone consistent-defect prediction via transfer-learning method

Code clones are comparable code snippets that are introduced into software by developers in order to increase software development productivity. A change to code clone may result in a consistent-defect if the developers forget to verify the consistency of the code after the change. To reduce such change-related maintenance costs, researchers have proposed a number of methods for predicting clone consistency in advance. Unfortunately, the effectiveness of these cross-project models is unsatisfactory, and performing such predictions with insufficient data remains a challenge. Meanwhile, cross-project defect prediction via transfer learning method is prevalent in the software engineering community. Consequently, we first construct an empirical study to explore whether transfer-learning techniques could well be utilized for clone cross-project consistent-defect prediction in the initial stages of software development. In this paper, we employ transfer-learning techniques to predict clone consistency at both the time of clone creating and clone changing in order to avoid clone consistent-defects and maintenance. We conduct an experiment on open-source projects to evaluate the effectiveness of various transfer-learning methods. Our investigation demonstrates that transfer-learning techniques have a beneficial impact on predicting cross-project clone consistent-defect, and that the size of the dataset also has a positive effect on prediction. In order to promote software safety and security, we recommend that developers leverage transfer-learning to enhance the capability for clone cross-project consistent-defect prediction early in the software development phase.

Candidate project selection in cross project defect prediction using hybrid method

Cross Project Defect Prediction (CPDP) is a process that develops a defect prediction model on source projects and then applies the same model to the target project. Day by day, new software projects are being developed, so selecting an appropriate training project from existing projects and from new upcoming projects, to train a predictor model is a challenging task in CPDP. In the present study, we have proposed a hybrid selection method to select a candidate project from existing projects and a probabilistic method to select a candidate project from new projects. The proposed hybrid method is a weighted combination of the Collaborative filtering (CF) method and the Content Based (CB) method. The probabilistic method is based on a Naïve Bayes classifier and is used to predict the relation between the target project and the new target project. In the CF method, a usability score is generated for each project by making use of classification techniques, and the CB method calculates the matching score of candidate projects by using the K-dimensional tree. Finally, both the methods are combined by parallelized hybridization design, and weights for the proposed method are estimated with an empirical bootstrapping method. The score generated by the proposed hybrid technique is then used to identify the most suitable candidate project for the new project. The experimental results show that the suggestion list of the best three candidate projects is consistent when employing different classifiers. The recommendation performance is evaluated in terms of F-score and Mean Average Precision (MAP), and the proposed method has shown improved performance as compared to the existing methods in both terms.

Optimal Feature Selection through Search-Based Optimizer in Cross Project

Cross project defect prediction (CPDP) is a key method for estimating defect-prone modules of software products. CPDP is a tempting approach since it provides information about predicted defects for those projects in which data are insufficient. Recent studies specifically include instructions on how to pick training data from large datasets using feature selection (FS) process which contributes the most in the end results. The classifier helps classify the picked-up dataset in specified classes in order to predict the defective and non-defective classes. The aim of our research is to select the optimal set of features from multi-class data through a search-based optimizer for CPDP. We used the explanatory research type and quantitative approach for our experimentation. We have F1 measure as our dependent variable while as independent variables we have KNN filter, ANN filter, random forest ensemble (RFE) model, genetic algorithm (GA), and classifiers as manipulative independent variables. Our experiment follows 1 factor 1 treatment (1F1T) for RQ1 whereas for RQ2, RQ3, and RQ4, there are 1 factor 2 treatments (1F2T) design. We first carried out the explanatory data analysis (EDA) to know the nature of our dataset. Then we pre-processed our data by removing and solving the issues identified. During data preprocessing, we analyze that we have multi-class data; therefore, we first rank features and select multiple feature sets using the info gain algorithm to get maximum variation in features for multi-class dataset. To remove noise, we use ANN-filter and get significant results more than 40% to 60% compared to NN filter with base paper (all, ckloc, IG). Then we applied search-based optimizer i.e., random forest ensemble (RFE) to get the best features set for a software prediction model and we get 30% to 50% significant results compared with genetic instance selection (GIS). Then we used a classifier to predict defects for CPDP. We compare the results of the classifier with base paper classifier using F1-measure and we get almost 35% more than base paper. We validate the experiment using Wilcoxon and Cohen’s d test.

An effective feature selection based cross-project defect prediction model for software quality improvement

An effective feature selection based cross-project defect prediction model for software quality improvement

Addressing Noise and Class Imbalance Problems in Heterogeneous Cross-Project Defect Prediction

When a software project either lacks adequate historical data to build a defect prediction (DP) model or is in the initial phases of development, the DP model based on related source project's defect data might be used. This kind of SDP is categorized as heterogeneous cross-project defect prediction (HCPDP). According to a comprehensive literature review, no research has been done in the field of CPDP to deal with noise and class imbalance problem (CIP) at the same time. In this paper, the impact of noise and imbalanced data on the efficiency of the HCPDP and with-in project defect prediction (WPDP) model is examined empirically and conceptually using four different classification algorithms. In addition, CIP is handled using a novel technique known as chunk balancing algorithm (CBA). Ten prediction combinations from three open-source projects are used in the experimental investigation. The findings show that noise in an imbalanced dataset has a significant impact on defect prediction accuracy.