Fault-prone Software Modules Research Articles

Concept Drift in Software Defect Prediction: A Method for Detecting and Handling the Drift

Software Defect Prediction (SDP) is crucial towards software quality assurance in software engineering. SDP analyzes the software metrics data for timely prediction of defect prone software modules. Prediction process is automated by constructing defect prediction classification models using machine learning techniques. These models are trained using metrics data from historical projects of similar types. Based on the learned experience, models are used to predict defect prone modules in currently tested software. These models perform well if the concept is stationary in a dynamic software development environment. But their performance degrades unexpectedly in the presence of change in concept (Concept Drift). Therefore, concept drift (CD) detection is an important activity for improving the overall accuracy of the prediction model. Previous studies on SDP have shown that CD may occur in software defect data and the used defect prediction model may require to be updated to deal with CD. This phenomenon of handling the CD is known as CD adaptation. It is observed that still efforts need to be done in this direction in the SDP domain. In this article, we have proposed a pair of paired learners (PoPL) approach for handling CD in SDP. We combined the drift detection capabilities of two independent paired learners and used the paired learner (PL) with the best performance in recent time for next prediction. We experimented on various publicly available software defect datasets garnered from public data repositories. Experimentation results showed that our proposed approach performed better than the existing similar works and the base PL model based on various performance measures.

A random approximate reduct-based ensemble learning approach and its application in software defect prediction

Software defect prediction (SDP) is an important research topic in software engineering. It can optimize the allocation of testing resources by indicating the defect-prone software modules. In recent years, ensemble learning has been frequently used in SDP. Ensemble learning can effectively improve the generalization ability of individual learners, but how to increase the diversity of individual learners is still an open issue. In this paper, we propose a random approximate reduct-based ensemble algorithm called ERAR. To increase the diversity of learners generated by ERAR, a novel technique, called random approximate reduct (RAR), is proposed to perturb the attribute space, and the resampling technique is used to perturb the instance space. RAR is derived from the random subspace method and the granular decision entropy-based reduct in rough set theory. We then investigate the application of ERAR in SDP. In order to alleviate the class imbalance issue in SDP, we further design a hybrid mechanism by combining ERAR with the Synthetic Minority Over-sampling Technique (SMOTE). Experimental results on 20 benchmark data sets show that the hybrid mechanism can provide competitive solutions for SDP.

Machine Learning Empowered Software Defect Prediction System

Production of high-quality software at lower cost has always been the main concern of developers. However, due to exponential increases in size and complexity, the development of qualitative software with lower costs is almost impossible. This issue can be resolved by identifying defects at the early stages of the development lifecycle. As a significant amount of resources are consumed in testing activities, if only those software modules are shortlisted for testing that is identified as defective, then the overall cost of development can be reduced with the assurance of high quality. An artificial neural network is considered as one of the extensively used machine-learning techniques for predicting defect-prone software modules. In this paper, a cloud-based framework for real-time software-defect prediction is presented. In the proposed framework, empirical analysis is performed to compare the performance of four training algorithms of the back-propagation technique on software-defect prediction: Bayesian regularization (BR), Scaled Conjugate Gradient, Broyden–Fletcher–Goldfarb–Shanno Quasi-Newton, and Levenberg-Marquardt algorithms. The proposed framework also includes a fuzzy layer to identify the best training function based on performance. Publicly available cleaned versions of NASA datasets are used in this study. Various measures are used for performance evaluation including specificity, precision, recall, F-measure, an area under the receiver operating characteristic curve, accuracy, R2, and mean-square error. Two graphical user interface tools are developed in MatLab software to implement the proposed framework. The first tool is developed for comparing training functions as well as for extracting the results; the second tool is developed for the selection of the best training function using fuzzy logic. A BR training algorithm is selected by the fuzzy layer as it outperformed the others in most of the performance measures. The accuracy of the BR training function is also compared with other widely used machine-learning techniques, from which it was found that the BR performed better among all training functions.

An Enhanced Evolutionary Software Defect Prediction Method Using Island Moth Flame Optimization

Software defect prediction (SDP) is crucial in the early stages of defect-free software development before testing operations take place. Effective SDP can help test managers locate defects and defect-prone software modules. This facilitates the allocation of limited software quality assurance resources optimally and economically. Feature selection (FS) is a complicated problem with a polynomial time complexity. For a dataset with N features, the complete search space has 2N feature subsets, which means that the algorithm needs an exponential running time to traverse all these feature subsets. Swarm intelligence algorithms have shown impressive performance in mitigating the FS problem and reducing the running time. The moth flame optimization (MFO) algorithm is a well-known swarm intelligence algorithm that has been used widely and proven its capability in solving various optimization problems. An efficient binary variant of MFO (BMFO) is proposed in this paper by using the island BMFO (IsBMFO) model. IsBMFO divides the solutions in the population into a set of sub-populations named islands. Each island is treated independently using a variant of BMFO. To increase the diversification capability of the algorithm, a migration step is performed after a specific number of iterations to exchange the solutions between islands. Twenty-one public software datasets are used for evaluating the proposed method. The results of the experiments show that FS using IsBMFO improves the classification results. IsBMFO followed by support vector machine (SVM) classification is the best model for the SDP problem over other compared models, with an average G-mean of 78%.

Impact of Parameter Tuning for Optimizing Deep Neural Network Models for Predicting Software Faults

Deep neural network models built by the appropriate design decisions are crucial to obtain the desired classifier performance. This is especially desired when predicting fault proneness of software modules. When correctly identified, this could help in reducing the testing cost by directing the efforts more towards the modules identified to be fault prone. To be able to build an efficient deep neural network model, it is important that the parameters such as number of hidden layers, number of nodes in each layer, and training details such as learning rate and regularization methods be investigated in detail. The objective of this paper is to show the importance of hyperparameter tuning in developing efficient deep neural network models for predicting fault proneness of software modules and to compare the results with other machine learning algorithms. It is shown that the proposed model outperforms the other algorithms in most cases.

Cross-Project Defect Prediction via Landmark Selection-Based Kernelized Discriminant Subspace Alignment

Cross-project defect prediction (CPDP) refers to identifying defect-prone software modules in one project (target) using historical data collected from other projects (source), which can help developers find bugs and prioritize their testing efforts. Recently, CPDP has attracted great research interest. However, the source and target data usually exist redundancy and nonlinearity characteristics. Besides, most CPDP methods do not exploit source label information to uncover the underlying knowledge for label propagation. These factors usually lead to unsatisfactory CPDP performance. To address the above limitations, we propose a landmark selection-based kernelized discriminant subspace alignment (LSKDSA) approach for CPDP. LSKDSA not only reduces the discrepancy of the data distributions between the source and target projects, but also characterizes the complex data structures and increases the probability of linear separability of the data. Moreover, LSKDSA encodes label information of the source data into domain adaptation learning process and makes itself with good discriminant ability. Extensive experiments on 13 public projects from three benchmark datasets demonstrate that LSKDSA performs better than a range of competing CPDP methods. The improvement is 3.44%-11.23% in g-measure, 5.75%-11.76% in AUC, and 9.34%-33.63% in MCC, respectively.

Software Defect Prediction Using Supervised Machine Learning Techniques: A Systematic Literature Review

Software defect prediction (SDP) is the process of detecting defect-prone software modules before the testing stage. The testing stage in the software development life cycle is expensive and consumes the most resources of all the stages. SDP can minimize the cost of the testing stage, which can ultimately lead to the development of higher-quality software at a lower cost. With this approach, only those modules classified as defective are tested. Over the past two decades, many researchers have proposed methods and frameworks to improve the performance of the SDP process. The main research topics are association, estimation, clustering, classification, and dataset analysis. This study provides a systematic literature review that highlights the latest research trends in the area of SDP by providing a critical review of papers published between 2016 and 2019. Initially, 1012 papers were shortlisted from three online libraries (IEEE Xplore, ACM, and ScienceDirect); following a systematic research protocol, 22 of these papers were selected for detailed critical review. This review will serve researchers by providing the most current picture of the published work on software defect classification.

Heterogeneous Defect Prediction Based on Federated Transfer Learning via Knowledge Distillation

Heterogeneous defect prediction (HDP) aims to predict defect-prone software modules in one project using heterogeneous data collected from other projects. There are two characteristics of defect data: data islands, and data privacy. In this article, we propose a novel Federated Transfer Learning via Knowledge Distillation (FTLKD) approach for HDP, which takes into consideration two characteristics of defect data. Firstly, Shamir sharing technology achieves homomorphic encryption for private data. During subsequent processing and operations, data remains encrypted all the time. Secondly, each participant uses public data to train convolutional neural networks(CNN), the parameters of the pre-trained CNN are transferred to a private model. A small amount of labeled private data fine-tunes the private model. Finally, knowledge distillation realizes the communication between the participants. The average of all softmax output (logits) is used for knowledge distillation to update the private models. Extensive experiments on 9 projects in 3 public databases (NASA, AEEEM and SOFTLAB) show that FTLKD outperforms the related competing methods.

A comprehensive comparative study of clustering-based unsupervised defect prediction models

Software defect prediction recommends the most defect-prone software modules for optimization of the test resource allocation. The limitation of the extensively-studied supervised defect prediction methods is that they require labeled software modules which are not always available. An alternative solution is to apply clustering-based unsupervised models to the unlabeled defect data, called Clustering-based Unsupervised Defect Prediction (CUDP). However, there are few studies to explore the impacts of clustering-based models on defect prediction performance. In this work, we performed a large-scale empirical study on 40 unsupervised models to fill this gap. We chose an open-source dataset including 27 project versions with 3 types of features. The experimental results show that (1) different clustering-based models have significant performance differences and the performance of models in the instance-violation-score-based clustering family is obviously superior to that of models in hierarchy-based, density-based, grid-based, sequence-based, and hybrid-based clustering families; (2) the models in the instance-violation-score-based clustering family achieves competitive performance compared with typical supervised models; (3) the impacts of feature types on the performance of the models are related to the indicators used; and (4)the clustering-based unsupervised models do not always achieve better performance on defect data with the combination of the 3 types of features.

A Fuzzy Logic-Based Model for Classifying Software Modules in Order to Achieve Dependable Software

In this study, a fuzzy logic-based framework has been proposed to predict the situation of the software modules in the earlier phases of software lifecycle. The proposed model has taken into account domain experts' opinions and the available state of different software metrics as inputs. On the basis of dependability measures of software, different modules have been ranked earlier in the development process. Effect of the modules on the reliability, security, and availability of software has been judged by the proposed technique based on Mahalanobis distance metric. The study of software dependability in early phase assists the software developers to take corrective actions, which leads to minimize the testing efforts as well as development time. The proposed technique has been implemented on the promise software engineering repository data set. Performance of the proposed methodology is promising in identifying the fault-prone software modules. The result has also been compared with some known methodologies.

Prediction of Defect Prone Software Modules using MLP based Ensemble Techniques

Prediction of Defect Prone Software Modules using MLP based Ensemble Techniques

Software Defect Prediction Using Heterogeneous Ensemble Classification Based on Segmented Patterns

Software defect prediction is a promising approach aiming to improve software quality and testing efficiency by providing timely identification of defect-prone software modules before the actual testing process begins. These prediction results help software developers to effectively allocate their limited resources to the modules that are more prone to defects. In this paper, a hybrid heterogeneous ensemble approach is proposed for the purpose of software defect prediction. Heterogeneous ensembles consist of set of classifiers of different learning base methods in which each of them has its own strengths and weaknesses. The main idea of the proposed approach is to develop expert and robust heterogeneous classification models. Two versions of the proposed approach are developed and experimented. The first is based on simple classifiers, and the second is based on ensemble ones. For evaluation, 21 publicly available benchmark datasets are selected to conduct the experiments and benchmark the proposed approach. The evaluation results show the superiority of the ensemble version over other well-regarded basic and ensemble classifiers.

Linear regression with factor analysis in fault prediction of software

In software fault prediction, before beginning with the real software testing process, faultprone software modules are identified with the help of various properties related to the software project. This leads to the achievement of minimal cost apart from the desired software quality. In this study, Object oriented metrics were used to find the main factors. The technique used to find the important predictors is Factor analysis (FA) and regression is used to analyze the goodness of fit of different models drawn from previous studies. Following this, identification of various factors associated with the software fault prediction process is done. In addition, a robust model is developed on the basis of the distinct identified factors. The research work is the extension of our previous work. This research provides a novel approach of different aspects of the software fault prediction process and also assists in discovering the different types of problems associated with the term i.e. software fault prediction. Towards the end, the statistical comparison and the challenges of the paper are discussed along with the future guidelines of the study.

Lessons Learned from the Assessment of Software Defect Prediction on WLCG Software: A Study with Unlabelled Datasets and Machine Learning Techniques

Software defect prediction is an activity that aims at narrowing down the most likely defect-prone software modules and helping developers and testers to prioritize inspection and testing. This activity can be addressed by using Machine Learning techniques applied to software metrics datasets that are usually unlabelled, i.e. they lack modules classification in terms of defectiveness. To overcome this limitation, in addition to the usual data pre-processing operations to manage mission values and/or to remove inconsistencies, researches have to adopt an approach to label their unlabelled software datasets. The extraction of defectiveness data to label all the instances of the datasets is an extremely time and effort consuming operation. In literature, many studies have introduced approaches to build a defect prediction models on unlabelled datasets.In this paper, we describe the analysis of new unlabelled datasets from WLCG software, coming from HEP-related experiments and middleware, by using Machine Learning techniques. We have experimented new approaches to label the various modules due to the heterogeneity of software metrics distribution. We discuss a number of lessons learned from conducting these activities, what has worked, what has not and how our research can be improved.

Software defect prediction based on non-linear manifold learning and hybrid deep learning techniques

Software defect prediction plays a very important role in software quality assurance, which aims to inspect as many potentially defect-prone software modules as possible. However, the performance of the prediction model is susceptible to high dimensionality of the dataset that contains irrelevant and redundant features. In addition, software metrics for software defect prediction are almost entirely traditional features compared to the deep semantic feature representation from deep learning techniques. To address these two issues, we propose the following two solutions in this paper: (1) We leverage a novel non-linear manifold learning method - SOINN Landmark Isomap (SL-Isomap) to extract the representative features by selecting automatically the reasonable number and position of landmarks, which can reveal the complex intrinsic structure hidden behind the defect data. (2) We propose a novel defect prediction model named DLDD based on hybrid deep learning techniques, which leverages denoising autoencoder to learn true input features that are not contaminated by noise, and utilizes deep neural network to learn the abstract deep semantic features. We combine the squared error loss function of denoising autoencoder with the cross entropy loss function of deep neural network to achieve the best prediction performance by adjusting a hyperparameter. We compare the SL-Isomap with seven state-of-the-art feature extraction methods and compare the DLDD model with six baseline models across 20 open source software projects. The experimental results verify that the superiority of SL-Isomap and DLDD on four evaluation indicators.

On cost-effective software defect prediction: Classification or ranking?

Software defect prediction aims to identify defect-prone software modules in the purpose of allocating limited software reliability resources in a cost-effective manner. To identify these defect-prone modules, a classification model is usually built to classify software modules into defective or clean. However, a number of false alarms may be generated if the classifier is not so strong or the training modules are insufficient, which may waste many software reliability resources especially when the identified defect-prone modules are more than the available software reliability resources. A straight-forward way to prioritize the defect-prone modules to be checked is to rank the modules based on the predictive confidence generated by the classifier. Nevertheless, due to the ‘point-wise’ learning nature of the classifier, such predictive confidence may not be effective in generating the ordering of defect-proneness. To address this problem, we argue that the defect prediction model should be generated from a ‘pair-wise’ perspective instead of a ‘point-wise’ perspective, to explicitly model the pair-wise relations between software modules and help improve the prediction. We cast the software defect prediction as a learning to rank problem and address it by proposing a novel defect ranking model named DeRanken. Empirical studies on multiple open source software projects indicate that ranking model is able to provide better module prioritization than the classification model for Within-Project, Cross-Version and Cross-Project Defect Prediction, leading to a more cost effective software reliability.

Heterogeneous defect prediction with two-stage ensemble learning

Heterogeneous defect prediction (HDP) refers to predicting defect-prone software modules in one project (target) using heterogeneous data collected from other projects (source). Recently, several HDP methods have been proposed. However, these methods do not sufficiently incorporate the two characteristics of the defect data: (1) data could be linear inseparable, and (2) data could be highly imbalanced. These two data characteristics make it challenging to build an effective HDP model. In this paper, we propose a novel Two-Stage Ensemble Learning (TSEL) approach to HDP, which contains two stages: ensemble multi-kernel domain adaptation (EMDA) stage and ensemble data sampling (EDS) stage. In the EMDA stage, we develop an Ensemble Multiple Kernel Correlation Alignment (EMKCA) predictor, which combines the advantage of multiple kernel learning and domain adaptation techniques. In the EDS stage, we employ RESample with replacement (RES) technique to learn multiple different EMKCA predictors and use average ensemble to combine them together. These two stages create an ensemble of defect predictors. Extensive experiments on 30 public projects show that the proposed TSEL approach outperforms a range of competing methods. The improvement is 20.14–33.92% in AUC, 36.05–54.78% in f-measure, and 5.48–19.93% in balance, respectively.

Machine Learning Based Prediction of Complex Bugs in Source Code

During software development and maintenance phases, the fixing of severe bugs are mostly very challenging and needs more efforts to fix them on a priority basis. Several research works have been performed using software metrics and predict fault-prone software module. In this paper, we propose an approach to categorize different types of bugs according to their severity and priority basis and then use them to label software metrics’ data. Finally, we used labeled data to train the supervised machine learning models for the prediction of fault prone software modules. Moreover, to build an effective prediction model, we used genetic algorithm to search those sets of metrics which are highly correlated with severe bugs.

Software defect prediction based on kernel PCA and weighted extreme learning machine

ContextSoftware defect prediction strives to detect defect-prone software modules by mining the historical data. Effective prediction enables reasonable testing resource allocation, which eventually leads to a more reliable software. ObjectiveThe complex structures and the imbalanced class distribution in software defect data make it challenging to obtain suitable data features and learn an effective defect prediction model. In this paper, we propose a method to address these two challenges. MethodWe propose a defect prediction framework called KPWE that combines two techniques, i.e., Kernel Principal Component Analysis (KPCA) and Weighted Extreme Learning Machine (WELM). Our framework consists of two major stages. In the first stage, KPWE aims to extract representative data features. It leverages the KPCA technique to project the original data into a latent feature space by nonlinear mapping. In the second stage, KPWE aims to alleviate the class imbalance. It exploits the WELM technique to learn an effective defect prediction model with a weighting-based scheme. ResultsWe have conducted extensive experiments on 34 projects from the PROMISE dataset and 10 projects from the NASA dataset. The experimental results show that KPWE achieves promising performance compared with 41 baseline methods, including seven basic classifiers with KPCA, five variants of KPWE, eight representative feature selection methods with WELM, 21 imbalanced learning methods. ConclusionIn this paper, we propose KPWE, a new software defect prediction framework that considers the feature extraction and class imbalance issues. The empirical study on 44 software projects indicate that KPWE is superior to the baseline methods in most cases.

A New Framework Consisted of Data Preprocessing and Classifier Modelling for Software Defect Prediction

Different data preprocessing methods and classifiers have been established and evaluated earlier for the software defect prediction (SDP) across projects. These novel approaches have provided relatively acceptable prediction results for different software projects. However, to the best of our knowledge, few researchers have combined data preprocessing and building robust classifier simultaneously to improve prediction performances in SDP. Therefore, this paper presents a new whole framework for predicting fault-prone software modules. The proposed framework consists of instance filtering, feature selection, instance reduction, and establishing a new classifier. Additionally, we find that the 21 main software metrics commonly do follow nonnormal distribution after performing a Kolmogorov-Smirnov test. Therefore, the newly proposed classifier is built on the maximum correntropy criterion (MCC). The MCC is well-known for its effectiveness in handling non-Gaussian noise. To evaluate the new framework, the experimental study is designed with due care using nine open-source software projects with their 32 releases, obtained from the PROMISE data repository. The prediction accuracy is evaluated using F-measure. The state-of-the-art methods for Cross-Project Defect Prediction are also included for comparison. All of the evidences derived from the experimentation verify the effectiveness and robustness of our new framework.