Software Defect Data Research Articles

Software Defect Prediction via Deep Belief Network

Defect distribution prediction is a meaningful topic because software defects are the fundamental cause of many attacks and data loss. Building accurate prediction models can help developers find bugs and prioritize their testing efforts. Previous researches focus on exploring different machine learning algorithms based on the features that encode the characteristics of programs. The problem of data redundancy exists in software defect data set, which has great influence on prediction effect. We propose a defect distribution prediction model (Deep belief network prediction model, DBNPM), a system for detecting whether a program module contains defects. The key insight of DBNPM is Deep belief network (DBN) technology, which is an effective deep learning technique in image processing and natural language processing, whose features are similar to defects in source program. Experiment results show that DBNPM can efficiently extract and process the data characteristics of source program and the performance is better than Support vector machine (SVM), Locally linear embedding SVM (LLE-SVM), and Neighborhood preserving embedding SVM (NPE-SVM).

Incremental Feature Selection Method for Software Defect Prediction

Software defect prediction models are essential for understanding quality attributes relevant for software organization to deliver better software reliability. This paper focuses mainly based on the selection of attributes in the perspective of software quality estimation for incremental database. A new dimensionality reduction method Wilk’s Lambda Average Threshold (WLAT) is presented for selection of optimal features which are used for classifying modules as fault prone or not. This paper uses software metrics and defect data collected from benchmark data sets. The comparative results confirm that the statistical search algorithm (WLAT) outperforms the other relevant feature selection methods for most classifiers. The main advantage of the proposed WLAT method is: The selected features can be reused when there is increase or decrease in database size, without the need of extracting features afresh. In addition, performances of the defect prediction models either remains unchanged or improved even after eliminating 85% of the software metrics.

A new weighted naive Bayes method based on information diffusion for software defect prediction

Software defect prediction (SDP) plays a significant part in identifying the most defect-prone modules before software testing and allocating limited testing resources. One of the most commonly used classifiers in SDP is naive Bayes (NB). Despite the simplicity of the NB classifier, it can often perform better than more complicated classification models. In NB, the features are assumed to be equally important, and the numeric features are assumed to have a normal distribution. However, the features often do not contribute equivalently to the classification, and they usually do not have a normal distribution after performing a Kolmogorov-Smirnov test; this may harm the performance of the NB classifier. Therefore, this paper proposes a new weighted naive Bayes method based on information diffusion (WNB-ID) for SDP. More specifically, for the equal importance assumption, we investigate six weight assignment methods for setting the feature weights and then choose the most suitable one based on the F-measure. For the normal distribution assumption, we apply the information diffusion model (IDM) to compute the probability density of each feature instead of the acquiescent probability density function of the normal distribution. We carry out experiments on 10 software defect data sets of three types of projects in three different programming languages provided by the PROMISE repository. Several well-known classifiers and ensemble methods are included for comparison. The final experimental results demonstrate the effectiveness and practicability of the proposed method.

Ensemble MultiBoost Based on RIPPER Classifier for Prediction of Imbalanced Software Defect Data

Identifying defective software entities is essential to ensure software quality during software development. However, the high dimensionality and class distribution imbalance of software defect data seriously affect software defect prediction performance. In order to solve this problem, this paper proposes an E nsemble M ultiBoost based on R IPPER classifier for prediction of imbalanced S oftware D efect data, called EMR_SD . Firstly, the algorithm uses principal component analysis (PCA) method to find out the most effective features from the original features of the data set, so as to achieve the purpose of dimensionality reduction and redundancy removal. Furthermore, the combined sampling method of adaptive synthetic sampling (ADASYN) and random sampling without replacement is performed to solve the problem of data class imbalance. This classifier establishes association rules based on attributes and classes, using MultiBoost to reduce deviation and variance, so as to achieve the purpose of reducing classification error. The proposed prediction model is evaluated experimentally on the NASA MDP public datasets and compared with existing similar algorithms. The results show that EMR_SD algorithm is superior to DNC, CEL and other defect prediction techniques in most evaluation indicators, which proves the effectiveness of the algorithm.

Applying the FAHP to Improve the Performance Evaluation Reliability of Software Defect Classifiers

Today's software complexity makes developing defect-free software almost impossible. Consequently, developing classifiers to classify software modules into defective and non-defective before software releases have attracted great interest in academia and software industry alike. Although many classifiers have been proposed, no one has been proven superior over others. The major reason is that while a research shows that classifier A is better than classifier B, we can find other research that shows the opposite. These conflicts are usually triggered when researchers report results using their preferable performance evaluation measures such as, recall and precision. Although this approach is valid, it does not examine all possible facets of classifiers performance characteristics. Thus, the performance evaluation might improve or deteriorate if researchers choose other performance measures. As a result, software developers usually struggle to select the most suitable classifier to use in their projects. The goal of this paper is to apply the fuzzy analytical hierarchy process (FAHP) as a popular multicriteria decision-making technique to reliably evaluate classifiers' performance. This evaluation framework incorporates a wider spectrum of performance measures to evaluate classifiers performance rather than relying on selected preferable measures. The results show that this approach will increase software developers' confidence in research outcomes and help them in avoiding false conclusions and infer reasonable boundaries for them. We exploited 22 popular performance measures and 11 software defect classifiers. The analysis was carried out using KNIME data mining platform and 12 software defect data sets provided by the NASA metrics data program (MDP) repository.

Application of the Pearson’s Method in Model Software Quality Data Analysis

Third-party model software testing involves and generates large amounts of model software quality data. Currently, there is a deficiency in the collection, analysis and application of these data. Due to the characteristics of model software products (e.g., they are complex, uncertain and difficult to measure) as well as the limited and unsystematic understanding of model software quality, model software quality evaluation has always been a challenge in software quality research. Based on the features of third-party model software testing as well as the model software quality data collected during third-party testing, this study focuses on an in-depth analysis of defect data collected during model software testing by using the Pearson product-moment correlation coefficient, presents corresponding conclusions derived from the analysis of model software defect data, presents suggestions for software testing and evaluation organizations to scientifically manage model software quality data and improve software testing efficiency and provides quality control and improvement measures for software developers.

Software defect prediction based on kernel PCA and weighted extreme learning machine

ContextSoftware defect prediction strives to detect defect-prone software modules by mining the historical data. Effective prediction enables reasonable testing resource allocation, which eventually leads to a more reliable software. ObjectiveThe complex structures and the imbalanced class distribution in software defect data make it challenging to obtain suitable data features and learn an effective defect prediction model. In this paper, we propose a method to address these two challenges. MethodWe propose a defect prediction framework called KPWE that combines two techniques, i.e., Kernel Principal Component Analysis (KPCA) and Weighted Extreme Learning Machine (WELM). Our framework consists of two major stages. In the first stage, KPWE aims to extract representative data features. It leverages the KPCA technique to project the original data into a latent feature space by nonlinear mapping. In the second stage, KPWE aims to alleviate the class imbalance. It exploits the WELM technique to learn an effective defect prediction model with a weighting-based scheme. ResultsWe have conducted extensive experiments on 34 projects from the PROMISE dataset and 10 projects from the NASA dataset. The experimental results show that KPWE achieves promising performance compared with 41 baseline methods, including seven basic classifiers with KPCA, five variants of KPWE, eight representative feature selection methods with WELM, 21 imbalanced learning methods. ConclusionIn this paper, we propose KPWE, a new software defect prediction framework that considers the feature extraction and class imbalance issues. The empirical study on 44 software projects indicate that KPWE is superior to the baseline methods in most cases.

Binary teaching–learning-based optimization algorithm with a new update mechanism for sample subset optimization in software defect prediction

Software defect prediction has gained considerable attention in recent years. A broad range of computational methods has been developed for accurate prediction of faulty modules based on code and design metrics. One of the challenges in training classifiers is the highly imbalanced class distribution in available datasets, leading to an undesirable bias in the prediction performance for the minority class. Data sampling is a widespread technique to tackle this problem. However, traditional sampling methods, which depend mainly on random resampling from a given dataset, do not take advantage of useful information available in training sets, such as sample quality and representative instances. To cope with this limitation, evolutionary undersampling methods are usually used for identifying an optimal sample subset for the training dataset. This paper proposes a binary teaching–learning- based optimization algorithm employing a distribution-based solution update rule, namely BTLBOd, to generate a balanced subset of highly valuable examples. This subset is then applied to train a classifier for reliable prediction of potentially defective modules in a software system. Each individual in BTLBOd includes two vectors: a real-valued vector generated by the distribution-based update mechanism, and a binary vector produced from the corresponding real vector by a proposed mapping function. Empirical results showed that the optimal sample subset produced by BTLBOd might ameliorate the classification accuracy of the predictor on highly imbalanced software defect data. Obtained results also demonstrated the superior performance of the proposed sampling method compared to other popular sampling techniques.

On the relative value of data resampling approaches for software defect prediction

Software defect data sets are typically characterized by an unbalanced class distribution where the defective modules are fewer than the non-defective modules. Prediction performances of defect prediction models are detrimentally affected by the skewed distribution of the faulty minority modules in the data set since most algorithms assume both classes in the data set to be equally balanced. Resampling approaches address this concern by modifying the class distribution to balance the minority and majority class distribution. However, very little is known about the best distribution for attaining high performance especially in a more practical scenario. There are still inconclusive results pertaining to the suitable ratio of defect and clean instances (Pfp), the statistical and practical impacts of resampling approaches on prediction performance and the more stable resampling approach across several performance measures. To assess the impact of resampling approaches, we investigated the bias and effect of commonly used resampling approaches on prediction accuracy in software defect prediction. Analyzes of six resampling approaches on 40 releases of 20 open-source projects across five performance measures and five imbalance rates were performed. The experimental results obtained indicate that there were statistical differences between the prediction results with and without resampling methods when evaluated with the geometric-mean, recall(pd), probability of false alarms(pf ) and balance performance measures. However, resampling methods could not improve the AUC values across all prediction models implying that resampling methods can help in defect classification but not defect prioritization. A stable Pfp rate was dependent on the performance measure used. Lower Pfp rates are required for lower pf values while higher Pfp values are required for higher pd values. Random Under-Sampling and Borderline-SMOTE proved to be the more stable resampling method across several performance measures among the studied resampling methods. Performance of resampling methods are dependent on the imbalance ratio, evaluation measure and to some extent the prediction model. Newer oversampling methods should aim at generating relevant and informative data samples and not just increasing the minority samples.

RETRACTED: Cognitive Deep Neural Networks prediction method for software fault tendency module based on Bound Particle Swarm Optimization

Abstract Identification of module fault tendency is greatly important for cost reduction and software development effectiveness. A DNN (Deep Neural Networks) prediction method for software fault tendency module based on BPSO (Bound Particle Swarm Optimization) dimensionality reduction was proposed in the paper. Firstly, the calculation framework of the DNN prediction algorithm for software fault tendency module based on BPSO dimensionality reduction and 21 software fault measurement indexes as well as the normalization processing method of these index values were provided in the paper; then, the particle swarm optimization algorithm was adopted for the dimensionality reduction of software fault data set, and the particle position was represented by binary (0 or 1) character string to simplify data processing; then, the DNN algorithm was adopted to predict software fault tendency module; finally, the simulation experiments were implemented in four standard test sets—PC1, JM1, KC1 and KC3 to verify the performance advantage of the algorithm.

Are vulnerabilities discovered and resolved like other defects?

Software defect data has long been used to drive software development process improvement. If security defects (vulnerabilities) are discovered and resolved by different software development practices than non-security defects, the knowledge of that distinction could be applied to drive process improvement. The goal of this research is to support technical leaders in making security-specific software development process improvements by analyzing the differences between the discovery and resolution of defects versus that of vulnerabilities. We extend Orthogonal Defect Classification (ODC), a scheme for classifying software defects to support software development process improvement, to study process-related differences between vulnerabilities and defects, creating ODC + Vulnerabilities (ODC + V). We applied ODC + V to classify 583 vulnerabilities and 583 defects across 133 releases of three open-source projects (Firefox, phpMyAdmin, and Chrome). Compared with defects, vulnerabilities are found later in the development cycle and are more likely to be resolved through changes to conditional logic. In Firefox, vulnerabilities are resolved 33% more quickly than defects. From a process improvement perspective, these results indicate opportunities may exist for more efficient vulnerability detection and resolution.

Perbaikan Prediksi Kesalahan Perangkat Lunak Menggunakan Seleksi Fitur dan Cluster-Based Classification

High balance value of software fault prediction can help in conducting test effort, saving test costs, saving test resources, and improving software quality. Balance values in software fault prediction need to be considered, as in most cases, the class distribution of true and false in the software fault data set tends to be unbalanced. The balance value is obtained from trade-off between probability detection (pd) and probability false alarm (pf). Previous researchers had proposed Cluster-Based Classification (CBC) method which was integrated with Entropy-Based Discretization (EBD). However, predictive models with irrelevant and redundant features in data sets can decrease balance value. This study proposes improvement of software fault prediction outcomes on CBC by integrating feature selection methods. Some feature selection methods are integrated with CBC, i.e. Information Gain (IG), Gain Ration (GR), One-R (OR), Relief-F (RFF), and Symmetric Uncertainty (SU). The result shows that combination of CBC with IG gives best average balance value, compared to other feature selection methods used in this research. Using five NASA public MDP data sets, the combination of IG and CBC generates 63.91% average of balance, while CBC method without feature selection produce 54.79% average of balance. It shows that IG can increase CBC balance average by 9.12%.

The application of ROC analysis in threshold identification, data imbalance and metrics selection for software fault prediction

Software engineers have limited resources and need metrics analysis tools to investigate software quality such as fault-proneness of modules. There are a large number of software metrics available to investigate quality. However, not all metrics are strongly correlated with faults. In addition, software fault data are imbalanced and affect quality assessment tools such as fault prediction or threshold values that are used to identify risky modules. Software quality is investigated for three purposes. First, the receiver operating characteristics (ROC) analysis is used to identify threshold values to identify risky modules. Second, the ROC analysis is investigated for imbalanced data. Third, the ROC analysis is considered for feature selection. This work validated the use of ROC to identify thresholds for four metrics (WMC, CBO, RFC and LCOM). The ROC results after sampling the data are not significantly different from before sampling. The ROC analysis selects the same metrics (WMC, CBO and RFC) in most datasets, while other techniques have a large variation in selecting metrics.

Fuzzy Rule-Based Approach for Software Fault Prediction

Knowing faulty modules prior to testing makes testing more effective and helps to obtain reliable software. Here, we develop a framework for automatic extraction of human understandable fuzzy rules for software fault detection/classification. This is an integrated framework to simultaneously identify useful determinants (attributes) of faults and fuzzy rules using those attributes. At the beginning of the training, the system assumes every attribute (feature) as a useless feature and then uses a concept of feature attenuating gate to select useful features. The learning process opens the gates or closes them more tightly based on utility of the features. Our system can discard derogatory and indifferent attributes and select the useful ones. It can also exploit subtle nonlinear interaction between attributes. In order to demonstrate the effectiveness of the framework, we have used several publicly available software fault data sets and compared the performance of our method with that of some existing methods. The results using tenfold cross-validation setup show that our system can find useful fuzzy rules for fault prediction.

A Neuro-Based Software Fault Prediction with Box-Cox Power Transformation

Software fault prediction is one of the most fundamental but significant management techniques in software dependability assessment. In this paper we concern the software fault prediction using a multilayer-perceptron neural network, where the underlying software fault count data are transformed to the Gaussian data, by means of the well-known Box-Cox power transformation. More specially, we investigate the long-term behavior of software fault counts by the neural network, and perform the multi-stage look ahead prediction of the cumulative number of software faults detected in the future software testing. In numerical examples with two actual software fault data sets, we compare our neural network approach with the existing software reliability growth models based on nonhomogeneous Poisson process, in terms of predictive performance with average relative error, and show that the data transformation employed in this paper leads to an improvement in prediction accuracy.

Identifying and eliminating less complex instances from software fault data

Software quality is costly to achieve for large systems. Developers and testers need to investigate a large number of software modules to ensure software quality. This investigation is time consuming and formal models such as machine learning techniques are used to predict the fault-proneness of software modules to predict where faults are more probable. However, many modules are small in either implementation or design size and have low priority to investigate their quality. In this research, machine learners are used to classify the most complex parts of software. The less complex software modules are filtered out using two size measures: number of public parameters in a software construct and line of code (LOC) as a surrogate of implementation size. Modules at lowest 10, 20 and 30% of each measure are filtered out of the trained and tested data sets. The remaining modules are used to build four classifiers: Naive Bayes, logistic regression, nearest neighbors and decision trees. The resulting classifiers were evaluated and compared against the original data. The classifiers based on NPM filtering were more consistent with the data without filtering than the LOC size measure.

Non‐negative sparse‐based SemiBoost for software defect prediction

SummarySoftware defect prediction is an important decision support activity in software quality assurance. The limitation of the labelled modules usually makes the prediction difficult, and the class‐imbalance characteristic of software defect data leads to negative influence on decision of classifiers. Semi‐supervised learning can build high‐performance classifiers by using large amount of unlabelled modules together with the labelled modules. Ensemble learning achieves a better prediction capability for class‐imbalance data by using a series of weak classifiers to reduce the bias generated by the majority class. In this paper, we propose a new semi‐supervised software defect prediction approach, non‐negative sparse‐based SemiBoost learning. The approach is capable of exploiting both labelled and unlabelled data and is formulated in a boosting framework. In order to enhance the prediction ability, we design a flexible non‐negative sparse similarity matrix, which can fully exploit the similarity of historical data by incorporating the non‐negativity constraint into sparse learning for better learning the latent clustering relationship among software modules. The widely used datasets from NASA projects are employed as test data to evaluate the performance of all compared methods. Experimental results show that non‐negative sparse‐based SemiBoost learning outperforms several representative state‐of‐the‐art semi‐supervised software defect prediction methods. Copyright © 2016 John Wiley & Sons, Ltd.

Predicting software reliability via completely monotone nonparametric estimator with grouped data

Nonparametric software reliability analysis is a challenging issue to predict software reliability under incomplete knowledge on software fault-detection time distribution, because the underlying stochastic model is a function of only software fault data and is not predictable in principle for unknown patterns in future long term. Sofer and Miller (1991) develop a unique approach based on a completely monotone nonparametric estimator, but just focus on the fault-detection time data. However, such data sets are seldom available in practice, and their approach is not applicable to many actual software development processes. In this paper, we revisit the seminal completely monotone estimator by Sofer and Miller (1991) to use in the major case with grouped data, and develop both estimation and prediction methods of software fault count. We investigate the potential performance of our distribution-free method with thirteen actual software development project data, and compare it with the existing parametric models known as nonhomogeneous Poisson process-based software reliability models.

Zero‐inflated prediction model in software‐fault data

Software fault data with many zeroes in addition to large non-zero values are common in the software estimation area. A two-component prediction approach that provides a robust way to predict this type of data is introduced in this study. This approach allows to combine parametric and non-parametric models to improve the prediction accuracy. This way provides a more flexible structure to understand data. To show the usefulness of the proposed approach, experiments using eight projects from the NASA repository are considered. In addition, this method is compared with methods from the machine learning and statistical literature. The performance of the methods is measured by the prediction accuracy that is assessed based on the mean magnitude of relative errors.

Aggregating Data Sampling with Feature Subset Selection to Address Skewed Software Defect Data

Defect prediction is an important process activity frequently used for improving the quality and reliability of software products. Defect prediction results provide a list of fault-prone modules which are necessary in helping project managers better utilize valuable project resources. In the software quality modeling process, high dimensionality and class imbalance are the two potential problems that may exist in data repositories. In this study, we investigate three data preprocessing approaches, in which feature selection is combined with data sampling, to overcome these problems in the context of software quality estimation. These three approaches are: Approach 1 — sampling performed prior to feature selection, but retaining the unsampled data instances; Approach 2 — sampling performed prior to feature selection, retaining the sampled data instances; and Approach 3 — sampling performed after feature selection. A comparative investigation is presented for evaluating the three approaches. In the experiments, we employed three sampling methods (random undersampling, random oversampling, and synthetic minority oversampling), each combined with a filter-based feature subset selection technique called correlation-based feature selection. We built the defect prediction models using five common classification algorithms. The case study was based on software metrics and defect data collected from multiple releases of a real-world software system. The results demonstrated that the type of sampling methods used in data preprocessing significantly affected the performance of the combination approaches. It was found that when the random undersampling technique was used, Approach 1 performed better than the other two approaches. However, when the feature selection technique was used in conjunction with an oversampling method (random oversampling or synthetic minority oversampling), we strongly recommended Approach 3.