PROMISE Data Repository Research Articles

An optimized case-based software project effort estimation using genetic algorithm

Software development companies have long suffered from inaccurate estimation of their software projects. This in turn led to huge losses, especially in the financial resources available for the project as well as the time required to complete it. As a result of this, the research community has developed different methods for estimating effort in software projects in the hope of achieving high levels of accuracy and efficiency in the use of available resources. Among those methods that have proven to be accurate in estimating the effort of software projects is the use of machine learning (ML) techniques, especially the case-based reasoning technique (CBR). This technique is based on adapting previously successful solutions for similar software projects. However, the CBR technique suffers from a problem which is its multiple parameters that are difficult to be tuned. This justifies the importance of the adaptation and adjustment process as an essential part of CBR to produce accurate and efficient results with least absolute estimation error. In this paper, one of the most efficient multi-objective evolutionary techniques, the Genetic Algorithm (GA), are used to help find the best set of classical CBR parameters (feature selection, feature weighting, similarity measures, and k number of nearest neighbors) to produce the most accurate effort estimates for software projects. The proposed CBR-GA model showed the effectiveness of using the GA algorithm to search for the best combination of CBR parameters and thus improve its accuracy. This in turn is beneficial for project managers in the early financial planning phase for effort estimation and thus project cost control. To validate the proposed CBR-GA model, we used a set of public benchmark datasets available on PROMISE data repository, in addition we used a set of reliable evaluation metrics. The obtained results are promising in terms of accuracy and significance tests. This implies the importance of search-based techniques for tuning effort estimation methods.

Empirical comparison and evaluation of Artificial Immune Systems in inter-release software fault prediction

Artificial immune systems are bio-inspired machine learning algorithms based on the mammalian immune paradigms. One of the possible uses of these methods is Software Fault Prediction, which consists of classifying the modules of an application as being fault-prone or not, thus allowing a developer to better target the modules during the test phase leading to a high-quality software with lower cost. Despite the high number of works in the field, only five studies included Artificial Immune Systems in their approaches and exclusively focused on the intra-project fault prediction scheme. In this study, our objective is to appraise 8 immunological systems on the rarely treated inter-project software defect prediction scenario over three different benchmarks, hence, we selected 41 datasets corresponding to 11 java projects from the PROMISE data repository. According to the Friedman and Nemenyi Post-hoc test results, none of the performance of the studied algorithms was better than Immunos-1 and Immunos-99 in terms of the Recall measure. Furthermore, the outcomes of the Wilcoxon test suggest that the researches addressing the intra-projects defect prediction problems should also evaluate their models on inter-release scenarios.

Ensemble Techniques-Based Software Fault Prediction in an Open-Source Project

Software engineering repositories have been attracted by researchers to mine useful information about the different quality attributes of the software. These repositories have been helpful to software professionals to efficiently allocate various resources in the life cycle of software development. Software fault prediction is a quality assurance activity. In fault prediction, software faults are predicted before actual software testing. As exhaustive software testing is impossible, the use of software fault prediction models can help the proper allocation of testing resources. Various machine learning techniques have been applied to create software fault prediction models. In this study, ensemble models are used for software fault prediction. Change metrics-based data are collected for an open-source android project from GIT repository and code-based metrics data are obtained from PROMISE data repository and datasets kc1, kc2, cm1, and pc1 are used for experimental purpose. Results showed that ensemble models performed better compared to machine learning and hybrid search-based algorithms. Bagging ensemble was found to be more effective in the prediction of faults in comparison to soft and hard voting.

An Approach for the Prediction of Number of Software Faults Based on the Dynamic Selection of Learning Techniques

Determining the most appropriate learning technique(s) is vital for the accurate and effective software fault prediction (SFP). Earlier techniques used for SFP have reported varying performance for different software projects and none of them has always reported the best performance across different projects. The problem of varying performance can be solved by using an approach, which partitions the fault dataset into different module subsets, trains learning techniques for each subset, and integrates the outcomes of all the learning techniques. This paper presents an approach that dynamically selects learning techniques to predict the number of software faults. For a given testing module, the presented approach first locates its neighbor module subset that contained modules similar to testing module using a distance function and then chooses the best learning technique in the region of that module subset to make the prediction for testing module. The learning technique is selected based on its past performance in the region of module subset. We have performed an evaluation of the proposed approach using fault datasets garnered from the PROMISE data repository and Eclipse bug data repository. Experimental results showed that the proposed approach led to an improved performance when predicting the number of faults in software systems.

A New Framework Consisted of Data Preprocessing and Classifier Modelling for Software Defect Prediction

Different data preprocessing methods and classifiers have been established and evaluated earlier for the software defect prediction (SDP) across projects. These novel approaches have provided relatively acceptable prediction results for different software projects. However, to the best of our knowledge, few researchers have combined data preprocessing and building robust classifier simultaneously to improve prediction performances in SDP. Therefore, this paper presents a new whole framework for predicting fault-prone software modules. The proposed framework consists of instance filtering, feature selection, instance reduction, and establishing a new classifier. Additionally, we find that the 21 main software metrics commonly do follow nonnormal distribution after performing a Kolmogorov-Smirnov test. Therefore, the newly proposed classifier is built on the maximum correntropy criterion (MCC). The MCC is well-known for its effectiveness in handling non-Gaussian noise. To evaluate the new framework, the experimental study is designed with due care using nine open-source software projects with their 32 releases, obtained from the PROMISE data repository. The prediction accuracy is evaluated using F-measure. The state-of-the-art methods for Cross-Project Defect Prediction are also included for comparison. All of the evidences derived from the experimentation verify the effectiveness and robustness of our new framework.

Linear and non-linear heterogeneous ensemble methods to predict the number of faults in software systems

Several classification techniques have been investigated and evaluated earlier for the software fault prediction. These techniques have produced different prediction accuracy for the different software systems and none of the technique has always performed consistently better across different domains. On the other hand, software fault prediction using ensemble methods can be very effective, as they take the advantage of each participating technique for the given dataset and try to come up with better prediction results compared to the individual techniques. Many works are available for classifying software modules being faulty or non-faulty using the ensemble methods. These works are only specifying that whether a given software module is faulty or not, but number of faults in that module are not predicted by them. The use of ensemble methods for the prediction of number of faults has not been explored so far. To fulfill this gap, this paper presents ensemble methods for the prediction of number of faults in the given software modules. The experimental study is designed and conducted for five open-source software projects with their fifteen releases, collected from the PROMISE data repository. The results are evaluated under two different scenarios, intra-release prediction and inter-releases prediction. The prediction accuracy of ensemble methods is evaluated using absolute error, relative error, prediction at level l, and measure of completeness performance measures. Results show that the presented ensemble methods yield improved prediction accuracy over the individual fault prediction techniques under consideration. Further, the results are consistent for all the used datasets. The evidences obtained from the prediction at level l and measure of completeness analysis have also confirmed the effectiveness of the proposed ensemble methods for predicting the number of faults.

An empirical study of some software fault prediction techniques for the number of faults prediction

During the software development process, prediction of the number of faults in software modules can be more helpful instead of predicting the modules being faulty or non-faulty. Such an approach may help in more focused software testing process and may enhance the reliability of the software system. Most of the earlier works on software fault prediction have used classification techniques for classifying software modules into faulty or non-faulty categories. The techniques such as Poisson regression, negative binomial regression, genetic programming, decision tree regression, and multilayer perceptron can be used for the prediction of the number of faults. In this paper, we present an experimental study to evaluate and compare the capability of six fault prediction techniques such as genetic programming, multilayer perceptron, linear regression, decision tree regression, zero-inflated Poisson regression, and negative binomial regression for the prediction of number of faults. The experimental investigation is carried out for eighteen software project datasets collected from the PROMISE data repository. The results of the investigation are evaluated using average absolute error, average relative error, measure of completeness, and prediction at level l measures. We also perform Kruskal–Wallis test and Dunn’s multiple comparison test to compare the relative performance of the considered fault prediction techniques.

A Decision Tree Regression based Approach for the Number of Software Faults Prediction

Software fault prediction is an important activity to make software quality assurance (SQA) process more efficient, economic and targeted. Most of earlier works related to software fault prediction have focused on classifying software modules as faulty or non-faulty. However, prediction of the number of faults in a given software module is not adequately investigated. In this paper, we explore the capability of decision tree regression (DTR) for the number of faults prediction in two different scenarios, intra-release prediction and inter-releases prediction for the given software system. The experimental study is performed over five open-source software projects with their nineteen releases collected from the PROMISE data repository. The predictive accuracy of DTR is evaluated using absolute error and relative error, prediction at level l and goodness-of-t measure. The results show that decision tree regression produced significant prediction accuracy for the number of faults prediction in both the considered scenarios. The relative comparison of intra-release and inter-releases fault prediction shows that intra-project prediction produced better accuracy compared to inter-releases prediction across all the datasets

Predicting Number of Faults in Software System using Genetic Programming

Abstract In software development perspective, dealing with software faults is a vital and foremost important task. Presence of faults not only reduces the quality of the software, but also increases its development cost. A large number of models have been presented in the past to predict the fault proneness of the software system. However, most of them provide inadequate information and thus make the task of fault prediction difficult. In this paper, we present an approach to predict the number of faults in the given software system using the Genetic Programming (GP). We validate the proposed approach using an experimental investigation where we use the fault datasets of the ten software projects available in the PROMISE data repository. The Error rate, Recall and Completeness of the fault prediction model are used to evaluate the performance of the proposed approach. The results show that GP based models have produced the significant results for the number of faults prediction.

Balancing Privacy and Utility in Cross-Company Defect Prediction

Background: Cross-company defect prediction (CCDP) is a field of study where an organization lacking enough local data can use data from other organizations for building defect predictors. To support CCDP, data must be shared. Such shared data must be privatized, but that privatization could severely damage the utility of the data. Aim: To enable effective defect prediction from shared data while preserving privacy. Method: We explore privatization algorithms that maintain class boundaries in a dataset. CLIFF is an instance pruner that deletes irrelevant examples. MORPH is a data mutator that moves the data a random distance, taking care not to cross class boundaries. CLIFF+MORPH are tested in a CCDP study among 10 defect datasets from the PROMISE data repository. Results: We find: 1) The CLIFFed+MORPHed algorithms provide more privacy than the state-of-the-art privacy algorithms; 2) in terms of utility measured by defect prediction, we find that CLIFF+MORPH performs significantly better. Conclusions: For the OO defect data studied here, data can be privatized and shared without a significant degradation in utility. To the best of our knowledge, this is the first published result where privatization does not compromise defect prediction.

Software defect prediction using Bayesian networks

There are lots of different software metrics discovered and used for defect prediction in the literature. Instead of dealing with so many metrics, it would be practical and easy if we could determine the set of metrics that are most important and focus on them more to predict defectiveness. We use Bayesian networks to determine the probabilistic influential relationships among software metrics and defect proneness. In addition to the metrics used in Promise data repository, we define two more metrics, i.e. NOD for the number of developers and LOCQ for the source code quality. We extract these metrics by inspecting the source code repositories of the selected Promise data repository data sets. At the end of our modeling, we learn the marginal defect proneness probability of the whole software system, the set of most effective metrics, and the influential relationships among metrics and defectiveness. Our experiments on nine open source Promise data repository data sets show that response for class (RFC), lines of code (LOC), and lack of coding quality (LOCQ) are the most effective metrics whereas coupling between objects (CBO), weighted method per class (WMC), and lack of cohesion of methods (LCOM) are less effective metrics on defect proneness. Furthermore, number of children (NOC) and depth of inheritance tree (DIT) have very limited effect and are untrustworthy. On the other hand, based on the experiments on Poi, Tomcat, and Xalan data sets, we observe that there is a positive correlation between the number of developers (NOD) and the level of defectiveness. However, further investigation involving a greater number of projects is needed to confirm our findings.

A replicated assessment and comparison of adaptation techniques for analogy-based effort estimation

Variants of adaptation techniques have been proposed in previous studies to improve the performance of analogy-based effort estimation. The results of these studies are often contradictory and cannot simply be generalized because there are many uncontrollable source of variations between adaptation studies. The study presented in this paper has been carried out in order to replicate the assessment and comparison of different adaptation techniques utilised in analogy-based software effort prediction. Empirical evaluation of variants of adaptation techniques with Jack-knifing procedure have been carried out. Seven datasets come from PROMISE data repository were used for benchmarking. The results are also investigated within the presence/absence of feature subset selection algorithm. The current study permitted us to discover that linear adjustment approaches are more accurate than nonlinear adjustment because of the nature of the employed datasets that have, in most cases, normality characteristics.