Investigating heterogeneous ensembles with filter feature selection for software effort estimation

Abstract

Ensemble Effort Estimation (EEE) consists on predicting the software development effort by combining more than one single estimation technique. EEE has recently been investigated in software development effort estimation (SDEE) in order to improve the estimation accuracy. The overall results suggested that the EEE yield better prediction accuracy than single techniques. On the other hand, feature selection (FS) methods have been used in the area of SDEE for the purpose of reducing the dimensionality of a dataset size by eliminating the irrelevant and redundant features. Thus, the SDEE techniques are trained on a dataset with relevant features which can lead to improving the accuracy of their estimations. This paper aims at investigating the impact of two Filter feature selection methods: Correlation based Feature Selection (CFS) and RReliefF on the estimation accuracy of Heterogeneous (HT) ensembles. Four machine learning techniques (K-Nearest Neighbor, Support Vector Regression, Multilayer Perceptron and Decision Trees) were used as base techniques for the HT ensembles of this study. We evaluate the accuracy of these HT ensembles when their base techniques were trained on datasets preprocessed by the two feature selection methods. The HT ensembles use three combination rules: average, median, and inverse ranked weighted mean. The evaluation was carried out by means of eight unbiased accuracy measures through the leave-one-out-cross validation (LOOCV) technique over six datasets. The overall results suggest that all the attributes of most datasets used are relevant for building an accurate predictive technique since the ensembles constructed without features selection outperformed in general the ones using features selection. As for the combination rule, the median generally produces better results than the other two used in this empirical study.