Missing data techniques in analogy-based software development effort estimation

Abstract

Missing Data (MD) is a widespread problem that can affect the ability to use data to construct effective software development effort prediction systems. This paper investigates the use of missing data (MD) techniques with two analogy-based software development effort estimation techniques: Classical Analogy and Fuzzy Analogy. More specifically, we analyze the predictive performance of these two analogy-based techniques when using toleration, deletion or k-nearest neighbors (KNN) imputation techniques. A total of 1512 experiments were conducted involving seven data sets, three MD techniques (toleration, deletion and KNN imputation), three missingness mechanisms (MCAR: missing completely at random, MAR: missing at random, NIM: non-ignorable missing), and MD percentages from 10 percent to 90 percent. The results suggest that Fuzzy Analogy generates more accurate estimates in terms of the Standardized Accuracy measure (SA) than Classical Analogy regardless of the MD technique, the data set used, the missingness mechanism or the MD percentage. Moreover, this study found that the use of KNN imputation, rather than toleration or deletion, may improve the prediction accuracy of both analogy-based techniques. However, toleration, deletion and KNN imputation are affected by the missingness mechanism and the MD percentage, both of which have a strong negative impact upon effort prediction accuracy.