Evaluating noise elimination techniques for software quality estimation

Abstract

The poor quality of a training dataset can have untoward consequences in software quality estimation problems. The presence of noise in software measurement data may hinder the prediction accuracy of a given learner. A filter improves the quality of training datasets by removing data that is likely noise. We evaluate the Ensemble Filter against the Partitioning Filter and the Classification Filter. These filtering techniques combine the predictions of base classifiers in such a way that an instance is identified as noisy if it is misclassified by a given number of these learners. The Partitioning Filter first splits the training dataset into subsets, and different base learners are induced on each subset. Two different implementations of the Partitioning Filter are presented: the Multiple-Partitioning Filter and the Iterative-Partitioning Filter. In contrast, the Ensemble Filter uses base classifiers induced on the entire training dataset. The filtering level and/or the number of iterations modify the filtering conservativeness: a conservative filter is less likely to remove good data at the expense of retaining noisy instances. A unique measure for comparing the relative efficiencies of two filters is also presented. Empirical studies on a high assurance software project evaluate the relative performances of the Ensemble Filter, Multiple-Partitioning Filter, Iterative-Partitioning Filter, and Classification Filter. Our study demonstrates that with a conservative filtering approach, using several different base learners can improve the efficiency of the filtering schemes.