Prototype reduction based on Direct Weighted Pruning

Abstract

Instance-based learning methods often suffer from problems related to high storage requirements, large computational costs for searching through the stored instances to find the ones most similar to the queries, and also sensitivity to noisy samples. In order to deal with these issues, various condensation algorithms have been proposed in the literature to reduce the set of prototypes that need to be stored. In this paper, we propose a new algorithm that uses a set of weights to directly control which prototypes have to be discarded or survive. Instead of relying on indirect heuristics, it explicitly optimizes a bi-objective index which incorporates the condensation rate and a measure of the classification inaccuracy as reflected by the nearest neighbor rule. The proposed algorithm, referred to as DWP (Direct Weighted Pruning), performs an efficient search using a simple genetic algorithm, which is however equipped with three novel acceleration mechanisms to notably speed up its convergence. Experiments over a large number of datasets and comparisons against many other successful condensation algorithms, show that DWP is very effective and achieves the highest classification accuracy along with competitive condensation rates.