Generalized k-nearest neighbor rules

Abstract

This paper discusses a suitable framework for generalizing the k-nearest neighbor ( k-NNR) algorithms to cases where the design labels are not necessarily crisp, i.e., not binary-valued. The proposed framework imbeds all crisp k-NNR's into a larger structure of fuzzy k-NNR's. The resultant model enables neighborhood voting to be a continuous function of local labels at a point to be classified. We emphasize that the decision itself may be crisp even when a fuzzy k-NNR is utilized. The usefulness of this extension of the conventional technique is illustrated by comparing the observed error rates of four classifiers (the hard k-NNR, two fuzzy k-NNR's, and a fuzzy 1-nearest prototype rule (1-NPR) on three data sets: Anderson's Iris data, and samples from (synthetic) univariate and bivariate normal mixtures. Our conclusions: all four designs yield comparable (usually within 4%) error rates; the Fuzzy c-Means (FCM) based k-NNR is usually the best design; the FCM/1-NPR is the most efficient and perhaps most useful of the four designs; and finally, that generalized NNR's are an important and useful extension of the conventional ones.