BP$k$ NN: $k$ -Nearest Neighbor Classifier With Pairwise Distance Metrics and Belief Function Theory

Abstract

The k-nearest neighbor (kNN) rule is one of the most popular classification algorithms in pattern recognition field because it is very simple to understand but works quite well in practice. However, the performance of the kNN rule depends critically on its being given a good distance metric over the input space, especially in small data set situations. In this paper, a new kNN-based classifier, called BPkNN, is developed based on pairwise distance metrics and belief function theory. The idea of the proposal is that instead of learning a global distance metric, we first decompose it into learning a group of pairwise distance metrics. Then, based on each learned pairwise distance metric, a pairwise kNN (PkNN) sub-classifier can be adaptively designed to separate two classes. Finally, a polychotomous classification problem is solved by combining the outputs of these PkNN sub-classifiers in belief function framework. The BPkNN classifier improves the classification performance thanks to the new distance metrics which provide more flexibility to design the feature weights and the belief function-based combination method which can better address the uncertainty involved in the outputs of the sub-classifiers. Experimental results based on synthetic and real data sets show that the proposed BPkNN can achieve better classification accuracy in comparison with some state-of-the-art methods.