Ensemble k-nearest neighbors based on centroid displacement

Abstract

k-nearest neighbors (k-NN) is a well-known classification algorithm that is widely used in different domains. Despite its simplicity, effectiveness and robustness, k-NN is limited by the use of the Euclidean distance as the similarity metric, the arbitrarily selected neighborhood size k, the computational challenge of high-dimensional data, and the use of the simple majority voting rule in class determination. We sought to address the last issue and proposed the Centroid Displacement-based k-NN algorithm, where centroid displacement is used for class determination. This paper presents a simple yet efficient variant of our previous work, named Ensemble Centroid Displacement-based k-NN, which leverages the homogeneity of the nearest neighbors of test instances. Extensive experiments on various real and synthetic datasets were conducted to show the effectiveness and robustness of the proposed algorithm. Our experimental results demonstrate that the proposed algorithm is able to enhance the classification performance of the standard k-NN algorithm and its variants and also improve the computational efficiency. The performance of our algorithm was consistent and robust for both balanced and imbalanced datasets.