K-nearest neighbors rule combining prototype selection and local feature weighting for classification

Abstract

K-Nearest Neighbors (KNN) rule is a simple yet powerful classification technique in machine learning. Nevertheless, it suffers from some drawbacks such as high memory consumption, low time efficiency, class overlapping and difficulty of setting an appropriate K value. In this study, we propose an Improved K-Nearest Neighbor rule combining Prototype Selection and Local Feature Weighting (IKNN_PSLFW) to address the above issues in one framework. Differing from conventional prototype selection, IKNN_PSLFW not only selects the representative instances as prototypes, but also preserves the information of instances that are not selected. To deal with the class overlapping problem, IKNN_PSLFW explores the feature relevance in local regions by assigning different weights to different features. For an instance with unknown class label, IKNN_PSLFW uses three classification rules corresponding to three scenarios, according to the distance between the instance and each prototype, for classification. To evaluate the performance of IKNN_PSLFW, we conduct experimental study on 20 benchmark datasets. The experimental results show that compared with the conventional KNN rule, some state-of-the-art prototype selection methods and other machine learning algorithms, the proposed IKNN_PSLFW achieves promising classification performance with high time efficiency.