A locally weighted KNN algorithm based on eigenvector of SVM

Abstract

K-Nearest Neighbours (KNN) is one of the fundamental classification methods in machine learning. The performance of KNN method is restricted by the number of neighbours k. It is obvious that the outliers appear when dealing with small data samples. In this paper, we propose a hybrid framework of the feature weighted support vector machine as well as locally weighted k-nearest neighbour (SLKNN) to overcome this problem. In our method, we first use support vector machine to calculate the eigenvector of feature of data, then apply this eigenvector into distance metric as the weight of the feature. Finally, the distance metric is used in locally weighted k-nearest neighbour. The experiments on UCI data sets show that the proposed SLKNN performs better than some KNN-based methods.