Ordinal semi-supervised k-nearest neighbor algorithm for small training datasets

Abstract

The traditional k-nearest neighbor (k-NN) algorithms with sufficient training data points seem robust; however, problems, such as decision boundary shift and performance deterioration, occur when the training sets are small. In this paper, a novel algorithm named ordinal semi-supervised k-NN is proposed to handle the cases with small training sets. The method consists of two parts: instance ranking and semi-supervised learning. Using semi-supervised learning techniques, the performance of k-NN can be improved even when the training set is small because they enlarge the training set by including a few high confidence prediction instances. In addition, the performance could be improved further by using an ordinal test set rather than an arbitrary one. Utilizing instance ranking, those instances closer to class boundaries are predicted first, and they are more likely to be the high confidence instances. The semi-supervised learning, thus, benefits from combining with instance ranking. Results for four benchmark datasets show that in the cases with insufficient training data (training ratio≤1/2), the proposed method can greatly improve the classification accuracy and outperform the semi-supervised k-NN and the traditional k-NN methods.