An Iterative Recovery Strategy based KNN Method to Edit the Data with Label Noise

Abstract

Learning label noise, one of the most challenging tasks in supervised machine learning, has become a hot research topic recently. The k nearest neighbors (kNN) classifiers are sensitive to label noise and are often used to detect and filter label noise. However, relying on a single parameter k and single prediction result of kNN classifier, current detecting methods are prone to incur over-cleaning problems, resulting in the under-fit of the classification models. In this paper, we propose an iterative recovery strategy-based kNN method (IRS-kNN) to edit the training set with label noise. IRS-kNN has three significant advantages. Firstly, instead of relying on a single parameter k, it leverages a finite set of neighbors to detect the candidate label noise to improve the stability of the detected label noise; Secondly, by cascaded recovering the labels of the training data set for further detection, more difficult-to-learn label noise can be detected, enhancing the precision of the detected label noise. Thirdly, the iterative detection results provide meaningful guidance to deal with the label noise in different measures, which contributes to recovering the distribution of the corrupted data set as if it was not noisy. Experimental results on 8 UCI benchmarks demonstrate the effectiveness of the proposed method.