Large Margin Weighted k -Nearest Neighbors Label Distribution Learning for Classification.

Abstract

Label distribution learning (LDL) helps solve label ambiguity and has found wide applications. However, it may suffer from the challenge of objective inconsistency when adopted to classification problems because the learning objective of LDL is inconsistent with that of classification. Some LDL algorithms have been proposed to solve this issue, but they presume that label distribution can be represented by the maximum entropy model, which may not hold in many real-world problems. In this article, we design two novel LDL methods based on the k -nearest neighbors ( k NNs) approach without assuming any form of label distribution. First, we propose the large margin weighted k NN LDL (LW- k NNLDL). It learns a weight vector for the k NN algorithm to learn label distribution and implement a large margin to address the objective inconsistency. Second, we put forward the large margin distance-weighted k NN LDL (LD k NN-LDL) that learns distance-dependent weight vectors to consider the difference in the neighborhoods of different instances. Theoretical results show that our methods can learn any general-form label distribution. Moreover, extensive experimental studies validate that our methods significantly outperform state-of-the-art LDL approaches.