Classification by semi-supervised discriminative regularization

Abstract

Linear discriminant analysis (LDA) is a well-known dimensionality reduction method which can be easily extended for data classification. Traditional LDA aims to preserve the separability of different classes and the compactness of the same class in the output space by maximizing the between-class covariance and simultaneously minimizing the within-class covariance. However, the performance of LDA usually deteriorates when labeled information is insufficient. In order to resolve this problem, semi-supervised learning can be used, among which, manifold regularization (MR) provides an elegant framework to learn from labeled and unlabeled data. However, MR tends to misclassify data near the boundaries of different clusters during classification. In this paper, we propose a novel method, referred to as semi-supervised discriminative regularization (SSDR), to incorporate LDA and MR into a coherent framework for data classification, which exploits both label information and data distribution. Extensive experiments demonstrate the effectiveness of our proposed method in comparison with classical classification algorithms including SVM, LDA and MR.