Multi-objective Approach for Semi-Supervised Discriminant Analysis with Relative Distance

Abstract

Data such as videos, genetic information, etc. from real-world applications reside in a high dimensional space. Before performing classification, it is required to project the data from the high dimensional space to a lower dimensional space without losing too much information. Linear discriminant analysis (LDA) is one of the most widely used methods for dimensionality reduction, that maximizes the ratio of the between-class scatter and total data scatter in the projected space using the labeled information. However, in the real world scenario, labeled information is hardly ever available in large quantities, but an abundant amount of unlabeled data is available. In this paper, we propose a Semi-Supervised Discriminant Analysis method called SSDARD, which considers the unlabeled information in the form of a k-NN graph. Different from the existing semi-supervised dimensionality reduction algorithms, our algorithm is more consistent in propagating the label information from labeled data to unlabeled data due to the use of relative distance function instead of normal Euclidean distance function to generate the k-NN graph. To find an appropriate relative distance function, we use pairwise constraints generated from labeled data and satisfy them using Bregman projection. Since the projection is not orthogonal, we require an appropriate subset of constraints. In order to select such subset of constraints, we have further developed a framework called MO-SSDARD, which uses an evolutionary algorithm while optimizing various cluster validity indices simultaneously. The experimental results on various datasets show that our proposed method is superior than various methods concerning various validity indices.