Discriminative geodesic Gaussian process latent variable model for structure preserving dimension reduction in clustering and classification problems

Abstract

Dimension reduction is a common approach for analyzing complex high-dimensional data and allows efficient implementation of classification and decision algorithms. Gaussian process latent variable model (GPLVM) is a widely applicable dimension reduction method which represents latent space without considering the class labels. Preserving the structure and topology of data are key factors that influence the performance of dimensionality reduction models. A conventional measure which reflects the topological structure of data points is geodesic distance. In this study, we propose an enriched GPLVM mapping between low-dimensional space and high-dimensional data. One of the contributions of the proposed approach is to calculate geodesic distance under the influence of class labels and introducing an improved GPLVM kernel using the distance. Also, the objective function of the model is reformulated to consider the trade-off between class separation and structure preservation which improves discrimination power and compactness of data. The efficiency of the proposed approach is compared with other dimension reduction techniques such as the kernel principal component analysis (KPCA), locally linear embedding (LLE), Laplacian eigenmaps and also discriminative and supervised extensions of standard GPLVM. Based on the experiments, it is suggested that the proposed model has a higher capacity for accurate classification and clustering of data as compared with the mentioned approaches.