Joint dimensionality reduction and metric learning for image set classification

Abstract

Compared with the traditional classification task based on a single image, an image set contains more complementary information, which is of great benefit to correctly classify a query subject. Thus, image set classification has attracted much attention from researchers. However, the main challenge is how to effectively represent an image set to fully exploit the latent discriminative feature. Unlike in previous works where an image set was represented by a single or a hybrid mode, in this paper, we propose a novel multi-model fusion method across the Euclidean space to the Riemannian manifold to jointly accomplish dimensionality reduction and metric learning. To achieve the goal of our framework, we first introduce three distance metric learning models, namely, Euclidean-Euclidean, Riemannian-Riemannian and Euclidean-Riemannian to better exploit the complementary information of an image set. Then, we aim to simultaneously learn two mappings performing dimensionality reduction and a metric matrix by integrating the two heterogeneous spaces (i.e., the Euclidean space and the Riemannian manifold space) into the common induced Mahalanobis space in which the within-class data sets are close and the between-class data sets are separated. This strategy can effectively handle the severe drawback of not considering the distance metric learning when performing dimensionality reduction in the existing set based methods. Furthermore, to learn a complete Mahalanobis metric, we adopt the L2,1 regularized metric matrix for optimal feature selection and classification. The results of extensive experiments on face recognition, object classification, gesture recognition and handwritten classification demonstrated well the effectiveness of the proposed method compared with other image set based algorithms.