Hybrid Euclidean-and-Riemannian Metric Learning for Image Set Classification

Abstract

AbstractWe propose a novel hybrid metric learning approach to combine multiple heterogenous statistics for robust image set classification. Specifically, we represent each set with multiple statistics – mean, covariance matrix and Gaussian distribution, which generally complement each other for set modeling. However, it is not trivial to fuse them since the mean vector with \(d\)-dimension often lies in Euclidean space \(\mathbb {R}^d\), whereas the covariance matrix typically resides on Riemannian manifold \(Sym^+_{d}\). Besides, according to information geometry, the space of Gaussian distribution can be embedded into another Riemannian manifold \(Sym^+_{d+1}\). To fuse these statistics from heterogeneous spaces, we propose a Hybrid Euclidean-and-Riemannian Metric Learning (HERML) method to exploit both Euclidean and Riemannian metrics for embedding their original spaces into high dimensional Hilbert spaces and then jointly learn hybrid metrics with discriminant constraint. The proposed method is evaluated on two tasks: set-based object categorization and video-based face recognition. Extensive experimental results demonstrate that our method has a clear superiority over the state-of-the-art methods.KeywordsGaussian Mixture ModelReproduce Kernel Hilbert SpaceSymmetric Positive Definite MatriceSymmetric Positive DefiniteHeterogeneous SpaceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.