From one to many: Pose-Aware Metric Learning for single-sample face recognition

Abstract

Pose and illumination variations are very challenging for face recognition with a single sample per person (SSPP). In this paper, we address this issue by a Pose-Aware Metric Learning (PAML) approach. Our primary idea is “from one to many”: Synthesizing many images of sufficient pose and illumination variability from the single training image, based on which metric learning approach is applied to reduce these “synthesized” variations at each quantified pose. For this purpose, given a single frontal training image, a multi-depth generic elastic model and an extended generic elastic model are developed to synthesize facial images of the target pose with varying 3D shape (depth) and illumination variations respectively. To reduce these “synthesized” variability, Pose-Aware Metric spaces are separately learnt by linear regression analysis at each quantized pose, and pose-invariant recognition is performed in the corresponding metric space. By preserving the detailed texture and reducing the shape variability, the PAML method achieves an 100% accuracy on the Multi-PIE database under the test setting across poses, which is significantly better than the traditional methods that use a large generic image ensemble to learn the cross-pose transformations. On the more challenging setting across both poses and illuminations, PAML outperforms the recent deep learning approaches by over 10% accuracy.