Frontal face synthesizing according to multiple non-frontal inputs and its application in face recognition

Abstract

A multi-to-one frontal view face synthesizing strategy, and how it could be utilized to improve traditional face recognition algorithms on pose variant problems, is introduced in this paper. The word multi-to-one means more than one input source images and one output synthetic image, and this is an information selection procedure. Through picking up the gray intensity most similar with that of frontal view face from multiple non-frontal input images, proposed algorithm tries to simulate real natural pose variance of human face. The similarity is evaluated according to the magnitude of non-rigid bending deformation involved during synthesizing, the underlying observation of which is, the more the bending deformation are utilized, the less natural the synthesized image looks like. The specific approach is realized based on Moving Least Squares (MLS). Besides synthesizing frontal faces, our Minimum Bending Synthesizing (MBS) strategy could also be utilized to unify the poses of both gallery and probe images, and hence effectively reduce the influence of variant pose to 2D face recognition. From experiments on the CMU PIE and FERET databases, it could be observed that the frontal view faces synthesized by MBS could effectively approximate the real ground truth frontal faces, and MBS could greatly improve the performance of classic face recognition algorithms, PCA and LDA, on pose variant problems. Apart from specific algorithms, the idea of synthesizing frontal face according to more than one input images, is much valuable as well.