Automatic face image annotation based on a single template with constrained warping deformation

Abstract

In this study, an automatic face image annotation method is proposed by aligning faces with different expressions to an annotated neutral face. This work is useful in reducing tedious manual work for labelling image data in large databases. However, it is challenging because of the appearance variations caused by non‐rigid face deformations under various expressions. Unlike some conventional approaches acquiring sufficient image templates to model the query appearance, only a single given template is necessary for the proposed method. The authors address the problem through dense image alignment. Specifically, image warping in the alignment process is constrained by prior knowledge about facial shape deformation. The proposed method is independent of the appearance model, and is available for unseen faces. In addition, to initialise warping parameters, the authors present a robust patch‐based estimation method. Context information for feature points is carefully modelled to propagate the searching path for local patch matching. The face annotation experiments are performed on some large expressions, with noisy image qualities and in low image resolutions. Comparison results with conventional methods demonstrate the proposed method's superiority on both accuracy and robustness.