3-D Face Alignment Using A Convolutional Point-Set Representation

Abstract

We present a robust method for estimating the facial pose and shape parameters with a Projected Normalized Coordinate Code (PNCC), a derived image where each pixel is annotated with its corresponding vertex position in a generic 3D model. The method relies on a Convolutional Point-set Representation (CPR), a carefully designed feature map that extracts useful information encoded in pairs or triplets of vertices of PNCC. To estimate the shape and pose parameters, different CPRs are designed to represent mutually independent pose and shape parameters. The CPRs are then fed into multiple convolutional networks (with recurrent layers) in a sequential manner based on the transformation order of weak-perspective projection model to predict the target parameters. Reprojection errors are sampled along a predicted direction and the parameters are updated based on the pattern of reprojection errors. This technique boosts the model’s capability to find a local minimum under challenging scenarios. It is also demonstrated that CPRs obtained from different sources can be merged and contribute to outperforming the current state-of-the-art solutions for 3D face alignment. Experiments indicate that the proposed CPRFA (CPR-based Face Alignment) significantly improves 3D alignment accuracy. Compared to 3DDFA, the proposed method reduces the normalized mean error on AFLW2000 and AFLW database by 25.3% and 20.2% respectively.