Disentangled representation transformer network for 3D face reconstruction and robust dense alignment

Abstract

AbstractThis paper proposes a disentangled representation transformer network (DRTN) for 3D dense face alignment and reconstruction. Unlike traditional 3DMM-based approaches, the target parameters, such as shape, expression, and pose, are individually estimated without considering their direct influences on one another and then jointly optimized. Hence, DRTN aims to enhance the representation of facial attributes in a semantic sense by learning the correlation of different 3D facial attribute parameters. To achieve this, we present a novel strategy to design disentangled 3D face attribute representation, which decomposes the given facial attributes into identity, expression, and poses. Specifically, the 3D face parameter estimation in the regression network depends on the correlation of other face attribute parameters rather than being independent. The branching of the identity component aims to reinforce learning the expression and pose attributes by preserving the overall face geometry structure and identity. Accordingly, the expression and pose parts of the branch preserve the consistency of expression and pose attributes, respectively. Moreover, DRTN helps refine the reconstruction and alignment of facial details in large poses, mainly by coupling other facial attribute parameters. Extensive qualitative and quantitative experimental results on widely evaluated benchmarking datasets demonstrate that our approach achieves competitive performance compared to state-of-the-art methods.