LDDMM-Face: Large deformation diffeomorphic metric learning for cross-annotation face alignment

Abstract

We propose an innovative, flexible, and consistent cross-annotation face alignment framework, LDDMM-Face, the key contribution of which is a deformation layer that naturally embeds facial geometry in a diffeomorphic way. This enables and solves cross-annotation face alignment tasks that were impossible in the existing works. Instead of predicting facial landmarks via a heatmap or coordinate regression, we formulate the face alignment task in a diffeomorphic registration manner and predict momenta that uniquely parameterize the deformation between the initial boundary and true boundary. We then perform large deformation diffeomorphic metric mapping (LDDMM) simultaneously for curve and landmark to localize the facial landmarks. The novel embedding of LDDMM into a deep network allows LDDMM-Face to consistently annotate facial landmarks without ambiguity and flexibly handle various annotation schemes, and can even predict dense annotations from sparse ones. To the best of our knowledge, this is the first study to leverage learning-based diffeomorphic mapping for face alignment. Our method can be easily integrated into various face alignment networks. We extensively evaluate LDDMM-Face on five benchmark datasets: 300 W, WFLW, HELEN, COFW-68,and AFLW. LDDMM-Face distinguishes itself with outstanding performance when dealing with within-dataset cross-annotation learning (sparse-to-dense) and cross-dataset learning (different training and testing datasets). In addition, LDDMM-Face shows promising results on the most challenging task of cross-dataset cross-annotation learning (different training and testing datasets with different annotations). Our codes are available at https://github.com/ForTest66656/ForTest.