Learning the appearance of faces: a morphable model for the analysis and synthesis of images

Abstract

In my talk, I introduce a new technique for modeling textured 3D faces. 3D faces can either be generated automatically from one or more photographs, or modeled directly through an intuitive user interface. Users are assisted in two key problems of computer aided face modeling. First, new face images or new 3D face models can be registered automatically by computing dense one-to-one correspondence to an internal flexible face model. Second, the approach regulates the naturalness of modeled faces avoiding faces with an unlikely appearance. Starting from an example set of 3D face models, a morphable face model is learned from example faces by transforming the shape and texture of all prototypes into a vector space representation. New faces and expressions can be modeled by forming linear combinations of the prototypes. Shape and texture constraints derived from the statistics of our example faces are used to guide manual modeling or automated matching algorithms. We show 3D face reconstructions from single images and their applications for photo-realistic image manipulations. We also demonstrate face manipulations according to complex parameters such as gender, fullness of a face or its distinctiveness.