Three-Dimensional Face Reconstruction from Uncalibrated Photographs: Application to Early Detection of Genetic Syndromes

Abstract

Facial analysis from photography supports the early identification of genetic syndromes, but clinically-acquired uncalibrated images suffer from image pose and illumination variability. Although 3D photography overcomes some of the challenges of 2D images, 3D scanners are not typically available. We present an optimization method for 3D face reconstruction from uncalibrated 2D photographs of the face using a novel statistical shape model of the infant face. First, our method creates an initial estimation of the camera pose for each 2D photograph using the average shape of the statistical model and a set of 2D facial landmarks. Second, it calculates the camera pose and the parameters of the statistical model by minimizing the distance between the projection of the estimated 3D face in the image plane of each camera and the observed 2D face geometry. Using the reconstructed 3D faces, we automatically extract a set of 3D geometric and appearance descriptors and we use them to train a classifier to identify facial dysmorphology associated with genetic syndromes. We evaluated our face reconstruction method on 3D photographs of 54 subjects (age range 0–3 years), and we obtained a point-to-surface error of 2.01 \( \pm \) 0.54%, which was a significant improvement over 2.98 \( \pm \) 0.64% using state-of-the-art methods (p < 0.001). Our classifier detected genetic syndromes from the reconstructed 3D faces from the 2D photographs with 100% sensitivity and 92.11% specificity.