Illumination invariant recognition and 3D reconstruction of faces using desktop optics

Abstract

We propose illumination invariant face recognition and 3D face reconstruction using desktop optics. The computer screen is used as a programmable extended light source to illuminate the face from different directions and acquire images. Features are extracted from these images and projected to multiple linear subspaces in an effort to preserve unique features rather than the most varying ones. Experiments were performed using our database of 4347 images (106 subjects), the extended Yale B and CMU-PIE databases and better results were achieved compared to the existing state-of-the-art. We also propose an efficient algorithm for reconstructing the 3D face models from three images under arbitrary illumination. The subspace coefficients of training faces are used as input patterns to train multiple Support Vector Machines (SVM) where the output labels are the subspace parameters of ground truth 3D face models. Support Vector Regression is used to learn multiple functions that map the input coefficients to the parameters of the 3D face. During testing, three images of an unknown/novel face under arbitrary illumination are used to estimate its 3D model. Quantitative results are presented using our database of 106 subjects and qualitative results are presented on the Yale B database.