Abstract
Occlusions are a common occurrence in unconstrained face images. Single image 3D reconstruction from such face images often suffers from corruption due to the pres-ence of occlusions. Furthermore, while a plurality of 3D reconstructions is plausible in the occluded regions, existing approaches are limited to generating only a single so-lution. To address both of these challenges, we present Diverse3DFace, which is specifically designed to simulta-neously generate a diverse and realistic set of 3D reconstructions from a single occluded face image. It comprises three components; a global+local shape fitting process, a graph neural network-based mesh VAE, and a determinan-tal point process based diversity-promoting iterative opti-mization procedure. Quantitative and qualitative comparisons of 3D reconstruction on occluded faces show that Di-verse3DFace can estimate 3D shapes that are consistent with the visible regions in the target image while exhibiting high, yet realistic, levels of diversity in the occluded regions. On face images occluded by masks, glasses, and other random objects, Diverse3DFace generates a distri-bution of 3D shapes having ~50% higher diversity on the occluded regions compared to the baselines. Moreover, our closest sample to the ground truth has ~40% lower MSE than the singular reconstructions by existing approaches. Code and data available at: https://github.com/human-analysis/diverse3dface
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have