Fusing deep convolutional network with SFM for 3D face reconstruction

Abstract

Listen

Translate article

3D face reconstruction has long been a Research Focus and application hotsopt, in the field of 3D reconstruction from 2D face image sequences, the reconstruction result is impeded by the problems of the occlusion and illumination or pose variations frequently. Different from 3D morphable model fitting algorithm, the reconstruction method based on face feature points puts forward higher requirements to the robustness and accuracy. This paper propose a reconstruction method of fusing deep convolutional network for facial feature points extraction and factorization for SFM(Structure from Motion), we investigated the way of solving accurate sparse face structure matrix by importing a modified matrix, and the possibility of improving the realistic of 3D face reconstruction result by registering the sparse face structure and the general 3D face model. To reconstruct final dense face model, we use thin plate spline interpolation. The reconstruction result based on our method proved to be efficient and robust.