Abstract
Abstract State-of-the-art 3D morphable model (3DMM) is used widely for 3D face reconstruction based on a single image. However, this method has a high computational cost, and hence, a simplified 3D morphable model (S3DMM) was proposed as an alternative. Unlike the original 3DMM, S3DMM uses only a sparse 3D facial shape, and therefore, it incurs a lower computational cost. However, this method is vulnerable to self-occlusion due to head rotation. Therefore, we propose a solution to the self-occlusion problem in S3DMM-based 3D face reconstruction. This research is novel compared with previous works, in the following three respects. First, self-occlusion of the input face is detected automatically by estimating the head pose using a cylindrical head model. Second, a 3D model fitting scheme is designed based on selected visible facial feature points, which facilitates 3D face reconstruction without any effect from self-occlusion. Third, the reconstruction performance is enhanced by using the estimated pose as the initial pose parameter during the 3D model fitting process. The experimental results showed that the self-occlusion detection had high accuracy and our proposed method delivered a noticeable improvement in the 3D face reconstruction performance compared with previous methods.
Highlights
3D face modeling originated with Parke’s pioneering studies [1,2], which aimed to generate realistic faces for computer animation
We found that an accurate initial pose parameter led to a performance improvement during 3D face reconstruction based on our proposed algorithm and reduced the number of iterations required for 3D model fitting
We analyzed the self-occlusion problem that occurs in simplified 3D morphable model (S3DMM)-based 3D face reconstruction and proposed a method for solving this problem
Summary
- No requirement for training data - Infeasible constraints (Lambertian reflectance model and known light source direction ). Simplified version of 3D morphable model[18,19] - Low computational complexity - Requirement of training data. - Low computational complexity - Requirement of training data - Self-occlusion problem. - Low computational complexity - Requirement of training data - Robust to self-occlusion original 3DMM generally uses a dense shape with thousands of vertices, whereas S3DMM uses a sparse shape with only dozens of vertices. 3n × 1 model shape vector S obtained using (1), ~s2d is a 2 × n matrix that is reshaped from the 2n × 1 input shape vector s2d , P is a 2 × 3 orthographic projection matrix, T~. The shape parameter β0 and translation parameter T0 are initialized to 0 and the input 2D FFPs s2d are aligned with the 2D mean shape obtained by projecting the 3D mean shape (s0) with a frontal pose onto the x– y plane. Alignment s2d is aligned with the 2D mean shape obtained by projecting the frontal 3D mean shape (s0) onto the x–y plane
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
