Abstract
In this paper, we present a novel facial albedo and 3D shape recovery method with a local spherical harmonic illumination model. From a face in an image, the proposed method can produce a high-quality 3D shape and albedo using a novel parameterization of local illuminations. Because a facial shape is partially convex, a single spherical harmonics is generally used for the illumination of a face within a constrained illumination environment. However, when a facial image is captured in an unconstrained scene, the illumination is inappropriately estimated due to the presence of shadow and specular reflections. To address this issue, we propose a novel local spherical harmonic illumination model for representing the illumination of a face. Unlike the existing parameterization of local illumination, our local spherical harmonic illumination model utilizes a smooth weight function for the seamless representation of natural illumination. Therefore, the albedo and shape information in an image can be precisely estimated using the first-order spherical harmonics. For accurate estimation of albedo, we also utilize facial albedo statistics to prevent the estimated albedo from becoming biased toward input image. Furthermore, we developed an accurate and reliable 3D shape reconstruction method from a normal map based on tetrahedron-based deformation. Comparing to the Laplacian deformation based method, our method is applicable to any mesh regardless of its structure. Through rigorous experiments, we demonstrate that the proposed local spherical harmonic illumination model is effective in estimating the complex illumination and can recover a high-quality facial albedo and 3D shape.
Highlights
The analysis of facial geometry and appearance is a classical problem and its applications are related to many computer vision and graphics tasks such as face recognition [1], pose estimation [2]–[4], and facial animation [5]. 3D face reconstruction, which is the process of inferring the 3D geometry of a human face from 2D images, is the most very fundamental core that powers those applications
For an image with spatially varying appearances, this method can produce aliasing artifacts near the boundary of the grid. Motivated by these previous works, we address the problem of complex illumination for facial 3D shape and albedo recovery from a single image using a novel local spherical harmonics (SH) illumination model
The proposed local SH illumination model with an anisotropic weighting strategy allows facial albedo and 3D shape recovery to enhance the details of a face while preventing noisy deformation
Summary
The analysis of facial geometry and appearance is a classical problem and its applications are related to many computer vision and graphics tasks such as face recognition [1], pose estimation [2]–[4], and facial animation [5]. 3D face reconstruction, which is the process of inferring the 3D geometry of a human face from 2D images, is the most very fundamental core that powers those applications. Heo et al.: Local Spherical Harmonics for Facial Shape and Albedo Estimation reflectance model, most of studies have focused on Lambertian reflectance in conjunction with second-order spherical harmonics (SH) for illumination model [9], [17], [18] This approach is effective for images taken under controlled illumination conditions. For an image with spatially varying appearances, this method can produce aliasing artifacts near the boundary of the grid Motivated by these previous works, we address the problem of complex illumination for facial 3D shape and albedo recovery from a single image using a novel local SH illumination model. By using the first-order basis, we can linearize each problem while retaining the advantage of local SH illumination To this end, we develop a simple but robust method for reconstructing a 3D facial shape by using tetrahedron-based deformation technique with triangle normals.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
