Abstract
The paper proposes a robust framework for 3D facial movement tracking in real time using a monocular camera. It is designed to estimate the 3D face pose and local facial animation such as eyelid movement and mouth movement. The framework firstly utilizes the Discriminative Shape Regression method to locate the facial feature points on the 2D image and fuses the 2D data with a 3D face model using Extended Kalman Filter to yield 3D facial movement information. An alternating optimizing strategy is adopted to fit to different persons automatically. Experiments show that the proposed framework could track the 3D facial movement across various poses and illumination conditions. Given the real face scale the framework could track the eyelid with an error of 1 mm and mouth with an error of 2 mm. The tracking result is reliable for expression analysis or mental state inference.
Highlights
The face image is widely used to discriminate and identify people, in lip reading and to understand one’s emotion and intentions based on the facial expressions [1,2,3]
The reason is that some facial feature point (FFP) will become self-occluded for large yaw rotation and the FFPs extraction error will increase
The precision of pitch tracking is not as good as roll and yaw, that is because the pitch parameter is coupled with other parameters, say ty
Summary
The face image is widely used to discriminate and identify people, in lip reading and to understand one’s emotion and intentions based on the facial expressions [1,2,3]. This paper aims to develop a 3D facial movement tracking framework for real time human computer interface applications such as expression recognition, intention prediction, mental state estimation, etc. In such contexts, the 3D facial movement includes: (a) rigid global head movement and (b) non-rigid facial muscle movement. Video cameras are more widely available on PCs and mobile devices than RGBD cameras, and video-based facial tracking remains a challenging problem
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
