Abstract

This paper presents a joint head pose and facial landmark regression method with input from depth images for realtime application. Our main contributions are: firstly, a joint optimization method to estimate head pose and facial landmarks, i.e., the pose regression result provides supervised initialization for cascaded facial landmark regression, while the regression result for the facial landmarks can also help to further refine the head pose at each stage. Secondly, we classify the head pose space into 9 sub-spaces, and then use a cascaded random forest with a global shape constraint for training facial landmarks in each specific space. This classification-guided method can effectively handle the problem of large pose changes and occlusion. Lastly, we have built a 3D face database containing 73 subjects, each with 14 expressions in various head poses. Experiments on challenging databases show our method achieves state-of-the-art performance on both head pose estimation and facial landmark regression.

Highlights

  • Estimation of human head pose and detection of facial landmarks such as eye corners, nose tip, mouth, and chin are of central importance to facial animation [1], expression analysis [2], and face recognition, etc

  • Manuscript received: 2017-01-16; accepted: 2017-03-08 the development of RGBD technology like Microsoft Kinect and Intel Realsense, head pose estimation and facial landmark detection methods based on depth data have attracted more and more attention due to the rich geometric information in depth images

  • Joint head pose and facial landmark regression can be helpful for mutual optimization and improving the accuracy of both tasks

Read more

Summary

Introduction

Estimation of human head pose and detection of facial landmarks such as eye corners, nose tip, mouth, and chin are of central importance to facial animation [1], expression analysis [2], and face recognition, etc These two problems have been separately studied for many years [3,4,5,6,7], with significant progress for images [8,9,10,11]. Unlike existing methods, which train one general model for a variety types of input, our proposed method divides the input data into 9 types according to head orientation, and trains a specific model for each type In this way, our classification-guided facial landmark regression method achieves more accurate results than existing methods, and it can handle some challenging cases including large pose variations and occlusion.

Head pose estimation
Facial landmark localization
Method
Face detection
Feature selection
Feature selection for pose regression
Feature selection for facial landmark regression
Training
Testing
Pose classification
Supervised initialization via head pose
Joint head pose and facial landmark regression
Experiments
ETHZ databases
4.95 InnMouthR
Findings
Conclusions
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call