Free-Angle 3D Head Pose Tracking Based on Online Shape Acquisition

Abstract

This paper introduces a video-based real-time 3D head pose tracking method for analyzing human interactions. Most of the existing image-based face tracking approaches, such as active appearance model, generally have a limit in the measurable range of head orientation angles and can only track frontal or near-frontal face views owing to the use of frontal face models. To overcome this limitation, we propose an online head shape and pose acquisition approach. This approach incrementally acquires the 3D surface shape of an individual's head as a single rigid object. The approach includes not only the frontal face surface region but also side regions of the face, ears, hair, and some extraneous objects like glasses if present. An individual's 3D head shape is a collection of a multitude of head appearances taken from various viewpoints. Our approach suffers no limitation regarding the head orientation angle and position. We formalized this approach as a combination of two real-time processes: an incremental head 3D surface shape acquisition and its pose tracking by using six degrees-of-freedom positions and orientations. We also built a real-time demonstration system by using an RGB-D camera and a modern GPGPU. Experimental results show our approach can track six DoF head poses in various angles and position ranges.