Learning Accurate Head Pose for Consumer Technology From 3D Synthetic Data

Abstract

Accurate 3D head pose estimation from a 2D image frame is an essential component of modern consumer technology (CT). It enables a better determination of user attentiveness and engagement and can support immersive audio and AR experiences. While deep learning methods have improved the accuracy of head pose estimation models, these depend on the accurate annotation of training data. The acquisition of real-world head pose data with a large variation of yaw, pitch and roll is a very challenging task. Available head-pose datasets often have limitations in terms of the number of data samples, image resolution, annotation accuracy and sample diversity (gender, race, age). In this work, a rendering pipeline is proposed to generate pixel-perfect synthetic 2D headshot images from high-quality 3D facial models with accurate pose angle annotations. A diverse range of variations in age, race, and gender are provided. The resulting dataset includes more than 300k pairs of RGB images with the corresponding head pose annotations. For every hundred 3D models there are multiple variations in pose, illumination and background. The dataset is evaluated by training a state-of-the-art head pose estimation model and testing against the popular evaluation dataset BIWI. The results show training with purely synthetic data produced by the proposed methodology can achieve close to state-of-the-art results on the head pose estimation task and is better generalized for age, gender and racial diversity than solutions trained on ‘real-World’ datasets.