POSEidon: Face-from-Depth for Driver Pose Estimation

Guido Borghi,Roberto Vezzani,Marco Venturelli,Rita Cucchiara

doi:10.1109/cvpr.2017.583

Abstract

Fast and accurate upper-body and head pose estimation is a key task for automatic monitoring of driver attention, a challenging context characterized by severe illumination changes, occlusions and extreme poses. In this work, we present a new deep learning framework for head localization and pose estimation on depth images. The core of the proposal is a regressive neural network, called POSEidon, which is composed of three independent convolutional nets followed by a fusion layer, specially conceived for understanding the pose by depth. In addition, to recover the intrinsic value of face appearance for understanding head position and orientation, we propose a new Face-from-Depth model for learning image faces from depth. Results in face reconstruction are qualitatively impressive. We test the proposed framework on two public datasets, namely Biwi Kinect Head Pose and ICT-3DHP, and on Pandora, a new challenging dataset mainly inspired by the automotive setup. Results show that our method overcomes all recent state-of-art works, running in real time at more than 30 frames per second.

Highlights

Nowadays, we are witnessing a revolution in the automotive field, where ICT technologies are becoming sometimes more important than the engine itself
Standard techniques based on intensity images are not always applicable, due to the poor illumination conditions during the night and the continuous illumination changes during the day
Computer vision solutions based on illumination-insensitive data sources such as thermal [51] or depth [35] cameras are emerging

Summary

Introduction

We are witnessing a revolution in the automotive field, where ICT technologies are becoming sometimes more important than the engine itself. Starting from head localization, the ultimate goal of the framework is the estimation of the head and shoulder pose, measured as pitch, roll and yaw rotation angles. To this aim, a new triple regressive Convolutional Neural Network architecture, called POSEidon, is proposed, that combines depth, motion images and appearance. One of the most innovative contribution is a Face-fromDepth network, that is able to reconstruct gray-level faces directly from head depth images This solution derives from the awareness that intensity face images are very useful to detect head pose [1, 17]: without having intensity data we would like to have similar benefits. Gray-level faces extracted by depth images have a qualitatively impressive sim-

Objectives

Results

Conclusion