Challenges in head pose estimation of drivers in naturalistic recordings using existing tools

Abstract

Head pose is an important feature to understand the driver's behavior and their level of attention. While current head pose estimation (HPE) algorithms are suitable for many applications under controlled conditions, the performance drops in driving environments where images commonly have varying illumination, occlusions, and extreme head rotations. It is important to understand the limitations of current HPE algorithms to create computer vision solutions that target in-vehicle applications. This paper analyzes the HPE algorithms OpenFace, IntraFace and ZFace, with images recorded under natural driving environment where the goal is to find consistent conditions that affect the performance of current HPE systems. The key feature of the recordings is the use of a headband with AprilTags, which are used to estimate reference head pose angles. We study the effect of different factors including head pose angles, illumination in the frames and occlusions due to glasses. We identify the range of yaw and pitch rotations for which these HPE algorithms provide reliable estimations. While the HPE algorithms work reasonably well for normal/rimless glasses, occlusion due to thicker glasses is a major problem. We identified frames where all the HPE algorithms failed to provide an estimate. We are releasing the data to serve as a benchmark for future HPE algorithms in naturalistic driving conditions.