Joint Super-Resolution and Head Pose Estimation for Extreme Low-Resolution Faces

Abstract

State-of-the-art deep learning-based Head Pose Estimation (HPE) techniques have reached spectacular performance on High-Resolution (HR) face images. However, they still fail to achieve expected performance on low-resolution images at large scales. This work presents an end-to-end HPE framework assisted by a Face Super-Resolution (FSR) algorithm. The proposed FSR model is specifically guided to enhance the HPE performance rather than considering FSR as an independent task. To this end, we utilized a Multi-Stage Generative Adversarial Network (MSGAN) which benefit from a pose-aware adversarial loss and head pose estimation feedback to generate super-resolved images that are properly aligned for HPE. Also, we propose a degradation strategy rather than simple down-sampling approach to mimic the diverse properties of real-world Low-Resolution (LR) images. We evaluate the performance of our proposed method on both synthetic and real-world LR datasets and show the superiority of our approach in both visual and HPE metrics on the AFLW2000, BIWI, and WiderFace Datasets.