A Complementary Dual-Branch Network for Appearance-Based Gaze Estimation From Low-Resolution Facial Image

Abstract

Estimating gaze from a low-resolution facial image is a challenging task. Most current networks for gaze estimation focus on using face images of adequate resolution. Their performance degrades when the image resolution decreases due to information loss. This work aims to explore more helpful face and gaze information in a novel way to alleviate the problem of information loss in the low-resolution gaze estimation task. Considering that all faces have a relatively fixed structure, it is feasible to reconstruct the residual information of face and gaze based on the solid constraint of the prior knowledge of face structure through learning an end-to-end mapping from pairs of low-and high-resolution images. This paper proposes a complementary dual-branch network (CDBN) to achieve this task. A fundamental branch is designed to extract features of the major structural information from low-resolution input. A residual branch is employed to reconstruct features containing the residual information as a supplement under the supervision of both the high-resolution image and gaze direction. These two features are then fused and processed for gaze estimation. Experimental results on three widely used datasets, MPIIFaceGaze, EYEDIAP, and RT-GENE, show that the proposed CDBN achieves more accurate gaze estimation from the low-resolution input image compared with the state-of-the-art methods.