Gaze estimation with semi-supervised eye landmark detection as an auxiliary task

Abstract

The changes in gaze are often reflected in the movements of eye landmarks, highlighting the relevance of eye landmark learning for accurate gaze estimation. To leverage eye landmarks, we propose a gaze estimation framework that incorporates eye landmark detection as an auxiliary task. However, obtaining eye landmark annotations for real-world gaze datasets is challenging. To address this, we exploit synthetic data, which provides precise eye landmark labels, by jointly training an eye landmark detector using labeled synthetic data and unlabeled real-world data in a semi-supervised manner. To reduce the influence of discrepancy between synthetic and real-world data, we improve the generalization ability of the landmark detector by performing a self-supervised learning strategy on a large scale of unlabeled real-world images. The proposed method outperforms other state-of-the-art gaze estimation methods on three gaze datasets, indicating the effectiveness of leveraging eye landmark detection as an auxiliary task to enhance gaze estimation performance.