Deep Fusion for 3D Gaze Estimation From Natural Face Images Using Multi-Stream CNNs

Abid Ali,Yong-Guk Kim

doi:10.1109/access.2020.2986815

Abstract

Over the last few decades, eye gaze estimation techniques have been thoroughly investigated by many researchers. However, predicting a 3D gaze from a 2D natural image remains challenging because it has to deal with several issues such as diverse head positions, face shape transformation, illumination variations, and subject individuality. Many previous studies employ convolutional neural networks (CNNs) for this task, and yet the accuracy needs improvement for its practical use. In this paper, we propose a 3D gaze estimation framework based on the data science perspective: First, a novel neural network architecture is designed to exploit every possible visual attribute such as the states of both eyes and the head position, including several augmentations; secondly, the data fusion method is utilized by incorporating multiple gaze datasets. Extensive experiments were carried out using two standard eye gaze datasets, including comparative analysis. The experimental results suggest that our method outperforms state-of-the-art with 2.8 degrees for MPIIGaze and 3.05 degrees for EYEDIAP dataset, respectively, indicating that it has a potential for real applications.

Highlights

Eye movement and gaze estimation are important in terms of visual and cognitive processing [1]
Estimating gaze direction accurately from an image acquired with a mobile camera typically under the unstable illumination condition is not an easy task, given that the traditional way of estimating a human gaze was to gear up a massive eye movement setup, which was inconvenient and expensive
Detecting a human gaze during the interaction between people can play an essential role in social survival because one can understand what the other person intends during a conversation

Summary

Introduction

Eye movement and gaze estimation are important in terms of visual and cognitive processing [1]. Eye movements have been widely studied for human visual attention [2], [3], emotion analysis [3] and for behavioral disorder identification [2], [4]. Gaze estimation has been studied thoroughly in the computer vision area because it has a wide range of applications in human-computer interaction [5], psychology [1], [6], [7], disability studies [8], navigation and detecting driver’s behavior [9], surgical robots [10] and marketing research [11]–[15].

Methods

Findings

Discussion

Conclusion