Human gaze prediction for 3D light field display based on multi-attention fusion network

Abstract

Existing methods for simulating human visual attention primarily focus on 2D displays and limited research has been conducted on predicting visual attention in three-dimensional (3D) light field content. 3D light field displays provide a heightened sense of stereoscopic realism to viewers. To ensure that the content of the 3D light field display appears more consistent with human visual characteristics, we proposed a novel method for predicting human eye fixation in 3D light field display images. Firstly, we collected real eye movement data and utilized it to create an eye movement dataset based on 3D light field display images. This solves the problem of missing datasets in the field of human gaze based on three-dimensional light field images. Then, we proposed a convolutional neural network model with multiple inputs and outputs, integrating attention modules. This model was trained and used to predict eye fixation within the constructed eye movement dataset. A correlation exists between predicted human gaze of multiple distinct views of same light field image. Finally, we predicted the human gaze area of light field multi-view images based on our model. Experimental results demonstrate that our model accurately predicts human gaze regions across different views of a 3D light field image. The human gaze predicted by the model on each view is basically consistent and relatively accurate. By leveraging proposed method, we can effectively anticipate where viewers will focus their attention on the 3D light field display, which is beneficial for targeted improvement of 3D light field display content.