Human scanpath prediction based on deep convolutional saccadic model

Abstract

Human scanpath represents the sequence of human eye fixations, revealing the dynamic process of saccadic eye movement when a natural scene is freely viewed by humans. It is valuable to have an in-depth understanding of the dynamic visual attention and visual search behavior. In this paper, a deep convolutional saccadic model (DCSM) is proposed to predict human scanpath. The model simultaneously predicts the foveal saliency maps and fixation durations with considering on modeling the inhibition of return, which is a well recognized physiological mechanism to mimic human saccadic behavior. Both the foveal saliency and fixation durations are predicted by convolutional neural networks, which associate the inhibition of return with image content from spatial and temporal aspects. With the proposed DCSM, fixations of a scanpath are sequentially predicted with only a single image as input. Our method is capable of handling the challenges of temporal dependency and spatial association with image content. Experimental results on MIT1003 and FIGRIM datasets demonstrate the effectiveness of our proposed method when compared with state-of-the-art methods.