Sensing, predicting, and utilizing human visual attention

Abstract

In this talk, I will describe our recent works on human visual attention from three aspects: sensing, predicting, and utilizing visual attention. Over the last two decades, the concept of visual saliency and its computational models have attracted a lot of interest, inspired by the seminal work by Koch and Ullman on a computational model of visual saliency. Visual saliency models predict our eye fixations driven by our vision system's bottom-up control triggered by visual stimuli, and it has been shown experimentally that a visual saliency map computed by a visual saliency model is highly correlated with an actual distribution of our fixation points. Based on this observation, we introduce a method for estimating gaze directions using visual saliency maps without explicit personal calibration. The key idea is to use the saliency maps of the video frames that a person is looking at as the probability distributions of the gaze points so that we can avoid cumbersome calibration procedures asking a user to fixate calibration targets. I will explain the details of our method and experimental results. I will also talk about our recent attempt to develop a new computational model of visual saliency for better accuracy in predicting gaze fixation points. Unlike the existing visual saliency models, our model elaborates the fact that the characteristics of human eyes vary significantly within the visual field, e.g., fovea and peripheral vision. Experiments using human fixation data in a wide field of view setting demonstrate that our visual saliency model achieves higher accuracy than a current state-of-the-art model. Lastly, we briefly talk about several applications of eye movements for activity recognition and image analysis.