Glimpse: A Gaze-Based Measure of Temporal Salience.

V Javier Traver,Luis A Leiva,Judith Zorío

doi:10.3390/s21093099

V Javier Traver, Luis A Leiva + Show 1 more

Open Access

https://doi.org/10.3390/s21093099

Copy DOI

Abstract

Temporal salience considers how visual attention varies over time. Although visual salience has been widely studied from a spatial perspective, its temporal dimension has been mostly ignored, despite arguably being of utmost importance to understand the temporal evolution of attention on dynamic contents. To address this gap, we proposed Glimpse, a novel measure to compute temporal salience based on the observer-spatio-temporal consistency of raw gaze data. The measure is conceptually simple, training free, and provides a semantically meaningful quantification of visual attention over time. As an extension, we explored scoring algorithms to estimate temporal salience from spatial salience maps predicted with existing computational models. However, these approaches generally fall short when compared with our proposed gaze-based measure. Glimpse could serve as the basis for several downstream tasks such as segmentation or summarization of videos. Glimpse’s software and data are publicly available.

Highlights

Measure of Temporal Salience.Visual salience refers to the ability of an object, or part of a scene, to attract our visual attention
visual interestingness estimation method [48] (VisInt) produced a flat signal in v36, which rightfully corresponded to the homogeneous contents of that video, but it did so in v22, missing the subtle image changes corresponding to people moving in the hall (Figure 1) and that G LIMPSE
Since G LIMPSE provides a consistent and reliable reference of temporal salience, we investigated whether temporal salience can be alternatively estimated from spatial salience maps predicted by computational models

Summary

Introduction

Visual salience (or saliency) refers to the ability of an object, or part of a scene, to attract our visual attention. The biological basis for this phenomenon is well known [1]: salience emerges in parallel processing of retinal input at lower levels in the visual cortex [2]. Concepts other than salience, such as surprise [3], have been found to explain human gaze in dynamic natural scenes. While the concept of spatial salience has been extensively investigated for static contents such as natural images [4,5] and graphic displays [6,7], the temporal salience of dynamic contents such as videos remains largely unexplored. Spatial salience predicts where attention is allocated in the image domain, whereas temporal salience predicts when attention happens and how it varies over time

Objectives

Results

Conclusion