Active vision in immersive, 360\xb0 real-world environments

Amanda J Haskins,Thomas L Botch,Caroline E Robertson,Jeff Mentch

doi:10.1038/s41598-020-71125-4

Amanda J Haskins, Thomas L Botch + Show 2 more

Open Access

https://doi.org/10.1038/s41598-020-71125-4

Copy DOI

Abstract

How do we construct a sense of place in a real-world environment? Real-world environments are actively explored via saccades, head turns, and body movements. Yet, little is known about how humans process real-world scene information during active viewing conditions. Here, we exploited recent developments in virtual reality (VR) and in-headset eye-tracking to test the impact of active vs. passive viewing conditions on gaze behavior while participants explored novel, real-world, 360° scenes. In one condition, participants actively explored 360° photospheres from a first-person perspective via self-directed motion (saccades and head turns). In another condition, photospheres were passively displayed to participants while they were head-restricted. We found that, relative to passive viewers, active viewers displayed increased attention to semantically meaningful scene regions, suggesting more exploratory, information-seeking gaze behavior. We also observed signatures of exploratory behavior in eye movements, such as quicker, more entropic fixations during active as compared with passive viewing conditions. These results show that active viewing influences every aspect of gaze behavior, from the way we move our eyes to what we choose to attend to. Moreover, these results offer key benchmark measurements of gaze behavior in 360°, naturalistic environments.

Highlights

Constructing a sense of place in a complex, dynamic environment is an active process
Our results demonstrate that active viewing conditions fundamentally impact the size of gaze shifts a viewer will routinely choose to make when exploring a real-world visual environment: people make faster fixations and larger gaze shifts, once more suggesting more exploratory gaze behavior
Our results provide novel insights into the process by which we actively construct representations of immersive, real-world visual environments

Summary

Introduction

Constructing a sense of place in a complex, dynamic environment is an active process. Diverse, real-world scenes were displayed with the same wide-angle field of view (100 DVA), and participants were exposed to comparable portions of the display over the course of the trial This paradigm enabled us to perform quantitative, in-depth comparisons of gaze behavior and attentional deployment as subjects encoded a diverse set of novel, real-world scenes during active vs passive exploration. Our central hypothesis was that active viewing conditions would increase a viewer’s exploratory, informationseeking behavior in a real-world scene We tested this by measuring the degree to which participants’ overt attention was dominantly predicted by the spatial distribution of scene features that are semantically informative (e.g., objects, faces, doors)[11,12], as compared with scene regions that are rich in salient visual features (e.g., luminance, contrast, color, and orientation)[13,14]. Previous studies have shown that these information sources compete for participants’ top-down vs. bottom-up attention as scene understanding unfolds, attention is predominantly predicted by the distribution of semantic information[12,15]

Methods

Results

Conclusion