Eye movements to absent objects during mental imagery and visual memory in immersive virtual reality

Sandra Chiquet,Corinna S Martarelli,Fred W Mast

doi:10.1007/s10055-020-00478-y

Sandra Chiquet, Corinna S Martarelli + Show 1 more

Open Access

https://doi.org/10.1007/s10055-020-00478-y

Copy DOI

Abstract

The role of eye movements in mental imagery and visual memory is typically investigated by presenting stimuli or scenes on a two-dimensional (2D) computer screen. When questioned about objects that had previously been presented on-screen, people gaze back to the location of the stimuli, even though those regions are blank during retrieval. It remains unclear whether this behavior is limited to a highly controlled experimental setting using 2D screens or whether it also occurs in a more naturalistic setting. The present study aims to overcome this shortcoming. Three-dimensional (3D) objects were presented along a circular path in an immersive virtual room. During retrieval, participants were given two tasks: to visualize the objects, which they had encoded before, and to evaluate a statement about visual details of the object. We observed longer fixation duration in the area, on which the object was previously displayed, when compared to other possible target locations. However, in 89% of the time, participants fixated none of the predefined areas. On the one hand, this shows that looking at nothing may be overestimated in 2D screen-based paradigm, on the other hand, the looking at nothing effect was still present in the 3D immersive virtual reality setting, and thus it extends external validity of previous findings. Eye movements during retrieval reinstate spatial information of previously inspected stimuli.

Highlights

We propose that immersive virtual reality (IVR) provides a relatively close approximation of human visual behavior in mental imagery and visual memory, since IVR matches some crucial cues of the real world
Participants spent on average 4.56 s (SE = 0.05) in the area, where the objects were presented, whereas they fixated the non-corresponding areas of interest (AOI) 0.43 s (SE = 0.02) and the empty space 0.73 s (SE = 0.03). This confirms proper encoding of the stimuli, which appeared for 6 s each, and it validates the measurement of eye movements in IVR by means of an head-mounted display (HMD) with an integrated eye-tracker
The primary goal of the current study was to examine the looking at nothing effect in an immersive virtual environment

Summary

Introduction

During retrieval people fixate on empty locations that have been associated with task-relevant stimuli during encoding (Altmann 2004; Bone et al 2019; Brandt and Stark 1997; Johansson et al 2012; Johansson et al 2006; Kumcu and Thompson 2018; Laeng et al 2014; Laeng and Teodorescu 2002; Richardson and Spivey 2000; Scholz et al 2018; Scholz et al 2016; Spivey and Geng 2001). The typical screen-based variant of looking at nothing leads to increased experimental control but is not necessarily a suitable model for mental imagery and visual memory processes in real life. Participants were exposed to isolated stimuli on a 2D computer screen, which makes them take on the role of external observers of stimuli on a computer screen. This is a key difference between the experimental situation in the laboratory and real life, where people are able to interactively control their view based on head motion and have the feeling of being inside the environment (Gorini et al 2011; Makransky et al 2019; Moreno and Mayer 2002).

Objectives

Methods

Results

Conclusion