Perceptual uncertainty and action consequences independently affect hand movements in a virtual environment

Constanze Hesse,Julie M. Harris,Anna Nowakowska,Martin Giesel

doi:10.1038/s41598-020-78378-z

Constanze Hesse, Julie M. Harris + Show 2 more

Open Access

https://doi.org/10.1038/s41598-020-78378-z

Copy DOI

Abstract

When we use virtual and augmented reality (VR/AR) environments to investigate behaviour or train motor skills, we expect that the insights or skills acquired in VR/AR transfer to real-world settings. Motor behaviour is strongly influenced by perceptual uncertainty and the expected consequences of actions. VR/AR differ in both of these aspects from natural environments. Perceptual information in VR/AR is less reliable than in natural environments, and the knowledge of acting in a virtual environment might modulate our expectations of action consequences. Using mirror reflections to create a virtual environment free of perceptual artefacts, we show that hand movements in an obstacle avoidance task systematically differed between real and virtual obstacles and that these behavioural differences occurred independent of the quality of the available perceptual information. This suggests that even when perceptual correspondence between natural and virtual environments is achieved, action correspondence does not necessarily follow due to the disparity in the expected consequences of actions in the two environments.

Highlights

When we use virtual and augmented reality (VR/AR) environments to investigate behaviour or train motor skills, we expect that the insights or skills acquired in VR/AR transfer to real-world settings
If we want actions in VR/AR to be an adequate model for actions in natural environments, we must ensure that the same sources of information that influence the planning and execution of movements in natural environments are available in VR/AR4
We will first discuss how these findings advance our knowledge about sensorimotor control and sensorimotor decision making, and elaborate on what they imply for the use of VR/AR as a tool for investigating or training motor behaviour

Summary

Introduction

When we use virtual and augmented reality (VR/AR) environments to investigate behaviour or train motor skills, we expect that the insights or skills acquired in VR/AR transfer to real-world settings. Using mirror reflections to create a virtual environment free of perceptual artefacts, we show that hand movements in an obstacle avoidance task systematically differed between real and virtual obstacles and that these behavioural differences occurred independent of the quality of the available perceptual information This suggests that even when perceptual correspondence between natural and virtual environments is achieved, action correspondence does not necessarily follow due to the disparity in the expected consequences of actions in the two environments. Research on sensorimotor decision making has shown that motor behaviour is strongly influenced by perceptual uncertainty and the expected consequences of actions, i.e., gains and/or costs (for reviews s ee[5,6]) We study these two factors, perceptual uncertainty and expected consequences of actions, jointly, and directly measure the effects of acting in virtual environments on visually-guided hand movements. Using mirror reflections, rather than computer-generated VR/AR environments (that often contain perceptual artefacts), we created a virtual environment that allowed us to vary the costs of potential movement errors without changing the perceptual information or the appearance of the obstacles[19] and, to measure the isolated effects of action consequences on avoidance behaviour

Methods

Results

Conclusion