The Role of Physiological Responses in a VR-Based Sound Localization Task

Adrielle N Moraes,Andrew Hines,Niall Murray,Ronan Flynn

doi:10.1109/access.2021.3108446

Abstract

Virtual reality (VR) has recently emerged as a platform that can be employed in the context of e-health applications. Even though the majority of VR-based applications focus on visual stimuli as the main content, audio also plays a very important role. If someone has an issue with auditory processing, the comprehension of auditory information is compromised. This condition reflects negatively on one’s quality of life, given the impact of audio perception on one’s ability to communicate effectively or to differentiate and ignore “noise”. This work aims to design a VR application that can be used to: (a) optimize multimodal VR experiences based on user trials (b) extract user data continuously throughout the experiment (c) evaluate a user’s auditory processing ability. To accomplish this goal, participants are required to localise a sound source in space in the presence of multiple listening conditions with simple and complex sound stimuli configurations. Data collected from users consists of physiological and objective metrics. The results of this study highlight the relationship between user behaviour (head movement, fixation points) and performance in the sound localisation task. This information can be used to design future applications with the purpose of training one’s auditory localisation ability. In addition, the evaluation compared the impact of using two interaction methods to perform this task: using a pointer or eye gaze to indicate the location of the target source. The findings from this study show statistically significant differences in terms of physiological response when subjects are exposed to different interaction methods, with greater immersion and performance for the pointer group.

Highlights

Virtual reality (VR) is a tool to present computer-generated content to its users
VR experiences are multimodal by their nature, and are dependent on a virtual environment, which contains all of the content and objects to be displayed
Since this study investigated the effects of gaze data in a sound localisation task, the headset lenses were adjusted for each participant, ensuring that both visual content and data collection were performed correctly

Summary

Introduction

Virtual reality (VR) is a tool to present computer-generated content to its users. It aims to be interactive, providing a sense of being immersed in a virtual environment [1]. VR experiences are multimodal by their nature, and are dependent on a virtual environment, which contains all of the content and objects to be displayed. The perception of these environments objects is given via multiple sensory pathways G. visual, auditory, sensory), as they share the same properties as real objects such as shape, colour, texture and temperature [4] The perception of these environments objects is given via multiple sensory pathways (e. g. visual, auditory, sensory), as they share the same properties as real objects such as shape, colour, texture and temperature [4]

Objectives

Methods

Results