The Effect of Audiovisual Spatial Design on User Experience of Bare-Hand Interaction in VR

Abstract

This study investigated whether the audio-visual spatial design could improve and how it improved the user experience of bare-hand interaction (BHI) in virtual reality (VR), by adjusting spatial audio accordingly (no sound, audiovisual spatial congruency, audio-visual spatial in congruency). We asked 32 participants to complete the grasping task in two interaction scenarios with three audio-visual spatial conditions. We evaluated the user experience in terms of both intrinsic cognitive load and usability through four metrics: subjective cognitive load scores, HbO2 data in the corresponding brain regions, task time, and user satisfaction. The results showed that in near range interaction, although there was no significant difference between audio-visual spatial in congruency and congruency in terms of cognitive load and usability, they both significantly improved the user experience compared to the no sound group. In distant range interaction, audio-visual spatial in congruency significantly reduced the cognitive load and increased usability, thus improving the user experience.