Interactive Audio Augmented Reality in Participatory Performance

Anna N Nagele,Valentin Bauer,Chris Pike,Henry Cooke,Tim Cowlishaw,Chris Baume,Patrick G T Healey,Joshua D Reiss

doi:10.3389/frvir.2020.610320

Abstract

Interactive Audio Augmented Reality (AAR) facilitates collaborative storytelling and human interaction in participatory performance. Spatial audio enhances the auditory environment and supports real-time control of media content and the experience. Nevertheless, AAR applied to interactive performance practices remains under-explored. This study examines how audio human-computer interaction can prompt and support actions, and how AAR can contribute to developing new kinds of interactions in participatory performance.This study investigates an AAR participatory performance based on the theater and performance practice by theater maker Augusto Boal. It draws from aspects of multi-player audio-only games and interactive storytelling. A user experience study of the performance shows that people are engaged with interactive content and interact and navigate within the spatial audio content using their whole body. Asymmetric audio cues, playing distinctive content for each participant, prompt verbal and non-verbal communication. The performative aspect was well-received and participants took on roles and responsibilities within their group during the experience.

Highlights

The emergence of affordable consumer technologies for interactive 3D audio listening and recording has facilitated the delivery of Audio Augmented Reality (AAR) experiences
This paper investigates the potential of AAR for multiplayer embodied interaction through participatory performance
To answer research question 1) (“How can audio humancomputer interaction prompt and support actions in AAR experiences?”) this study explores the perception of sound location considering the technology used, spatial sounds prompting movements, the participants’ engagement with spatial audio and their interest in it

Summary

Introduction

The emergence of affordable consumer technologies for interactive 3D audio listening and recording has facilitated the delivery of Audio Augmented Reality (AAR) experiences. The main difference of AAR toward traditional headphones is that it does not create a sound barrier between private and public auditory spheres. AAR technology should be acoustically transparent (Härmä et al, 2004), i.e., the listener should perceive the environment as if they had no headphones, without being able to determine which sound sources are real and which ones are not. This feature remains one of the main technical challenges in the field (Engel and Picinali, 2017)

Methods

Discussion

Conclusion