A QoE and Visual Attention Evaluation on the Influence of Spatial Audio in 360 Videos

Abstract

Recently, there has been growing interest from academia and industry on the application of immersive technologies across a range of domains. Once such technology, 360° video, can be captured using an omnidirectional multi-camera arrangement. These 360° videos can then be rendered via Virtual Reality (VR) Head Mounted Displays (HMD). Viewers then have the freedom to look around the scene in any direction they wish. Whereas a body of work exists that focused on modeling visual attention (VA) in VR, little research has considered the impact of the audio modality on VA in VR. It is well accepted that audio has an important role in VR experiences. High quality spatial audio offers listeners the opportunity to experience sound in all directions. One such technique, Ambisonics or 3D audio, offers a complete 360° soundscape. This paper reports the results of an empirical study that looked at understanding how (if at all) spatial audio influences visual attention in 360° videos. It also assessed the impact of spatial audio on the user’s Quality of Experience (QoE) by capturing implicit, explicit, and objective metrics. The results suggest surprisingly similar explicit QoE ratings for both the spatial and non-spatial audio environments. The implicit metrics indicate that users integrated with the spatial environment more quickly than the non-spatial environment. Users who experienced the spatial audio environment had a higher maximum mean head pose pitch value and were found to be more focused towards the sound-emitting regions in the spatial audio environment experiences.