A survey of challenges and methods for Quality of Experience assessment of interactive VR applications

Abstract

User acceptance of virtual reality (VR) applications is dependent on multiple aspects, such as usability, enjoyment, and cybersickness. To fully realize the disruptive potential of VR technology in light of recent technological advancements (e.g., advanced headsets, immersive graphics), gaining a deeper understanding of underlying factors and dimensions impacting and contributing to the overall end-user experience is of great benefit to hardware manufacturers, software and content developers, and service providers. To provide insight into user behaviour and preferences, researchers conduct user studies exploring the influence of various user-, system-, and context-related factors on the overall Quality of Experience (QoE) and its dimensions. When planning and executing such studies, researchers are faced with numerous methodological challenges related to study design aspects, such as specification of dependant and independent variables, subjective and objective assessment methods, preparation of test materials, test environment, and participant recruitment. Approaching these challenges from a multidisciplinary perspective, this paper reviews different aspects of performing perception-based QoE assessment for interactive VR applications and presents options and recommendations for research methodology design. We provide an overview of different influence factors and dimensions that may affect the overall QoE, with a focus on presence, immersion, and discomfort. Furthermore, we address ethical and practical issues regarding participant choice and test material, present different assessment methods and measures commonly used in VR research, and discuss approaches to choosing study duration and location. Lastly, we provide a concise analysis of key challenges that need to be addressed in future studies centered around VR QoE.