Head-area sensing in virtual reality: future visions for visual perception and cognitive state estimation

Abstract

Biosensing techniques are progressing rapidly, promising the emergence of sophisticated virtual reality (VR) headsets with versatile biosensing enabling an objective, yet unobtrusive way to monitor the user’s physiology. Additionally, modern artificial intelligence (AI) methods provide interpretations of multimodal data to obtain personalised estimations of the users’ oculomotor behaviour, visual perception, and cognitive state, and their possibilities extend to controlling, adapting, and even creating the virtual audiovisual content in real-time. This article proposes a visionary approach for personalised virtual content adaptation via novel and precise oculomotor feature extraction from a freely moving user and sophisticated AI algorithms for cognitive state estimation. The approach is presented with an example use-case of a VR flight simulation session explaining in detail how cognitive workload, decrease in alertness level, and cybersickness symptoms could be modified in real-time by using the methods and embedded stimuli. We believe the envisioned approach will lead to significant cost savings and societal impact and will thus be a necessity in future VR setups. For instance, it will increase the efficiency of a VR training session by optimizing the task difficulty based on the user’s cognitive load and decrease the probability of human errors by guiding visual perception via content adaptation.