Multisensory Immersive Experiences: A Pilot Study on Subjective and Instrumental Human Influential Factors Assessment

Abstract

Recent technological advances have allowed for virtual reality applications to burgeon. With virtual reality (VR), so-called human influential factors play a crucial role in the final perceived immersive media experience (IMEx). While two individuals can use the same VR headset, play the same game in the same location, and have the same goals, the two individuals can have very different experiences, with varying perceptions of immersion, presence, realism, engagement, and cybersickness. This can be particularly true in multisensory immersive experiences where, in addition to audio-visual stimuli, olfactory and haptic feedback can be used. In this paper, we describe a pilot study in which a VR game was developed to combine audio-visual, olfactory, and haptic feedback to the user in real-time. After game play, participants were asked about their IMEx using five scales: realism, immersion, presence, engagement, and overall quality of experience (QoE). Moreover, using an instrumented VR headset we measure electroencephalography (EEG), electrocardiography (ECG), and electrooculography (EOG) signals and compute several instrumental measures of human influential factors, including an engagement index, arousal and valence indices, frontal alpha asymmetry, heart rate, several EEG sub-band powers, and eye blink rate. Using the subjective ratings, we measure the contribution that each IMEx subscale has on overall QoE, as a function of the type of sensory stimuli used. Results on 11 participants suggest very different contributions once smells and haptics are incorporated, relative to traditional audio-visual experiences. We also report on several instrumental measures that showed significant correlations with the IMEx subscales, suggesting that, in the future, real-time instrumental QoE measurement of multisensory experiences could be possible.