Kitchen Horrors: Unraveling the Influence of Multimodal Stressors on User Experience in Virtual Reality through Electrodermal Activity

Abstract

The last couple of years have seen a surge in the quality of on-site multiplayer virtual reality experiences. The shift to standalone VR headsets, the decrease in latency and the increase in reliability of rendering VR content have all benefited the rise of VR entertainment parks. The next frontier, however, seems to be the inclusion of sensor data (e.g., electrodermal activity signals) to aid the creation of adaptive VR experiences that are equally immersive for all users. If we can assess the specific impact certain stimuli have on the user during an immersive experience, creators will not only be able to create more engaging content but also design feedback loops to bring to users personalized VR experiences in real time. The current study takes a vital step in this direction by measuring electrodermal activity (EDA) to differentiate between stress responses to visual, audio, and audio-visual stimuli in a haunted VR kitchen game. The study leverages data from 13 participants who underwent a 40-minute-long virtual reality experience. The analysis suggests that relying solely on cleaned EDA data to differentiate between stress and no-stress conditions may not be effective, despite subjective reports of such distinctions. However, a more detailed analysis of EDA features (i.e., EDA peak amplitude and SCR peak amplitude) reveals the ability to not only differentiate between the impact of various stimuli modalities (audio, visual, and audio-visual) on stress responses but also discern between individuals’ responses. These findings underscore the imperative for adaptive VR experiences tailored to the unique responses of individual users, pointing towards a future where personalized, real-time immersive experiences can be finely crafted based on users' physiological reactions.