Decoding subjective emotional arousal from EEG during an immersive virtual reality experience.

Simon M Hofmann,Arno Villringer,Michael Gaebler,Alberto Mariola,Vadim Nikulin,Felix Klotzsche

doi:10.7554/elife.64812

Abstract

Immersive virtual reality (VR) enables naturalistic neuroscientific studies while maintaining experimental control, but dynamic and interactive stimuli pose methodological challenges. We here probed the link between emotional arousal, a fundamental property of affective experience, and parieto-occipital alpha power under naturalistic stimulation: 37 young healthy adults completed an immersive VR experience, which included rollercoaster rides, while their EEG was recorded. They then continuously rated their subjective emotional arousal while viewing a replay of their experience. The association between emotional arousal and parieto-occipital alpha power was tested and confirmed by (1) decomposing the continuous EEG signal while maximizing the comodulation between alpha power and arousal ratings and by (2) decoding periods of high and low arousal with discriminative common spatial patterns and a long short-term memory recurrent neural network. We successfully combine EEG and a naturalistic immersive VR experience to extend previous findings on the neurophysiology of emotional arousal towards real-world neuroscience.

Highlights

While humans almost constantly interact with complex, dynamic environments, lab-based studies typically use simplified stimuli in passive experimental situations
Our results provide evidence that the neural mechanisms reflected in modulations of alpha power – in parieto-occipital regions – bear information about the subjective emotional state of a person undergoing an immersive and emotionally arousing experience
We found that the Long Short-Term Memory (LSTM) can extract features from neural input components that reflect changes in subjective emotional arousal and that the accuracy of its predictions in both conditions matched closely the ones of Common Spatial Patterns (CSP)

Summary

Introduction

While humans almost constantly interact with complex, dynamic environments, lab-based studies typically use simplified stimuli in passive experimental situations. Trading realism for experimental control happens at the expense of the representativity of the experimental design (Brunswik, 1955), that is, the degree to which effects found in the lab generalize to practical everyday-life conditions This may be true for affective phenomena like emotions. [Footnote: Different types of arousal have been proposed and investigated, such as sexual, autonomic, emotional (Russell, 1980); in the context of altered states of consciousness, for example through anaesthesia or sleep. They may share psychological (e.g., increase in sensorimotor and emotional reactivity; Pfaff et al, 2012) and physiological aspects (e.g., sympathetic activation) but are not synonymous. It has thereby been difficult to consistently associate individual, discrete emotion categories with specific response patterns in the ANS (cf. Kragel & LaBar, 2013; Kreibig, 2010; Siegel et al, 2018) or in distinct brain regions (Lindquist et al, 2012; but cf. Saarimäki et al, 2016)

Objectives

Methods

Results

Conclusion