Heart rate variability analysis for the assessment of immersive emotional arousal using virtual reality: Comparing real and virtual scenarios.

Javier Marín-Morales,Mariano Alcañiz,Juan Luis Higuera-Trujillo,Gaetano Valenza,Carmen Llinares,Jaime Guixeres,Hedwig Eisenbarth

doi:10.1371/journal.pone.0254098

Javier Marín-Morales, Mariano Alcañiz + Show 5 more

Open Access

https://doi.org/10.1371/journal.pone.0254098

Copy DOI

Abstract

Many affective computing studies have developed automatic emotion recognition models, mostly using emotional images, audio and videos. In recent years, virtual reality (VR) has been also used as a method to elicit emotions in laboratory environments. However, there is still a need to analyse the validity of VR in order to extrapolate the results it produces and to assess the similarities and differences in physiological responses provoked by real and virtual environments. We investigated the cardiovascular oscillations of 60 participants during a free exploration of a real museum and its virtualisation viewed through a head-mounted display. The differences between the heart rate variability features in the high and low arousal stimuli conditions were analysed through statistical hypothesis testing; and automatic arousal recognition models were developed across the real and the virtual conditions using a support vector machine algorithm with recursive feature selection. The subjects' self-assessments suggested that both museums elicited low and high arousal levels. In addition, the real museum showed differences in terms of cardiovascular responses, differences in vagal activity, while arousal recognition reached 72.92% accuracy. However, we did not find the same arousal-based autonomic nervous system change pattern during the virtual museum exploration. The results showed that, while the direct virtualisation of a real environment might be self-reported as evoking psychological arousal, it does not necessarily evoke the same cardiovascular changes as a real arousing elicitation. These contribute to the understanding of the use of VR in emotion recognition research; future research is needed to study arousal and emotion elicitation in immersive VR.

Highlights

Of the features, the majority of the studies that have used Heart Rate Variability (HRV) and immersive virtual reality (VR) relied on classic statistical methods such as hypothesis testing and correlations [15]; automatic arousal recognition models have recently been recommended using machine-learning algorithms, which allows to discriminate between states at a single-subject level
In the real museum the model achieves 72.92% accuracy, being balanced in true positive rate (TPR) (67.24%) and true negative rate (TNR) (76.74%)
In this study we investigated cardiovascular dynamics during high and low arousing elicitations through exploration of a real and a virtual museum and assessed the validity of VR by analysing physiological responses

Summary

Introduction

The study of emotions is a very important research topic for the understanding of human behaviour, as well as human perception, decision-making, creativity, memory and social. The popularity of VR has increased exponentially in recent years due to the development of a new generation of head-mounted displays [16] These are fully immersive and interactive systems that isolate the user from external world stimuli and provide a complete simulated stereoscopic experience responsive to head movements, which in turn provokes a high sense of presence, that is, the strong illusion of being in the simulated environment [14]. Of the features, the majority of the studies that have used HRV and immersive VR relied on classic statistical methods such as hypothesis testing and correlations [15]; automatic arousal recognition models have recently been recommended using machine-learning algorithms, which allows to discriminate between states at a single-subject level. Heart rate variability analysis for the assessment of immersive emotional arousal using virtual reality performed a direct comparison of cardiovascular dynamics including in the time, frequency and non-linear domains. A support vector machine classifier was developed to recognise arousal levels in both experimental conditions; this included a recursive feature elimination wrapper to explore the importance of each feature

Participants

Results

Discussion