Multi-modal physiological signals based fear of heights analysis in virtual reality scenes

Abstract

Fear of heights (FoH) analysis and its association to physiological signals can better help understand people’s emotion and quantify human’s behavior, which have been found important in many applications, such as disease analysis, affective computing, etc. Existing studies are mainly on how to alleviate FoH while little literature on FoH analysis has been reported in the past. In this paper, we present the studies of correlation on FoH to multi-modal physiological signals in the virtual reality (VR). To stimulate the FoH to participants, 4 types of VR scenarios that consist of the virtual scene of the VR game “Richie’s plank experience” and the realistic stimulus of hitting by basketball are adopted in the experiment. The synchronized eye movement (EMO), pupil, and electrocardiogram (ECG) of 17 healthy subjects with an even mix of men and women are recorded for FoH analysis. The observations on the FoH analysis carried out in the paper are two-fold: (1) The recognition ability of multi-modal physiological signals for HoF has been evaluated using ReliefF feature selection algorithm. The results show that pupil feature based model generally achieves the best classification performance for FoH. Multiple features consisted of pupil diameter, power spectral density (PSD) of EMO, pupil, ECG, and mean value of EMO can effectively quantify the variability. (2) The FoH analysis model built on the correlation analysis algorithms combined with multi-modal feature-level fusion and strategy-level fusion methods can effectively overcome the drawbacks of conventional statistical models.