Use of multimodal physiological signals to explore pilots’ cognitive behaviour during flight strike task performance

Abstract

This study explored the use of multi-physiological signals and simultaneously recorded high-density electroencephalography (EEG), electrocardiogram (ECG), and eye movements to better understand pilots’ cognitive behaviour during flight simulator manoeuvres. Multimodal physiological signals were collected from 12 experienced pilots with international aviation qualifications under the wide-angle and impressive vision simulation. The data collection spanned two flight strike missions, each with three mission intensities, resulting in a data set of EEG, ECG, and eye movement signals from six subtasks. The multimodal data were analysed using signal processing methods. The results indicated that, when the flight missions were performed, the pilots’ physiological characteristics exhibited rhythmic changes in the power spectrum of θ waves in the EEG, r-MSSD in the ECG, and average gaze duration. Furthermore, the pilots’ physiological signals were more sensitive during the target mission than during the empty target mission. The results also showed correlations between different physiological characteristics. We showed that specific multimodal features are useful for advancing neuroscience research into pilots’ cognitive behaviour and processes related to brain activity, psychological rhythms, and eye movement.