Multi-modal physiological sensing approach for distinguishing high workload events in remotely piloted aircraft simulation

Abstract

Remotely piloted aircraft (RPA) operations are often characterized as highly taxing and dynamic. Physiological sensing technology can enhance personnel monitoring and training for these high-stress environments; however, work assessing the effectiveness of physiological sensors during RPA operations is limited. The proposed work tests two hypotheses: (1) physiological sensors can distinguish operator workload between scenario difficulty levels, and (2) the sensors can quantify the impact of RPA events on the operator workload. Twelve pilots completed RPA simulations at all three difficulty levels while physiological sensors collected electroencephalogram (EEG) and heart rate activity. Hypotheses were tested using mixed-effects models. Observed heart rate variability metrics did not differ among the three scenario difficulty levels except for LF/HF ratio. A 47% and 57% reduction in alpha band power was observed between easy and hard difficulty levels for the frontal and parietal channels, respectively. Abort and Reach objective events resulted in 0.2–0.3 dB lower beta activity and 66 ms increased heart rate, while losing sight of the objective (e.g., fog) had 0.72 dB increased high beta activity. Different physiological modalities (EEG and ECG) had varying effectiveness in distinguishing scenario difficulty and RPA events, suggesting a hybrid sensing approach may provide more insight than just using one modality. In conclusion, physiological sensing can distinguish operator response to scenario difficulty and events in high-fidelity RPA simulations.