Assessment of cognitive workload using simultaneous EEG and fNIRS: A comparison of feature combinations

Abstract

The assessment of cognitive workload is a critical component in evaluating mental activity. It is essential in psychology, especially in professions with heightened cognitive demands, such as aviation and surgery. Even marginal improvements in workload assessment can substantially enhance safety and overall performance. This study aimed to improve the accuracy of cognitive workload assessment by employing a combined EEG+fNIRS approach with a minimal number of features. The Tunable Q-factor Wavelet Transform (TQWT) was applied to signals from EEG and fNIRS to extract relevant features. TQWT was chosen for its ability to adapt to the oscillatory behavior of the signal, and the features extracted from the transform facilitated the differentiation between different signal classes. This study introduced six proposed approaches, with the first focusing on EEG signals, the second on fNIRS signals, and the subsequent three to six approaches involving combinations of these two signals. Features evaluation highlighted the pivotal role of channels related to the frontal region and sub-bands associated with alpha and theta rhythms in EEG signals. Additionally, the very low-frequency range and the primary undecomposed signal in fNIRS signals contributed significantly to cognitive workload classification. The classification results demonstrated the efficacy of combining EEG and fNIRS signals in cognitive workload detection, providing higher accuracy than single-signal approaches. Compared to other feature combination approaches, the proposed approach based on Canonical Correlation Analysis (CCA) with a linear SVM classifier achieved superior accuracy across all investigated cognitive workload levels, with accuracies of 99.6 %, 99.4 %, 98.9 %, and 97.5 % for the 0-back vs. 3-back, 0-back vs. 2-back, 2-back vs. 3-back, and 0-back vs. 2-back vs. 3-back comparisons, respectively.