Identifying stable EEG patterns over time for mental workload recognition using transfer DS-CNN framework

Abstract

Timely evaluating operator Mental Workload (MW) levels in human–computer interaction helps to assign tasks legitimately and guarantee safety and efficiency at the same time. This study presented a stable Electroencephalogram (EEG) pattern over time. A data augmentation (DA) method and a deep neural network (NN) are developed accordingly based on the peculiarity of the EEG pattern for MW assessment. The EEG pattern consists of rhythmic energies in the Temporal–Spatial–Spectral domain, named energy tensor. The DA method for energy tensors is realized by randomly disrupting the temporal dimension which was certified to be irrelevant to MW levels in our previous study. The NN is designed based on deep transfer learning and Depthwise Separable Convolutional Neural Networks (DS-CNN). Energy tensors after DA are used as the input of the proposed NN to construct MW classifiers and the entire operation is named transfer DS-CNN framework. The experiment results demonstrate that the optimal temporal resolution of EEG patterns for MW classification is 3.5 min. The transfer DS-CNN framework is effective enough to build a cross-session MW classifier for a specific subject in an average of 9.195 s, with an average prediction accuracy of 97.22%. Our study suggests that the stable EEG patterns and the DA method preserve features for cross-session MW recognition. It is befitting to discriminate MW levels in real-time with the transfer DS-CNN framework.