Identification of mental fatigue levels in a language understanding task based on multi-domain EEG features and an ensemble convolutional neural network

Abstract

In this paper, electroencephalogram (EEG) is used to assess mental fatigue of operators in a human-computer system aiming at preventing increasing risk of human operator performance degradation. We present an experimental design for fatigue identification in a language understanding task. The EEG signals of 14 channels from 15 healthy participants were collected via a wireless brain computer interface device to indicate instantaneous fatigue level. By extracting EEG features as temporal statistics, power spectral density and entropy indicators, we build four different spatial feature maps that restructure feature vectors. Further, a bootstrap-aggregating ensemble convolutional neural network of multi-domain features (ensCNN-MD) is proposed to improve the fatigue recognition accuracy. By examining seven different feature combinations, ensCNN-MD is significantly superior to classical shallow and deep classifiers. The highest classification accuracy under participant-specific training and testing paradigm is achieved at 87.69%. The results demonstrate both the effectiveness of experimental design and the ensCNN-MD of learning high-level spatial feature abstractions related to mental fatigue variations.