Spectrum-based channel attention cooperating with time continuity encoding in transformer for EEG emotion analysis

Abstract

Recently, electroencephalogram (EEG) emotion analysis has attracted increasing attention in many fields, such as neuroscience and brain-computer fusion. Due to the spatial channel difference and time continuity of emotional responses, it is necessary to capture relevant properties from EEG signals. In this paper, spectrum-based spatial channel attention is developed to reveal the emotional responses of different electrode channels located in different cerebral cortex regions. In addition, a time continuity encoding mechanism is developed to encode the temporal relations of time series signals in the transformer. In our model, the spectrum-based spatial channel attention cooperates with the time continuity encoding mechanism to fully utilize the spectral, spatial and temporal features of EEG signals (SST-Emo). Extensive experiments with different window sizes (1 s and 3 s) are carried out on DEAP for both binary and multiclass classification tasks. The accuracies are 96.28 %, 95.25 %, 95.52 %, 95.83 % and 93.88 % for 1 s window-size on arousal, valence, dominance, liking and 5-class, respectively. For 3 s window-size, the accuracies are 96.13 %, 95.74 %, 95.23 %, 95.7 % and 95.83 %, respectively. The results show that SST-Emo is superior to the state-of-the-art methods, which indicates that SST-Emo is able to extract more discriminative emotional representations.