MS-FTSCNN: An EEG emotion recognition method from the combination of multi-domain features

Abstract

Electroencephalography (EEG), as a physiological cue, is more objective and reliable in identifying emotions than non-physiological cues. Previous methods only consider one or two relationships among frequency, time and spatial domain features of EEG signals, and the designed models may still be relatively large in terms of parameters. Meanwhile, the training process of the previous networks is troublesome during algorithm optimization. To address these challenges, we design a simple and efficient feature preprocessing method to obtain a 3D feature structure that contains EEG signal information in the frequency, time and spatial domains simultaneously. Then, we propose a multiscale frequency–time–spatial convolutional model, MS-FTSCNN, which is able to capture frequency, time and spatial features from the input signals and fuse three features more efficiently. Moreover, the multi-scale one-dimensional convolutional kernel in our method can reduce network parameters, providing possibilities for real-time online applications. Finally, the recognition accuracies of arousal and valence of our proposed model are 93.82%, 94.48% on DEAP dataset and 92.64%, 92.15% on MOHNOB-HCI dataset, which is higher than most existing methods.