Transformers for EEG-Based Emotion Recognition: A Hierarchical Spatial Information Learning Model

Abstract

The spatial information of Electroencephalography (EEG) is essential for emotion recognition model to learn discriminative feature. The convolutional networks and recurrent networks are the conventional choices to learn the complex spatial dependencies through a number of electrodes and brain regions. However, these models have difficulty in capturing long-range dependencies due to the operations of local feature learning. To enhance EEG spatial dependencies capturing and improve the accuracy of emotion recognition, we propose a transformer- based model to hierarchically learn the discriminative spatial information from electrode level to brain-region-level. In the electrode-level spatial learning, the transformer encoders are adopted to integrate information within different brain regions. Next, in view of the different roles of brain regions in the emotion recognition, the self-attention within the transformer could emphasize the contributive brain regions. Hence, in the brain-region-level spatial learning, a transformer encoder is utilized to capture the spatial dependencies among the brain regions. Finally, to validate the effectiveness of the proposed model, the subject-independent experiments are conducted on the DEAP and MAHNOB-HCI database. The experimental results demonstrate that the proposed model achieves outstanding performance in emotion recognition with arousal and valence level. Moreover, the visualization of self-attention indicates that the proposed model could emphasize the discriminative spatial information from pre-frontal lobe, frontal lobe, temporal lobe and parietal lobe.