Horizontal and vertical features fusion network based on different brain regions for emotion recognition

Abstract

Deep learning technology has been universally adopted in emotion recognition, which becomes a promising method that has recently achieved good recognition performance. However, the existing methods cannot reflect the influence of electroencephalography (EEG) signals from different brain regions on emotion recognition. Therefore, in the article, we raise a horizontal and vertical features fusion network based on different brain regions (HVF2N-DBR) for emotion recognition. The HVF2N-DBR method not only considers the influence of EEG activity in the various brain regions on affective recognition but also obtains the temporal–spatial characteristics of EEG signals. Specifically, EEG signals firstly are grouped according to the various brain lobes (frontal, parietal, temporal, and occipital lobes). Then, a horizontal–vertical features fusion module (HVF2M) is designed to learn multiple direction features of different brain regions. Afterwards, finer features of different brain regions are learned separately through hybrid dilation convolutions, then they are concatenated to capture richer and more discriminative emotional information. The designed architecture can achieve higher recognition performance. Finally, on three databases, i.e., SEED, SEED-IV, and DEAP, extensive experiments are implemented to estimate the capability of the designed HVF2N-DBR network. Results validate that the recognition performance of proposed model is superior to many existing models. Meanwhile, the ablation experiments also reveal that EEG signals related to emotion recognition mainly situate the frontal and parietal lobes of the cerebral cortex.