Abstract
Due to the complexity of human emotions, there are some similarities between different emotion features. The existing emotion recognition method has the problems of difficulty of character extraction and low accuracy, so the bidirectional LSTM and attention mechanism based on the expression EEG multimodal emotion recognition method are proposed. Firstly, facial expression features are extracted based on the bilinear convolution network (BCN), and EEG signals are transformed into three groups of frequency band image sequences, and BCN is used to fuse the image features to obtain the multimodal emotion features of expression EEG. Then, through the LSTM with the attention mechanism, important data is extracted in the process of timing modeling, which effectively avoids the randomness or blindness of sampling methods. Finally, a feature fusion network with a three-layer bidirectional LSTM structure is designed to fuse the expression and EEG features, which is helpful to improve the accuracy of emotion recognition. On the MAHNOB-HCI and DEAP datasets, the proposed method is tested based on the MATLAB simulation platform. Experimental results show that the attention mechanism can enhance the visual effect of the image, and compared with other methods, the proposed method can extract emotion features from expressions and EEG signals more effectively, and the accuracy of emotion recognition is higher.
Highlights
With the rapid development of computer and artificial intelligence technology, computer intelligence is paid more and more attention
For the intrinsic bioelectric signals such as EEG and ECG, there is no intuitive expression of emotional information such as voice signals, the emotion recognition system built by bioelectric signals is more reliable because the bioelectric signals are not easy to be modified by subjective manipulation
Facial expression and EEG features are extracted based on bilinear convolution network (BCN), and the attention mechanism is introduced into the LSTM network
Summary
With the rapid development of computer and artificial intelligence technology, computer intelligence is paid more and more attention. Based on the above analysis, an expression electroencephalogram multimodal affective recognition method based on the bidirectional LSTM and attention mechanism is proposed to solve the problems of the single mode, which are difficulty of deeply extracting signal characteristics and low accuracy of affective recognition. The proposed strategy based on BCN extracts facial expression features, converts the EEG signal into three bands of image sequences, and uses BCN to fuse its image features to obtain more accurate expression EEG multimodal affective features (2) In order to avoid the randomness or blindness of sampling methods, the attention mechanism is introduced in the LSTM network, and a characteristic fusion network with a three-layer bidirectional LSTM structure is designed to combine emoticons and EEG characteristics to improve the accuracy of emotion recognition
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Computational and Mathematical Methods in Medicine
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
