Improved Aspect-level Sentiment Analysis Method based on Multi-Head Attention Mechanism

Abstract

The purpose of the aspect-level sentiment analysis task is to analyze the sentiment orientation expressed by different aspects in the text. It has more fine-grained sentiment evaluation objects and is more in line with actual needs. Therefore, it has received extensive attention in recent years. At present, the model that combines the attention mechanism with the Recurrent Neural Network and its variants has gradually become the main method to solve aspect level sentiment analysis tasks. However, this type of method is limited by the deficiencies of the RNN itself, the training time is too long and the dependence between words decreases with the increase of distance. Even if the attention mechanism is added, the above problems will still exist. Therefore, this paper introduces the multi-head attention mechanism into the GCAE model (Gated Convolutional network with Aspect Embedding), and then proposes the GCAE-MHA model (Improved GCAE model based on Multi-Head Attention mechanism). This model models the context and specific aspects of the text sequence at the same time, and learns the interactive relationship between them. And finally uses the two together as the basis for sentiment classification. At the same time, in order to make up for the problem that the convolutional neural network can only extract the dependencies between local words, the GCAE-MHA model also uses the Dilated Convolutional Neural Network to replace it, which can extract the semantic information between long-distance words, and by setting different dilation rate to obtain a richer semantic feature. Finally, experiments are carried out on the SemEval2014 dataset and Twitter dataset. The experimental results show that the GCAE-MHA model can effectively improve the effect of aspect level sentiment analysis while ensuring the simplicity of the model network structure.