Abstract
Aiming at the problem of insufficient sentiment word extraction ability in existing text sentiment analysis methods and OOV (out-of-vocabulary) problem of pre-training word vectors, a neural network model combining multi-head self-attention and character-level embedding is proposed. An encoder-decoder (Seq2Seq) model is used for sentiment analysis of text. By adding character-level embedding to the word embedding layer, the OOV problem of pre-trained word vectors is solved. At the same time, the self-attention mechanism is used to perform attention calculations within the input sequence to find intra-sequence connections. The model uses bidirectional LSTM (Long Short-Term Memory) independently encodes the context semantic information of the input sequence to obtain deeper emotional features, to more effectively identify the emotional polarity of short text. The effectiveness of our method was verified on the public Twitter dataset and the movie review dataset published by ACL. Experimental results show that the accuracy of the model on the three categories of Twitter, movie reviews and IMDB datasets reaches 66.45%, 79.48% and 90.34%, respectively, which verifies that the model has excellent performance in datasets in different fields.
Highlights
In recent years, with the emergence of social networks such as twitter and microblog, people can and conveniently share diversified information about individuals on social platforms such as texts, pictures, videos
We propose a twitter short text sentiment analysis model (Self-AT-LSTM) based on the multi head self attention mechanism and LSTM
(2) The twitter sentiment analysis method based on the combination of multi head self-attention mechanism and bidirectional LSTM is proposed
Summary
With the emergence of social networks such as twitter and microblog, people can and conveniently share diversified information about individuals on social platforms such as texts, pictures, videos. In the semeval-2017 competition, Baziotis et al [11] and others used deep LSTM combined with attention model to assign weight to key words in short text through attention mechanism, enhanced the influence of keywords on sentence emotion, and achieved good results in twitter emotion classification. We propose a twitter short text sentiment analysis model (Self-AT-LSTM) based on the multi head self attention mechanism and LSTM. (2) The twitter sentiment analysis method based on the combination of multi head self-attention mechanism and bidirectional LSTM is proposed. The multi head attention mechanism is introduced into Twitter text sentiment analysis, and the advanced characteristics of the above models in emotional analysis tasks are verified by our experiments. (3) Post attention LSTM is added after the attention layer to simulate the decoder function and extract the predicted features of the model
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
