Sentiment Analysis of Text Based on Bidirectional LSTM With Multi-Head Attention

Fei Long,Weihua Ou,Kai Zhou

doi:10.1109/access.2019.2942614

Fei Long, Weihua Ou + Show 1 more

Open Access

https://doi.org/10.1109/access.2019.2942614

Copy DOI

Journal: IEEE Access	Publication Date: Jan 1, 2019
Citations: 61	License type: CC BY 4.0

Affiliation: Guizhou University, Guizhou Normal University

Abstract

Recent years, many scientists address the research on text sentiment analysis of social media due to the exponential growth of social multimedia content. Natural language ambiguities and indirect sentiments within the social media text have made it hard to classify by using traditional machine learning approaches, such as support vector machines, naive Bayes, hybrid models and so on. This article aims to investigate the sentiment analysis of social media Chinese text by combining Bidirectional Long-Short Term Memory (BiLSTM) networks with a Multi-head Attention (MHAT) mechanism in order to overcome the deficiency of Sentiment Analysis that is performed with traditional machine learning. BiLSTM networks, not only solve the long-term dependency problem, but they also capture the actual context of the text. Due to the fact that the MHAT mechanism can learn the relevant information from a different representation subspace by using multiple distributed calculations, the purpose is to add influence weights to the constructed text sequence. The results of the numerical experiments show that the proposed model achieves better performance than the existing well-established methods.

Highlights

The sentiment Analysis [1], which is called to be the crown of the artificial intelligence field [2], aims to use corresponding technical means to tap the potential emotional attitude that is contained in text
Input train data set: 1) Build a bidirectional Long Short-Term Memory (LSTM) with an output shape of; 2) Build a multi-head attention with an output shape of; 3) Build a global average pooling with an output shape of; 4) Build the classification layer, in which the emotion classification is obtained by the softmax classifier (17), and the weight w is continuously updated by the loss function (24), and the final output shape is; 5) Save model and weight; Step 5
The original text is first pre-processed as a word vector, and the traditional LSTM model is replaced with a bidirectional LSTM to obtain the context of the text

Summary

INTRODUCTION

The sentiment Analysis [1], which is called to be the crown of the artificial intelligence field [2], aims to use corresponding technical means to tap the potential emotional attitude (such as criticism, praise, and so on) that is contained in text. The process of building sentiment dictionary is time-consuming and labor-intensive, and some prior knowledge is required Their effectiveness relies heavily on the application of feature selection strategies and. F. Long et al.: Sentiment Analysis of Text Based on BiLSTM With MHAT the long-distance dependence problem of Recurrent Neural Networks (RNN) [11], [12]. The first application of the attention mechanism was in the machine translation based on seq2seq in NLP. The main contribution of this paper is to make the multi-head attention mechanism limited to machine translation tasks, but to the field of text sentiment analysis. Considering that text sentiment analysis is fundamentally a sequence problem, this paper combines a multi-head attention mechanism with the BiLSTM into a hybrid model for text sentiment analysis [18]

WORD SEGMENTATION

GLOBAL AVERAGE POOLING LAYER

SOFTMAX LAYER

Findings

CONCLUSION