S2SAN: A sentence-to-sentence attention network for sentiment analysis of online reviews

Abstract

Many existing attention-based deep learning approaches to sentiment analysis have focused on words and represent an entire review text as a word sequence. However, these approaches overlook the differences in the importance of each sentence to the complete text. To solve this problem, some work has been performed to calculate sentence-level attention, but these studies use the same approach that is applied to word-level attention, which leads to unnecessary sequential structures and increased complexity of sentence representation. Therefore, in this paper, we propose a sentence-to-sentence attention network11https://github.com/JingruiHou/S2SAN (S2SAN) using multihead self-attention. We conducted several domain-specific, cross-domain and multidomain sentiment analysis experiments with real-world datasets. The experimental results show that S2SAN outperforms other state-of-the-art models. Some classical sentiment classifiers [e.g., convolutional neural network (CNN), recurrent neural network (RNN), and long short-term memory (LSTM) models] achieve better accuracies when they are reconfigured to include sentence-to-sentence attention.