The optimally designed dynamic memory networks for targeted sentiment classification

Abstract

Abstract This paper focuses on the task of targeted sentiment classification over dynamic memory networks. To remedy the flaw of the previous related work, only doing well in simple sentences, the new dynamic memory networks, in which the input module, question module and memory module are optimally designed, are established to model the task into question answering system for the first time. Specifically, in the input module, position embedding is considered to eliminate the influence of sentiment-irrelevant words that close to the target, and skip connection can help to build rich representations over the sentences. In the question module, for dealing with many cases where the target is composed of multiple words, the commonly used average target vector is replaced by the designed target sentiment question, which is encoded by a GRU. In the memory module, two improvements are performed. On the one hand, weight bias, caused by using original soft attention to extract memory information at each attention step, is avoided by both exploring attention and inner attention based GRU networks. On the other hand, memory updating block is capable to distill sentiment-relevant features from memory information, which is extracted by multiple attention block using multiple attention mechanism. Finally, experimental results demonstrate the effectiveness of the proposed models for the task of targeted sentiment classification among three different datasets. Additionally, the best model of this paper achieves state-of-the-art performance on different types of data.