Collaborative Attention Network with Word and N-Gram Sequences Modeling for Sentiment Classification

Abstract

Current state-of-the-art models for sentiment classification are CNN-RNN-based models. These models combine CNN and RNN in two ways: parallel models or serial models. Parallel models use CNN to capture n-grams and RNN to model word sequences, while serial models feed n-grams into RNN to model n-gram sequences. However, these models are different from the way humans read text. Intuitively, humans read text by capturing semantic elements that are made up of both words and n-grams. To tackle this problem, we propose a collaborative attention network with word and n-gram sequences modeling. Our model jointly processes sequences in both word granularity and n-gram granularity to form text embedding with collaborative attention. It utilizes a LSTM encoder to capture long-term dependencies among words and a CNN-LSTM component to capture long-term dependencies among n-grams on the same text. Next we incorporate these two parts via an attention mechanism to highlight keywords in sentences. Experimental results show our model effectively outperforms other state-of-the-art CNN-RNN-based models on several public datasets of sentiment classification.