Fine-Tuning of Word Embeddings for Semantic Sentiment Analysis

Abstract

In this paper, we present a state-of-the-art deep-learning approach for sentiment polarity classification. Our approach is based on a 2-layer bidirectional Long Short-Term Memory network, equipped with a neural attention mechanism to detect the most informative words in a natural language text. We test different pre-trained word embeddings, initially keeping these features frozen during the first epochs of the training process. Next, we allow the neural network to perform a fine-tuning of the word embeddings for the sentiment polarity classification task. This allows projecting the pre-trained embeddings in a new space which takes into account information about the polarity of each word, thereby being more suitable for semantic sentiment analysis. Experimental results are promising and show that the fine-tuning of the embeddings with a neural attention mechanism allows boosting the performance of the classifier.