Exploiting Out-of-Domain Datasets and Visual Representations for Image Sentiment Classification

Abstract

Visual sentiment analysis has recently gained attention as an important means of opinion mining, with many applications. It involves a high level of abstraction and subjectivity, which makes it a challenging task. The most recent works are based on deep convolutional neural networks, and exploit transfer learning from other image classification tasks. However, transferring knowledge from tasks other than image classification has not been investigated in the literature. Motivated by this, in this work we examine the potential of transferring knowledge from several pre-trained networks, some of which are out-of-domain. We show that by simply concatenating these diverse feature vectors we construct a rich image representation that can be used to train a classifier with state of the art performance on image sentiment analysis. We also evaluate a Mixture of Experts approach, for learning from this combination of representations, and highlight its performance advantages. We compare against the top-performing recently-published methods on four popular benchmark datasets and report new SOTA results on three of the four.