Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning

Abstract

Images are the easiest medium through which people can express their emotions on social networking sites. Social media users are increasingly using images and videos to express their opinions and share their experiences. Sentiment analysis of such large scale visual content can help better extract user sentiments toward events or topics, such as those in image tweets, so that prediction of sentiment from visual content is complementary to textual sentiment analysis. Significant progress has been made with this technology, however, there is little research focus on the picture sentiments. In this work, an image sentiment prediction framework is built with Convolutional Neural Networks (CNN). Specifically, this framework is pretrained on a large scale data for object recognition to further perform transfer learning. Extensive experiments were conducted on manually labeled Flickr image dataset. To make use of such labeled data, we employ a progressive strategy of domain specific fine tuning of the deep network. The results show that the proposed CNN training can achieve better performance in image sentiment analysis than competing networks.