Context Aware Image Sentiment Classification using Deep Learning Techniques

Abstract

Objectives: To propose context aware sentiment classification using deep learning techniques. Methods: We used EfficientNetB-7 deep learning framework for caption generation for the input image and to classify the sentiment of generated caption using machine learning techniques. First, we employ several real-time and synthetic image datasets, then apply pre-processing and normalization for data balancing. Then efficient module implementation for feature extraction and selection using convolutional and pooling layers were done. Despite this proceeding, it generates the caption for respective images. The various feature extraction and selection Natural Language Processing (NLP) techniques such as TF-IDF, lemmas, dependency and correlational features have been used and classify the sentiment label using attention model and greedy approach. Finally, generating the blue score for the entire testing dataset and show the effectiveness of the proposed system. Findings: Our model gives higher accuracy with different deep learning techniques which is demonstrated in result section. The proposed model archives 73.80% average accuracy for EMOTIC dataset. The module has evaluated with different features and deep learning classification algorithms proposed earlier. Novelty: This research is the collaboration of Deep learning and machine learning classification techniques. We first extract the visual features from the input image using deep learning and classify with machine learning with the collaboration of NLP processes. We also carried out various feature extraction techniques such as Ngram, dependency features, co-relational features and determined the sentiment of generated captions. Keywords: Image Sentiment analysis; Emotic dataset; CNN; EfficientNetB7; Attention based LSTM; GRU