Image Sentiment Research Articles

Sentiment analysis of students in ideological and political teaching based on artificial intelligence and data mining

In order to improve the efficiency of sentiment analysis of students in ideological and political classrooms, under the guidance of artificial intelligence ideas, this paper combines data mining and machine learning algorithms to improve and propose a method for quantifying the semantic ambiguity of sentiment words. Moreover, this paper designs different quantitative calculation methods of sentiment polarity intensity, and constructs video image sentiment recognition, text sentiment recognition, and speech sentiment recognition functional modules to obtain a combined sentiment recognition model. In addition, this article studies student emotions in ideological and political classrooms from the perspective of multimodal transfer learning, and optimizes the deep representation of images and texts and their corresponding deep networks through single-depth discriminative correlation analysis. Finally, this paper designs experiments to verify the model effect from two perspectives of single factor sentiment analysis and multi-factor sentiment analysis. The research results show that comprehensive analysis of multiple factors can effectively improve the effect of sentiment analysis of students in ideological and political classrooms, and enhance the effect of ideological and political classroom teaching.

Visual Sentiment Analysis With Social Relations-Guided Multiattention Networks.

These days, social media users tend to express their feelings through sharing images online. Capturing the emotions embedded in these social images involves great research challenges and practical values. Most existing works concentrate on extracting the visual feature from a global view, while ignoring the fact that visual objects are also rich in emotion. How to leverage the multilevel visual features to improve the sentiment analysis performance is important yet challenging. Besides, existing works view each social image as an independent sample while ignoring the rich correlations among social images, which may be helpful in detecting visual emotion. In this article, we propose a novel model called social relations-guided multiattention networks (SRGMANs) to incorporate both the multilevel (region-level and object-level) visual features of a single image and the correlations among multiple social images to conduct visual sentiment analysis. Specifically, we first construct a heterogeneous network consisting of various types of social relations and introduce a heterogeneous network embedding method to learn the network representation for each image. Then, two visual attention branches (region attention network and object attention network) are devised to extract emotional and discriminative visual features. For each branch, we design a self-attention module to capture the emotional dependencies among visual parts. Besides, a network-guided attention module is also designed in each branch to focus on more network-related emotional visual parts with the guidance of the topology information. Finally, the attended visual features from the two attention models, together with network representation features, are combined within a holistic framework to predict the sentiment of social images. Extensive experiments demonstrate the superiority of our model on three benchmark datasets.

An analysis and prediction model of outsiders percentage as a new popularity metric on Instagram

In this research, a new Instagram popularity metric was defined, i.e. outsiders percentage (OP) of a post. Outsiders are non-followers who liked a user’s post. It was found that OP is the most effective metric if compared to engagement rate and followers growth. Regression models were tested for predicting OP, using features from user data, post data, hashtag, engagement, and image sentiment. The prediction accuracy (R 2 ), reached up to 71.9% using Random Forest. This research also analyzed the trend of each feature against the OP. It was found that hashtag usage is the most important factor in raising OP.

OutdoorSent

Opinion mining in outdoor images posted by users during different activities can provide valuable information to better understand urban areas. In this regard, we propose a framework to classify the sentiment of outdoor images shared by users on social networks. We compare the performance of state-of-the-art ConvNet architectures and one specifically designed for sentiment analysis. We also evaluate how the merging of deep features and semantic information derived from the scene attributes can improve classification and cross-dataset generalization performance. The evaluation explores a novel dataset—namely, OutdoorSent—and other publicly available datasets. We observe that the incorporation of knowledge about semantic attributes improves the accuracy of all ConvNet architectures studied. Besides, we found that exploring only images related to the context of the study—outdoor, in our case—is recommended, i.e., indoor images were not significantly helpful. Furthermore, we demonstrated the applicability of our results in the United States city of Chicago, Illinois, showing that they can help to improve the knowledge of subjective characteristics of different areas of the city. For instance, particular areas of the city tend to concentrate more images of a specific class of sentiment, which are also correlated with median income, opening up opportunities in different fields.

Recognizing Image Semantic Information Through Multi-Feature Fusion and SSAE-Based Deep Network.

Images are powerful tools with which to convey human emotions, with different images stimulating diverse emotions. Numerous factors affect the emotions stimulated by the image, and many researchers have previously focused on low-level features such as color, texture and so on. Inspired by the successful use of deep convolutional neural networks (CNN) in the visual recognition field, we used a data augmentation method for small data sets to gain the sufficient number of the training dataset. In this paper, we use low-level features (color and texture features) of the image to assist the extraction of advanced features (image object category features and deep emotion features of images), which are automatically learned by deep networks, to obtain more effective image sentiment features. Then, we use the stack sparse auto-encoding network to recognize the emotions evoked by the image. Finally, high-level semantic descriptive phrases including image emotions and objects are output. Our experiments are carried out on the IAPS and GAPED data sets of the dimension space and the artphoto data set of the discrete space. Compared with the traditional manual extraction methods and other existing models, our method is superior to in terms of test performance.

深度学习结构优化的图像情感分类

深度学习结构优化的图像情感分类

Transfer Correlation Between Textual Content to Images for Sentiment Analysis

In social media, images and texts are used to convey individuals’ attitudes and feelings; thus, social media has become an indispensable part of people’s lives. To understand social behavior and provide better recommendations, sentiment analysis on social media is helpful. One sentiment analysis task is polarity prediction. Although current research on visual or textual sentiment analysis has achieved quite good progress, multimodal and cross-modal analysis combining visual and textual correlation is still in the exploration stage. To capture a semantic connection between images and captions, this paper proposes a cross-modal approach that considers both images and captions in classifying image sentiment polarity. This method transfers the correlation between textual content to images. First, the image and its corresponding caption are sent into an inner-class mapping model, where they are transformed into vectors in Hilbert space to obtain their labels by calculating the inner-class maximum mean discrepancy (MMD). Then, a class-aware sentence representation (CASR) model assigns the distributed representation to the labels with a class-aware attention-based gated recurrent unit (GRU). Finally, an inner-class dependency LSTM (IDLSTM) classifies the sentiment polarity. Experiments carried out on the Getty Images dataset and Twitter 1269 dataset demonstrate the effectiveness of our approach. Moreover, extensive experimental results show that our model outperforms baseline solutions.

深度学习结构优化的图像情感分类

深度学习结构优化的图像情感分类

Multidimensional Extra Evidence Mining for Image Sentiment Analysis

Image sentiment analysis is a hot research topic in the field of computer vision. However, two key issues need to be addressed. First, high-quality training samples are scarce. There are numerous ambiguous images in the original datasets owing to diverse subjective cognitions from different annotators. Second, the cross-modal sentimental semantics among heterogeneous image features has not been fully explored. To alleviate these problems, we propose a novel model called multidimensional extra evidence mining (ME 2 M) for image sentiment analysis, it involves sample-refinement and cross-modal sentimental semantics mining. A new soft voting-based sample-refinement strategy is designed to address the former problem, whereas the state-of-the-art discriminant correlation analysis (DCA) model is used to completely mine the cross-modal sentimental semantics among diverse image features. Image sentiment analysis is conducted based on the cross-modal sentimental semantics and a general classifier. The experimental results verify that the ME 2 M model is effective and robust and that it outperforms the most competitive baselines on two well-known datasets. Furthermore, it is versatile owing to its flexible structure.

Object semantics sentiment correlation analysis enhanced image sentiment classification

With the development of artificial intelligence and deep learning, image sentiment analysis has become a hotspot in computer vision and attracts more attention. Most of the existing methods focus on identifying the emotions by studying complex models or robust features from the whole image, which neglects the influence of object semantics on image sentiment analysis. In this paper, we propose a novel object semantics sentiment correlation model (OSSCM), which is based on Bayesian network, to guide the image sentiment classification. OSSCM is constructed by exploring the relationships between image emotions and the object semantics combination in the images, which can fully consider the effect of object semantics for image emotions. Then, a convolutional neural networks (CNN) based visual sentiment analysis model is proposed to analyze image sentiment from visual aspect. Finally, three fusion strategies are proposed to realize OSSCM enhanced image sentiment classification. Experiments on public emotion datasets FI and Flickr_LDL demonstrate that our proposed image sentiment classification method can achieve good performance on image emotion analysis, and outperform state of the art methods.

Sentiment analysis of multimodal twitter data

Text-driven sentiment analysis has been widely studied in the past decade, on both random and benchmark textual Twitter datasets. Few pertinent studies have also reported visual analysis of images to predict sentiment, but much of the work has analyzed a single modality data, that is either text or image or GIF video. More recently, as the images, memes and GIFs dominate the social feeds; typographic/infographic visual content has become a non-trivial element of social media. This multimodal text combines both text and image defining a novel visual language which needs to be analyzed as it has the potential to modify, confirm or grade the polarity of the sentiment. We propose a multimodal sentiment analysis model to determine the sentiment polarity and score for any incoming tweet, i.e., textual, image or info-graphic and typographic. Image sentiment scoring is done using SentiBank and SentiStrength scoring for Regions with convolution neural network (R-CNN). Text sentiment scoring is done using a novel context-aware hybrid (lexicon and machine learning) technique. Multimodal sentiment scoring is done by separating text from image using an optical character recognizer and then aggregating the independently processed image and text sentiment scores. High performance accuracy of 91.32% is observed for the random multimodal tweet dataset used to evaluate the proposed model. The research further demonstrates that combining both textual and image features outperforms separate models that rely exclusively on either images or text analysis.

Region-based convolutional neural network using group sparse regularization for image sentiment classification

As an information carrier with rich semantics, images contain more sentiment than texts and audios. So, images are increasingly used by people to express their opinions and sentiments in social network. The sentiments of the images are overall and should come from different regions. So, the recognition of the sentiment regions will help to concentrate on important factors the affect the sentiments. Meanwhile, deep learning method for image sentiment classification needs simple and efficient approach for simultaneously carrying out pruning and feature selection whilst optimizing the weights. Motivated by these observations, we design a region-based convolutional neural network using group sparse regularization for image sentiment classification: R-CNNGSR. The method obtains the initial sentiment prediction model through CNN using group sparse regularization to get compact neural network, and then automatically detect the sentiment regions by combining the underlying features and sentimental features. Finally, the whole image and the sentiment region are fused to predict the overall sentiment of the images. Experiment results demonstrate that our proposed R-CNNGSR significantly outperforms the state-of-the-art methods in image sentiment classification.

Boosting image sentiment analysis with visual attention

Abstract Sentiment analysis plays an important role in behavior sciences, which aims to determine the attitude of a speaker or a writer regarding some topic or the overall contextual polarity of a document. The problem nevertheless is not trivial, especially when inferring sentiment or emotion from visual contents, such as images and videos, which are becoming pervasive on the Web. Observing that the sentiment of an image may be reflected only by some spatial regions, a valid question is how to locate the attended spatial areas for enhancing image sentiment analysis. In this paper, we present Sentiment Networks with visual Attention (SentiNet-A) — a novel architecture that integrates visual attention into the successful Convolutional Neural Networks (CNN) sentiment classification framework, by training them in an end-to-end manner. To model visual attention, we develop multiple layers to generate the attention distribution over the regions of the image. Furthermore, the saliency map of the image is employed as a priori knowledge and regularizer to holistically refine the attention distribution for sentiment prediction. Extensive experiments are conducted on both Twitter and ARTphoto benchmarks, and our framework achieves superior results when compared to the state-of-the-art techniques.

Weighted Co-Training for Cross-Domain Image Sentiment Classification

Image sentiment classification, which aims to predict the polarities of sentiments conveyed by the images, has gained a lot of attention. Most existing methods address this problem by training a general classifier with certain visual features, ignoring the discrepancies across domains. In this paper, we propose a novel weighted co-training method for cross-domain image sentiment classification, which iteratively enlarges the labeled set by introducing new high-confidence classified samples to reduce the gap between the two domains. We train two sentiment classifiers with both the images and the corresponding textual comments separately, and set the similarity between the source domain and the target domain as the weight of a classifier. We perform extensive experiments on a real Flickr dataset to evaluate the proposed method, and the empirical study reveals that the weighted co-training method significantly outperforms some baseline solutions.

From pixels to sentiment: Fine-tuning CNNs for visual sentiment prediction

Visual multimedia have become an inseparable part of our digital social lives, and they often capture moments tied with deep affections. Automated visual sentiment analysis tools can provide a means of extracting the rich feelings and latent dispositions embedded in these media. In this work, we explore how Convolutional Neural Networks (CNNs), a now de facto computational machine learning tool particularly in the area of Computer Vision, can be specifically applied to the task of visual sentiment prediction. We accomplish this through fine-tuning experiments using a state-of-the-art CNN and via rigorous architecture analysis, we present several modifications that lead to accuracy improvements over prior art on a dataset of images from a popular social media platform. We additionally present visualizations of local patterns that the network learned to associate with image sentiment for insight into how visual positivity (or negativity) is perceived by the model.

Image sentiment prediction based on textual descriptions with adjective noun pairs

We aim to predict the sentiment related information reflected in images based on SentiBank, which is a library including Adjective Noun Pair (ANP) concept detectors for image sentiment analysis. Instead of using only ANP responses in images as mid-level features, we make full use of the ANPs’ textual sentiment. We first give each ANP concept in SentiBank a sentiment value by adding together the textual sentiment value of the adjective and that of the noun. Having detected the presence of ANPs in an image, we define an image sentiment value by computing the weighted sum of the textual sentiment values of ANPs describing this image with corresponding ANP responses as weights. On the one hand, we adopt a one-dimension logistic regression model to predict the sentiment orientation according to the image sentiment value. On the other hand, we use the ANP responses detected in an image as mid-level representations to train a regularized logistic regression classifier for sentiment prediction. We finally employ a late fusion algorithm to combine the prediction results from the two schemes. Experimental results reveal that the proposed method which takes into account the textual sentiment of ANPs improves the performance of SentiBank based image sentiment prediction.

Visual Sentiment Analysis with Network in Network

In modern society, visual content like images and videos is increasingly becoming a new form of media to express users’ opinions on the Internet. As a complement to textual sentiment analysis, visual sentiment analysis intends to provide more robust information for data analytics by extracting emotion and sentiment toward topics and events from images and videos. Inspired by recent works that applied deep convolutional neural networks (CNN) to this challenging problem, we proposed a framework for image sentiment analysis with a novel deep neural network called Network in Network (NIN) which intends to improve the discriminability for local patches within receptive fields. We trained our network on a dataset consisting of nearly half a million Flickr images and minimized the effect of noisy training data by fine-tuning the network in a progressive manner. Extensive experiments conducted on manually labeled Twitter images show that the proposed architecture performs better in visual sentiment analysis than conventional CNN and other traditional algorithms.

Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks

Sentiment analysis of online user generated content is important for many social media analytics tasks. Researchers have largely relied on textual sentiment analysis to develop systems to predict political elections, measure economic indicators, and so on. Recently, social media users are increasingly using images and videos to express their opinions and share their experiences. Sentiment analysis of such large scale visual content can help better extract user sentiments toward events or topics, such as those in image tweets, so that prediction of sentiment from visual content is complementary to textual sentiment analysis. Motivated by the needs in leveraging large scale yet noisy training data to solve the extremely challenging problem of image sentiment analysis, we employ Convolutional Neural Networks (CNN). We first design a suitable CNN architecture for image sentiment analysis. We obtain half a million training samples by using a baseline sentiment algorithm to label Flickr images. To make use of such noisy machine labeled data, we employ a progressive strategy to fine-tune the deep network. Furthermore, we improve the performance on Twitter images by inducing domain transfer with a small number of manually labeled Twitter images. We have conducted extensive experiments on manually labeled Twitter images. The results show that the proposed CNN can achieve better performance in image sentiment analysis than competing algorithms.

Visual sentiment topic model based microblog image sentiment analysis

With a growing number of images being used to express opinions in Microblog, text based sentiment analysis is not enough to understand the sentiments of users. To obtain the sentiments implied in M...