Sentiment Representation Research Articles

Semantic relational machine learning model for sentiment analysis using cascade feature selection and heterogeneous classifier ensemble.

The exponential rise in social media via microblogging sites like Twitter has sparked curiosity in sentiment analysis that exploits user feedback towards a targeted product or service. Considering its significance in business intelligence and decision-making, numerous efforts have been made in this area. However, lack of dictionaries, unannotated data, large-scale unstructured data, and low accuracies have plagued these approaches. Also, sentiment classification through classifier ensemble has been underexplored in literature. In this article, we propose a Semantic Relational Machine Learning (SRML) model that automatically classifies the sentiment of tweets by using classifier ensemble and optimal features. The model employs the Cascaded Feature Selection (CFS) strategy, a novel statistical assessment approach based on Wilcoxon rank sum test, univariate logistic regression assisted significant predictor test and cross-correlation test. It further uses the efficacy of word2vec-based continuous bag-of-words and n-gram feature extraction in conjunction with SentiWordNet for finding optimal features for classification. We experiment on six public Twitter sentiment datasets, the STS-Gold dataset, the Obama-McCain Debate (OMD) dataset, the healthcare reform (HCR) dataset and the SemEval2017 Task 4A, 4B and 4C on a heterogeneous classifier ensemble comprising fourteen individual classifiers from different paradigms. Results from the experimental study indicate that CFS supports in attaining a higher classification accuracy with up to 50% lesser features compared to count vectorizer approach. In Intra-model performance assessment, the Artificial Neural Network-Gradient Descent (ANN-GD) classifier performs comparatively better than other individual classifiers, but the Best Trained Ensemble (BTE) strategy outperforms on all metrics. In inter-model performance assessment with existing state-of-the-art systems, the proposed model achieved higher accuracy and outperforms more accomplished models employing quantum-inspired sentiment representation (QSR), transformer-based methods like BERT, BERTweet, RoBERTa and ensemble techniques. The research thus provides critical insights into implementing similar strategy into building more generic and robust expert system for sentiment analysis that can be leveraged across industries.

Aspect sentiment analysis with heterogeneous graph neural networks

Aspect-based sentiment analysis technologies may be a very practical methodology for securities trading, commodity sales, movie rating websites, etc. Most recent studies adopt the recurrent neural network or attention-based neural network methods to infer aspect sentiment using opinion context terms and sentence dependency trees. However, due to a sentence often having multiple aspects sentiment representation, these models are hard to achieve satisfactory classification results. In this paper, we discuss these problems by encoding sentence syntax tree, words relations and opinion dictionary information in a unified framework. We called this method heterogeneous graph neural networks (Hete_GNNs). Firstly, we adopt the interactive aspect words and contexts to encode the sentence sequence information for parameter sharing. Then, we utilized a novel heterogeneous graph neural network for encoding these sentences’ syntax dependency tree, prior sentiment dictionary, and some part-of-speech tagging information for sentiment prediction. We perform the Hete_GNNs sentiment judgment and report the experiments on five domain datasets, and the results confirm that the heterogeneous context information can be better captured with heterogeneous graph neural networks. The improvement of the proposed method is demonstrated by aspect sentiment classification task comparison.

Syntactic Knowledge Embedding Network for Aspect-Based Sentiment Classification

An aspect-based sentiment classification task is a fine-grained sentiment analysis task, which is aimed at identifying the sentiment polarity of a given aspect in subjective sentences. In recent years, some researchers have applied pretrained BERT models to this task. However, existing research only uses the BERT output layer and ignores the syntactic features in the middle layers, leading to deviations in the prediction results. In order to solve above problems, we propose a new model BERT-SFE. Firstly, we explicitly utilize the middle layers of BERT to capture the underlying syntactic features. Secondly, we construct a syntactic feature extraction unit based on Star-Transformer, which uses an auxiliary vector and the star network structure to capture both local and global syntactic information in a sentence. Finally, we merge the syntactic features with the semantic features from the BERT output layer in the feature fusion layer, obtaining a more accurate sentiment representation of the aspect. The experimental results on three public ABSA datasets show that using the syntactic feature extraction unit based on Star-Transformer to mine the syntactic knowledge in the middle layers of BERT can effectively improve the accuracy of sentiment classification. BERT-SFE achieves the best performance compared with existing models.

Sentiments Analysis and Text Summarization of Medicine Reviews Using Deep Sequential Iterative Model with Attention Encoder

In this day and age of data innovation, vast amounts of data are being generated daily, much of it is useless to the general public unless it is correctly handled. This year has seen a huge increase in social networks’ relevance, resulting in vast quantities of data provided by users. This vast amount of information is gathered from a variety of sources, including company websites, customer blogs, and item reviews. The text outline is probably the most significant point of view in daily life. Analysis of text sentiment is a method for assessing, summarising, and drawing conclusions about the most important content. In the field of sentiment analysis, attention methods have been crucial since they make use of sentiment lexicons to gather a huge amount of sentiment polarity information. With an attention mechanism that acts as a link between linguistic information with a strong emotional component and deep learning algorithms, it may be possible to boost text sentiment significantly. In the LSTM model, word sequence addictions can be captured over the long term. For the first time, scientists have developed an attention model that combines LSTM with an incredibly deep RNN model to tackle the problem of sentiment analysis in the real world. The iterative method trains the first set of word embeddings using a Word to Vector technique. With the Word2Vec algorithm, text strings are converted into a numerical value vector and distance between words and comparative words based on meaning are calculated. Using the attention strategy has the added benefit of potentially enhancing machine learning’s ability to learn sentiment representations. The attention model is more scalable and adaptable than previous approaches. The major objective of this work is to assess feelings and develop an abstracted content outline, decide on the semantic summarization of different materials and effectively process the data.

Feature discrimination and diversification for image emotion recognition

Emotion is strongly subjective, and different parts of the image may have a different degree of impact on emotion. The key for solving image emotion recognition is to effectively mine different discriminative local regions. We present a deep architecture to guide the network to extract discriminative and diverse affective semantic information. First, training a full convolutional network with a cross-channel max pooling strategy (CCMP) to extract discriminative feature maps. Second, to ensure that most of the discriminative sentiment regions are located accurately, we add a module consisting of the convolution layer and the CCMP. After obtaining the discriminative regions of the first module, the feature elements corresponding to the discriminant regions are erased, and then the erased features are fed into the second module. Such adversarial erasure operation can force the network to discover different sentiment discriminative regions. Third, an adaptive feature fusion mechanism is proposed to better integrate discriminative and diverse sentiment representations. Sufficient experiments are conducted on the benchmark datasets FI, EmotionROI, Instagram, and Twitter1 to achieve 72.17%, 61.13%, 81.97%, and 85.44% recognition accuracies, respectively. The results of the experiment demonstrate that the proposed network outperforms the state-of-the-art results.

Applying Existing Datasets as a Pseudo Corpus for Sentiment Representation on Social Media

This paper proposes a method to represent the sentiment characteristics of opinions on social media by using some datasets as a pseudo corpus without any annotations. The widespread social media enables us to easily share our own opinions on the Web and communicate with each other. The more the data on social media increase, the more the demands for analysis of the data increase, e.g., text classification and sentiment analysis. Usually, the annotated data should be required in the existing text classification using the supervised machine learning method. However, it is reasonable to say that the criteria for the annotated labels should differ for each period, culture, and independent sense. Effective text classification for such different criteria needs the different types of annotations corresponding to each measure, and it requires much time and human resources. A pseudo corpus consists of multiple existing datasets with different characteristics in the proposed method. The classification model for each dataset is obtained as learning the pseudo corpus. The sentiment of the input text, which domain is different from the learned datasets, is represented as the likelihood distribution for the datasets in the pseudo corpus. This paper discusses the potential and limitations of this idea through the experiment.

An Improved Model for Analyzing Textual Sentiment Based on a Deep Neural Network Using Multi-Head Attention Mechanism

Due to the increasing growth of social media content on websites such as Twitter and Facebook, analyzing textual sentiment has become a challenging task. Therefore, many studies have focused on textual sentiment analysis. Recently, deep learning models, such as convolutional neural networks and long short-term memory, have achieved promising performance in sentiment analysis. These models have proven their ability to cope with the arbitrary length of sequences. However, when they are used in the feature extraction layer, the feature distance is highly dimensional, the text data are sparse, and they assign equal importance to various features. To address these issues, we propose a hybrid model that combines a deep neural network with a multi-head attention mechanism (DNN–MHAT). In the DNN–MHAT model, we first design an improved deep neural network to capture the text’s actual context and extract the local features of position invariants by combining recurrent bidirectional long short-term memory units (Bi-LSTM) with a convolutional neural network (CNN). Second, we present a multi-head attention mechanism to capture the words in the text that are significantly related to long space and encoding dependencies, which adds a different focus to the information outputted from the hidden layers of BiLSTM. Finally, a global average pooling is applied for transforming the vector into a high-level sentiment representation to avoid model overfitting, and a sigmoid classifier is applied to carry out the sentiment polarity classification of texts. The DNN–MHAT model is tested on four reviews and two Twitter datasets. The results of the experiments illustrate the effectiveness of the DNN–MHAT model, which achieved excellent performance compared to the state-of-the-art baseline methods based on short tweets and long reviews.

Learning Representation From Concurrence-Words Graph For Aspect Sentiment Classification

Abstract Aspect sentiment classification is an important research topic in natural language processing and computational linguistics, assisting in automatically review analysis and emotional tendency judgement. Different from extant methods that focus on text sequence representations, this paper presents a network framework to learn representation from concurrence-words relation graph (LRCWG), so as to improve the Macro-F1 and accuracy. The LRCWG first employs the multi-head attention mechanism to capture the sentiment representation from the sentences which can learn the importance of text sequence representation. And then, it leverages the priori sentiment dictionary information to construct the concurrence relations of sentiment words with Graph Convolution Network (GCN). This assists in that the learnt context representation can keep both the semantics integrity and the features of sentiment concurrence-words relations. The designed algorithm is experimentally evaluated with all the five benchmark datasets and demonstrated that the proposed aspect sentiment classification can significantly improve the prediction performance of learning task.

Attention-Emotion-Enhanced Convolutional LSTM for Sentiment Analysis.

Long short-term memory (LSTM) neural networks and attention mechanism have been widely used in sentiment representation learning and detection of texts. However, most of the existing deep learning models for text sentiment analysis ignore emotion's modulation effect on sentiment feature extraction, and the attention mechanisms of these deep neural network architectures are based on word- or sentence-level abstractions. Ignoring higher level abstractions may pose a negative effect on learning text sentiment features and further degrade sentiment classification performance. To address this issue, in this article, a novel model named AEC-LSTM is proposed for text sentiment detection, which aims to improve the LSTM network by integrating emotional intelligence (EI) and attention mechanism. Specifically, an emotion-enhanced LSTM, named ELSTM, is first devised by utilizing EI to improve the feature learning ability of LSTM networks, which accomplishes its emotion modulation of learning system via the proposed emotion modulator and emotion estimator. In order to better capture various structure patterns in text sequence, ELSTM is further integrated with other operations, including convolution, pooling, and concatenation. Then, topic-level attention mechanism is proposed to adaptively adjust the weight of text hidden representation. With the introduction of EI and attention mechanism, sentiment representation and classification can be more effectively achieved by utilizing sentiment semantic information hidden in text topic and context. Experiments on real-world data sets show that our approach can improve sentiment classification performance effectively and outperform state-of-the-art deep learning-based methods significantly.

Refined Global Word Embeddings Based on Sentiment Concept for Sentiment Analysis

Sentiment Analysis is an important research direction of natural language processing, and it is widely used in politics, news and other fields. Word embeddings play a significant role in sentiment analysis. The existing sentiment embeddings methods directly embed the sentiment lexicons into traditional word representation. This sentiment representation methods can only differentiate the sentiment information of different words, not the same word in different contexts, so it cannot provide accurate sentiment information for word in different contexts. This paper proposes sentiment concept to solve the problem. First, we found the optimal sentiment concept of words in Microsoft Concept Graph according to the context of words. Then we obtained the sentiment information of words under optimal sentiment concept from the multi-semantics sentiment intensity lexicon which we constructed in this paper to achieve accurate embedding of sentiment information and provide more accurate semantics and sentiment representation for words. Finally, we combined two refined word embeddings methods to achieve a more comprehensive word representation. Compared with traditional and sentiment embeddings methods on six representative datasets, the validity of the word embeddings method based on sentiment concept proposed in this paper is verified.

Toward Aspect-Level Sentiment Modification Without Parallel Data

This article takes the lead to study aspect-level sentiment modification (ALSM) without parallel data. Given a sentence, the task of ALSM needs to reverse the sentiment with respect to the given aspect while preserving other content. The main challenge is reversing the sentiment of the given aspect without affecting the sentiments of other aspects in the sentences. To handle this problem, we propose a joint aspect-level sentiment modification (called JASM) model. JASM is a multitask system, which jointly trains two coupled modules: aspect-specific sentiment words extraction and aspect-level sentiment transformation. Besides, we propose a novel memory mechanism to learn aspect-aware sentiment representation and a gating mechanism to dynamically select aspect-aware sentiment information or content information for generating the next words. Experiments show that the proposed model substantially outperforms the compared methods in both aspect-level sentiment transformation and content preservation. For applications, we conduct data augmentation for aspect-based sentiment analysis (ABSA) through generating plausible training data with the trained ALSM model. Experiments show that data augmentation with generated data boosts the performance of a broad range of ABSA models.

Two-Level LSTM for Sentiment Analysis With Lexicon Embedding and Polar Flipping.

Sentiment analysis is a key component in various text mining applications. Numerous sentiment classification techniques, including conventional and deep-learning-based methods, have been proposed in the literature. In most existing methods, a high-quality training set is assumed to be given. Nevertheless, constructing a high-quality training set that consists of highly accurate labels is challenging in real applications. This difficulty stems from the fact that text samples usually contain complex sentiment representations, and their annotation is subjective. We address this challenge in this study by leveraging a new labeling strategy and utilizing a two-level long short-term memory network to construct a sentiment classifier. Lexical cues are useful for sentiment analysis, and they have been utilized in conventional studies. For example, polar and negation words play important roles in sentiment analysis. A new encoding strategy, that is, ρ -hot encoding, is proposed to alleviate the drawbacks of one-hot encoding and, thus, effectively incorporate useful lexical cues. Moreover, the sentimental polarity of a word may change in different sentences due to label noise or context. A flipping model is proposed to model the polar flipping of words in a sentence. We compile three Chinese datasets on the basis of our label strategy and proposed methodology. Experiments demonstrate that the proposed method outperforms state-of-the-art algorithms on both benchmark English data and our compiled Chinese data.

WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection

Automatic assessment of sentiment from visual content has gained considerable attention with the increasing tendency of expressing opinions online. In this paper, we solve the problem of visual sentiment analysis, which is challenging due to the high-level abstraction in the recognition process. Existing methods based on convolutional neural networks learn sentiment representations from the holistic image, despite the fact that different image regions can have different influence on the evoked sentiment. In this paper, we introduce a weakly supervised coupled convolutional network (WSCNet). Our method is dedicated to automatically selecting relevant soft proposals given weak annotations (e.g., global image labels), thereby significantly reducing the annotation burden, and encompasses the following contributions. First, the proposed WSCNet detects a sentiment-specific soft map by training a fully convolutional network with the cross spatial pooling strategy in the detection branch. Second, both the holistic and localized information are utilized by coupling the sentiment map with deep features as semantic vector in the classification branch. The sentiment detection and classification branches are integrated into a unified deep framework optimized in an end-to-end manner. Extensive experiments demonstrate that the proposed WSCNet outperforms the state-of-the-art results on seven benchmark datasets.

SHE: Sentiment Hashtag Embedding Through Multitask Learning

Recent studies have shown the importance of utilizing hashtags for sentiment analysis task on social media data. However, as the hashtag generation process is less restrictive, it throws several challenges, such as hashtag normalization, topic modeling, and semantic similarity. Recently, researchers have tried to address the above-mentioned challenges through representation learning. However, most of the studies on hashtag embedding try to capture the semantic distribution of hashtags and often fail to capture the sentiment polarity. Furthermore, generating a task-specific hashtag embedding can distort its semantic representation, which is undesirable for sentiment representation of hashtag. Therefore, this article proposes a semisupervised sentiment hashtag embedding (SHE) model, which is capable of preserving both semantic as well as sentiment distribution of the hashtags. In particular, SHE leverages a multitask learning approach using an autoencoder and a convolutional neural network-based classifier. To assess the efficacy of hashtag embedding, we compare the performance of SHE against suitable baselines for three different tasks, namely, hashtag sentiment classification, tweet sentiment classification, and retrieval of semantically similar hashtags. It is evident from various experimental results that SHE outperforms the majority of the baselines with significant margins.

ReMemNN: A novel memory neural network for powerful interaction in aspect-based sentiment analysis

Deep neural networks have been employed to analyze the sentiment of text sequences and achieved significant effect. However, these models still face the issues of weakness of pre-trained word embeddings and weak interaction between the specific aspect and the context in attention mechanism. The pre-trained word embeddings lack the specific semantic information from the context. The weak interaction results in poor attention weights and produces limited aspect dependent sentiment representation in aspect-based sentiment analysis (ABSA). In this paper, we propose a novel end-to-end memory neural network, termed Recurrent Memory Neural Network (ReMemNN), to mitigate the above-mentioned problems. In ReMemNN, to tackle weakness of pre-trained word embeddings, a specially module named embedding adjustment learning module is designed to transfer the pre-trained word embeddings into adjustment word embeddings. To tackle weak interaction in attention mechanism, a multielement attention mechanism is designed to generate powerful attention weights and more precise aspect dependent sentiment representation. Besides, an explicit memory module is designed to store these different representations and generate hidden states and representations. Extensive experimental results on all datasets show that ReMemNN outperforms typical baselines and achieve the state-of-the-art performance. Besides, these experimental results also demonstrate that ReMemNN is language-independent and dataset type-independent.

Correction: OpinionML—Opinion Markup Language for Sentiment Representation. Symmetry 2019, 11, 545

The authors wish to make the following corrections to their paper [...]

Learning Discriminative Sentiment Representation from Strongly- and Weakly Supervised CNNs

Visual sentiment analysis is attracting increasing attention with the rapidly growing amount of images uploaded to social networks. Learning rich visual representations often requires training deep convolutional neural networks (CNNs) on massive manually labeled data, which is expensive or scarce especially for a subjective task like visual sentiment analysis. Meanwhile, a large quantity of social images is quite available yet noisy by querying social networks using the sentiment categories as keywords, where various types of images related to the specific sentiment can be easily collected. In this article, we propose a multiple kernel network for visual sentiment recognition, which learns representation from strongly- and weakly supervised CNNs. Specifically, the weakly supervised deep model is trained using the large-scale data from social images, whereas the strongly supervised deep model is fine tuned on the affecitve datasets with manual annotation. We employ the multiple kernel scheme on the multiple layers of CNNs, which can automatically select the discriminative representation by learning a linear combination from a set of pre-defined kernels. In addition, we introduce a large-scale dataset collected from popular comics of various countries, such as America, Japan, China, and France, which consists of 11,821 images with various artistic styles. Experimental results show that the multiple kernel network achieves consistent improvements over the state-of-the-art methods on the public affective datasets, as well as the newly established Comics dataset. The Comics dataset can be found at http://cv.nankai.edu.cn/projects/Comic.

Aspect Aware Learning for Aspect Category Sentiment Analysis

Aspect category sentiment analysis (ACSA) is an underexploited subtask in aspect level sentiment analysis. It aims to identify the sentiment of predefined aspect categories. The main challenge in ACSA comes from the fact that the aspect category may not occur in the sentence in most of the cases. For example, the review “ they have delicious sandwiches ” positively talks about the aspect category “ food ” in an implicit manner. In this article, we propose a novel aspect aware learning (AAL) framework for ACSA tasks. Our key idea is to exploit the interaction between the aspect category and the contents under the guidance of both sentiment polarity and predefined categories. To this end, we design a two-way memory network for integrating AAL into the framework of sentiment classification. We further present two algorithms to incorporate the potential impacts of aspect categories. One is to capture the correlations between aspect terms and the aspect category like “sandwiches” and “food.” The other is to recognize the aspect category for sentiment representations like “food” for “delicious.” We conduct extensive experiments on four SemEval datasets. The results reveal the essential role of AAL in ACSA by achieving the state-of-the-art performance.

Interactive Attention Transfer Network for Cross-Domain Sentiment Classification

Cross-domain sentiment classification refers to utilizing useful knowledge in the source domain to help sentiment classification in the target domain which has few or no labeled data. Most existing methods mainly concentrate on extracting common features between domains. Unfortunately, they cannot fully consider the effects of the aspect (e.g., the battery life in reviewing an electronic product) information of the sentences. In order to better solve this problem, we propose an Interactive Attention Transfer Network (IATN) for crossdomain sentiment classification. IATN provides an interactive attention transfer mechanism, which can better transfer sentiment across domains by incorporating information of both sentences and aspects. Specifically, IATN comprises two attention networks, one of them is to identify the common features between domains through domain classification, and the other aims to extract information from the aspects by using the common features as a bridge. Then, we conduct interactive attention learning for those two networks so that both the sentences and the aspects can influence the final sentiment representation. Extensive experiments on the Amazon reviews dataset and crowdfunding reviews dataset not only demonstrate the effectiveness and universality of our method, but also give an interpretable way to track the attention information for sentiment.

Multi-layer Attention Based CNN for Target-Dependent Sentiment Classification

Target-dependent sentiment classification aims at identifying the sentiment polarities of targets in a given sentence. Previous approaches utilize recurrent neural network with attention mechanism incorporated to model the context and learn key sentiment intermediate representation in relation to a given target. However, such methods are incapable either of modeling complex contexts or of processing data parallelly. To address these problems, we propose, in this paper, a new model that employs a multi-layer convolutional neural network to process the context parallelly and model the context multiple times, where the neural network is able to explicitly learn the sentiment intermediate representation via an attention mechanism. Eventually, we integrate these features to form a final sentiment representation, which will be fed into the classifier. Experiments show that our model surpasses the existing approaches on several datasets.