Sentiment Knowledge Research Articles

Knowledge-enhanced neural networks for sentiment analysis of Chinese reviews

Sentiment analysis aims to extract structured opinions from unstructured reviews and determine their sentiment polarities. However, existing sentiment analysis systems fail to identify aspect-opinion pairs and perform poorly on small training corpora. To address these issues, we propose a novel framework to model aspect-opinion pair identification and aspect-level sentiment classification as a joint text classification task. Moreover, we incorporate external knowledge into neural networks to compensate for the lack of training data. In our approach, context features extracted from review sentences and external knowledge retrieved from a sentiment knowledge graph are used to identify aspect-opinion pairs and determine their sentiment polarities. In this way, our model is able to provide more detailed sentiment analysis results and achieve better performance with limited training corpora. We evaluate our approach using a Chinese car review dataset. Experimental results show that the knowledge-enhanced neural networks consistently outperform the conventional models.

Aspect-based sentiment analysis via fusing multiple sources of textual knowledge

The aim of aspect-based sentiment analysis (ABSA) is to predict sentiment polarity of text toward a specific aspect. Although existing neural network models show promising performances on ABSA, their capabilities can be unsatisfactory in cases where the amount of training data is limited. In this paper, we propose a unified model which exploits and incorporates multiple sources of knowledge to improve its ability on ABSA. Structure knowledge is extracted via clause recognition and fused in the model through the generation of multiple context representations to force the model to capture aspect-specific context information. Sentiment knowledge is exploited by means of training a general classification model with the sentiment labels of documents and fused through pretraining specific layers to extract contextual features and predict sentiment polarities more accurately. In addition, information of conjunctions is fused in our model to capture the relations between clauses and provide additional sentiment features. Experimental results on five publicly available ABSA datasets validate the effectiveness of our method and prove that multiple sources of knowledge can collaboratively enhance our model.

Multi-source data fusion for aspect-level sentiment classification

Neural networks have achieved great success in aspect-level sentiment classification due to their ability to learn sentiment knowledge from text. Generally, the effectiveness of neural networks relies on sufficiently large training corpora. However, existing aspect-level corpora are relatively small, which greatly limits the performance of neural network-based systems. In this paper, we propose a novel approach to aspect-level sentiment classification based on multi-source data fusion, which allows our system to learn sentiment knowledge from different types of resources. Specifically, we design a unified framework to integrate data from aspect-level corpora, sentence-level corpora, and word-level sentiment lexicons. Moreover, we take advantage of BERT, a pre-trained language model based on deep bidirectional Transformers, to generate aspect-specific sentence representations for sentiment classification. We evaluate our approach using laptop and restaurant datasets from SemEval 2014. Experimental results show that our approach consistently outperforms the state-of-the-art methods on all datasets.

Irony detection via sentiment-based transfer learning

Irony as a literary technique is widely used in online texts such as Twitter posts. Accurate irony detection is crucial for tasks such as effective sentiment analysis. A text’s ironic intent is defined by its context incongruity. For example in the phrase “I love being ignored”, the irony is defined by the incongruity between the positive word “love” and the negative context of “being ignored”. Existing studies mostly formulate irony detection as a standard supervised learning text categorization task, relying on explicit expressions for detecting context incongruity. In this paper we formulate irony detection instead as a transfer learning task where supervised learning on irony labeled text is enriched with knowledge transferred from external sentiment analysis resources. Importantly, we focus on identifying the hidden, implicit incongruity without relying on explicit incongruity expressions, as in “I like to think of myself as a broken down Justin Bieber – my philosophy professor.” We propose three transfer learning-based approaches to using sentiment knowledge to improve the attention mechanism of recurrent neural models for capturing hidden patterns for incongruity. Our main findings are: (1) Using sentiment knowledge from external resources is a very effective approach to improving irony detection; (2) For detecting implicit incongruity, transferring deep sentiment features seems to be the most effective way. Experiments show that our proposed models outperform state-of-the-art neural models for irony detection.

Deep learning-based sentiment classification of evaluative text based on Multi-feature fusion

Sentiment analysis concerns the study of opinions expressed in a text. Due to the huge amount of reviews, sentiment analysis plays a basic role to extract significant information and overall sentiment orientation of reviews. In this paper, we present a deep-learning-based method to classify a user's opinion expressed in reviews (called RNSA).To the best of our knowledge, a deep learning-based method in which a unified feature set which is representative of word embedding, sentiment knowledge, sentiment shifter rules, statistical and linguistic knowledge, has not been thoroughly studied for a sentiment analysis. The RNSA employs the Recurrent Neural Network (RNN) which is composed by Long Short-Term Memory (LSTM) to take advantage of sequential processing and overcome several flaws in traditional methods, where order and information about the word are vanished. Furthermore, it uses sentiment knowledge, sentiment shifter rules and multiple strategies to overcome the following drawbacks: words with similar semantic context but opposite sentiment polarity; contextual polarity; sentence types; word coverage limit of an individual lexicon; word sense variations. To verify the effectiveness of our work, we conduct sentence-level sentiment classification on large-scale review datasets. We obtained encouraging result. Experimental results show that (1) feature vectors in terms of (a) statistical, linguistic and sentiment knowledge, (b) sentiment shifter rules and (c) word-embedding can improve the classification accuracy of sentence-level sentiment analysis; (2) our method that learns from this unified feature set can obtain significant performance than one that learns from a feature subset; (3) our neural model yields superior performance improvements in comparison with other well-known approaches in the literature.

Adding Prior Knowledge in Hierarchical Attention Neural Network for Cross Domain Sentiment Classification

Domain adaptation tasks have raised much attention in recent years, especially, the task of cross-domain sentiment classification (CDSC). Due to the domain discrepancy, a sentiment classifier trained in a source domain often performs less well, when directly applied to a target domain. Adversarial neural networks have been used in mainstream approaches for learning domain independent features, such as pivots, which are words with the same sentiment polarity in different domains. However, domain specific features can often determine sentiment in its context. In this paper, we propose a hierarchical attention network with prior knowledge information (HANP) for the CDSC task. Unlike other existing methods, the HANP can obtain both domain independent and domain specific features at the same time by adding prior knowledge. In addition, the HANP also includes a hierarchical representation layer with attention mechanism, so that the HANP can capture important words and sentences in relation to sentiment. Moreover, the proposed model can offer a direct visualization of the sentimental prior knowledge. The experiments on the Amazon review datasets demonstrate that the proposed HANP can significantly outperform the state-of-the-art methods.

Linking Twitter Sentiment Knowledge with Infrastructure Development

Sentiment analysis is related to the automatic extraction of positive or negative opinions from the text. It is a special text mining application. It is important to classify implicit contents from citizen’s tweet using sentiment analysis. This research aimed to find out the opinion of infrastructure that sustained urban development in Surabaya, Indonesia’s second largest city. The procedures of text mining analysis were the data undergoes some preprocessing first, such as removing the link, retweet (RT), username, punctuation, digits, stopwords, case folding, and tokenizing. Then, the opinion was classified into positive and negative comments. Classification methods used in this research were support vector machine (SVM) and neural network (NN). The result of this research showed that NN classification method was better than SVM.

A semantic similarity-based perspective of affect lexicons for sentiment analysis

Lexical resources are widely popular in the field of Sentiment Analysis, as they represent a resource that directly encodes sentimental knowledge. Usually sentiment lexica are used for polarity estimation through the matching of words contained in a text and their associated lexicon sentiment polarities. Nevertheless, such resources have limitations in vocabulary coverage and domain adaptation. Besides, many recent techniques exploit the concept of distributed semantics, normally through word embeddings. In this work, a semantic similarity metric is computed between text words and lexica vocabulary. Using this metric, this paper proposes a sentiment classification model that uses the semantic similarity measure in combination with embedding representations. In order to assess the effectiveness of this model, we perform an extensive evaluation. Experiments show that the proposed method can improve Sentiment Analysis performance over a strong baseline, being this improvement statistically significant. Finally, some characteristics of the proposed technique are studied, showing that the selection of lexicon words has an effect in cross-dataset performance.

Transferring sentiment knowledge between words and tweets

Transferring sentiment knowledge between words and tweets

Lexicon-based approach outperforms Supervised Machine Learning approach for Urdu Sentiment Analysis in multiple domains

Web is facilitating people to express their views and opinions on different topics through reviews and blogs. Effective advantages can be reaped from these reviews and blogs by fusing the sentiment knowledge. In this research, Sentiment Analysis of Urdu blogs from multiple domains is done by using the two widely used approaches i.e. the Lexicon-based approach and the Supervised Machine Learning approach. Three well known classifiers i.e. Support Vector Machine, Decision Tree and K Nearest Neighbor are used in case of Supervised Machine Learning approach whereas a wide coverage Urdu Sentiment Lexicon and an efficient Urdu Sentiment Analyzer are used in Lexicon-based approach. In both the approaches the information are fused from two sources to successfully perform Sentiment Analysis. In case of Lexicon-based approach, the two sources are the wide coverage Urdu Sentiment Lexicon and the efficient Urdu Sentiment Analyzer. In case of Supervised Machine Learning approach, the two sources are the un-annotated data and annotated data along with important attributes. After performing Sentiment Analysis using both the approaches, the results are observed carefully and on the basis of experiments performed in this research, it is concluded that the Lexicon-based approach outperforms Supervised Machine Learning approach not only in terms of Accuracy, Precision, Recall and F-measure but also in terms of economy of time and efforts used.

Extracting sentiment knowledge from pros/cons product reviews: Discovering features along with the polarity strength of their associated opinions

Sentiment knowledge extraction is a growing area of research in the literature. It helps in analyzing users’ opinions about different entities or events, which can then be utilized by analysts for various purposes. Particularly, feature-based sentiment analysis is one of the challenging research areas that analyzes users’ opinions on various features of a product or service. Of the three formats for the product reviews, our focus in this paper is limited to analyzing the pros/cons type. Due to the nature of pros/cons reviews, they are mostly concise and follow a different structure from other review types. Therefore, specialized techniques are needed to analyze these reviews and extract the customers’ discussed product features along with their personal attitudes. In this paper, we propose the Pros/Cons Sentiment Analyzer (PCSA) framework that exploits dependency relations in extracting sentiment knowledge from pros/cons reviews. We also utilize two different lexicons to ascertain the polarity strength of the extracted features based on the customers’ opinions. Several experiments are conducted to evaluate the performance of PCSA in its different phases.

Machine learning-based multi-documents sentiment-oriented summarization using linguistic treatment

Sentiment summarization is the process of automatically creating a compressed version of the opinionated information expressed in a text. This paper presents a machine learning-based approach to summarize user's opinion expressed in reviews using: (1) Sentiment knowledge to calculate a sentence sentiment score as one of the features for sentence-level classification. It integrates multiple strategies to tackle the following problems: sentiment shifter, the types of sentences and word coverage limit. (2) Word embedding model, a deep-learning-inspired method to understand meaning and semantic relationships among words and to extract a vector representation for each word. (3) Statistical and linguistic knowledge to determine salient sentences.The proposed method combines several types of features into a unified feature set to design a more accurate classification system (“True”: the extractive reference summary; “False”: otherwise). Thus, to achieve better performance scores, we carried out a performance study of four well-known feature selection techniques and seven of the most famous classifiers to select the most relevant set of features and find an efficient machine learning classifier, respectively. The proposed method is applied to three different datasets and the results show the integration of support vector machine-based classification method and Information Gain (IG) as a feature selection technique can significantly improve the performance and make the method comparable to other existing methods. Furthermore, our method that learns from this unified feature set can obtain better performance than one that learns from a feature subset.

Weakly supervised topic sentiment joint model with word embeddings

Topic sentiment joint model aims to deal with the problem about the mixture of topics and sentiment simultaneously from online reviews. Most of existing topic sentiment modeling algorithms are mainly based on the state-of-art latent Dirichlet allocation (LDA) and probabilistic latent semantic analysis (PLSA), which infer sentiment and topic distributions from the co-occurrence of words. These methods have been proposed and successfully used for topic and sentiment analysis. However, when the training corpus is small or when the documents are short, the textual features become sparse, so that the results of the sentiment and topic distributions might be not very satisfied. In this paper, we propose a novel topic sentiment joint model called weakly supervised topic sentiment joint model with word embeddings (WS-TSWE), which incorporates word embeddings and HowNet lexicon simultaneously to improve the topic identification and sentiment recognition. The main contributions of WS-TSWE include the following two aspects. (1) Existing models generate the words only from the sentiment-topic-to-word Dirichlet multinomial component, but the WS-TSWE model replaces it with a mixture of two components, a Dirichlet multinomial component and a word embeddings component. Since the word embeddings are trained on a very large corpora and can be used to extend the semantic information of the words, they can provide a certain solution for the problem of the textual sparse. (2) Most of previous models incorporate sentiment knowledge in the β priors. And the priors are usually set from a dictionary and completely rely on previous domain knowledge to identify positive and negative words. In contrast, the WS-TSWE model calculates the sentiment orientation of each word with the HowNet lexicon and automatically infers sentiment-based β priors for sentiment analysis and opinion mining. Furthermore, we implement WS-TSWE with Gibbs sampling algorithms. The experimental results on Chinese and English data sets show that WS-TSWE achieved significant performance in the task of detecting sentiment and topics simultaneously.

Learning Word Representations for Sentiment Analysis

Word embedding has been proven to be a useful model for various natural language processing tasks. Traditional word embedding methods merely take into account word distributions independently from any specific tasks. Hence, the resulting representations could be sub-optimal for a given task. In the context of sentiment analysis, there are various types of prior knowledge available, e.g., sentiment labels of documents from available datasets or polarity values of words from sentiment lexicons. We incorporate such prior sentiment information at both word level and document level in order to investigate the influence each word has on the sentiment label of both target word and context words. By evaluating the performance of sentiment analysis in each category, we find the best way of incorporating prior sentiment information. Experimental results on real-world datasets demonstrate that the word representations learnt by DLJT2 can significantly improve the sentiment analysis performance. We prove that incorporating prior sentiment knowledge into the embedding process has the potential to learn better representations for sentiment analysis.

Collaboratively Training Sentiment Classifiers for Multiple Domains

We propose a collaborative multi-domain sentiment classification approach to train sentiment classifiers for multiple domains simultaneously. In our approach, the sentiment information in different domains is shared to train more accurate and robust sentiment classifiers for each domain when labeled data is scarce. Specifically, we decompose the sentiment classifier of each domain into two components, a global one and a domain-specific one. The global model can capture the general sentiment knowledge and is shared by various domains. The domain-specific model can capture the specific sentiment expressions in each domain. In addition, we extract domain-specific sentiment knowledge from both labeled and unlabeled samples in each domain and use it to enhance the learning of domain-specific sentiment classifiers. Besides, we incorporate the similarities between domains into our approach as regularization over the domain-specific sentiment classifiers to encourage the sharing of sentiment information between similar domains. Two kinds of domain similarity measures are explored, one based on textual content and the other one based on sentiment expressions. Moreover, we introduce two efficient algorithms to solve the model of our approach. Experimental results on benchmark datasets show that our approach can effectively improve the performance of multi-domain sentiment classification and significantly outperform baseline methods.

Facial appearance and texture feature-based robust facial expression recognition framework for sentiment knowledge discovery

Facial sentiment analysis has been an enthusiastic research area for the last two decades. A fair amount of work has been done by researchers in this field due to its utility in numerous applications such as facial expression driven knowledge discovery. However, developing an accurate and efficient facial expression recognition system is still a challenging problem. Although many efficient recognition systems have been introduced in the past, the recognition rate is not satisfactory in general due to inherent limitations including light, pose variations, noise, and occlusion. In this paper, a hybrid approach of facial expression based sentiment analysis has been presented combining local and global features. Feature extraction is performed fusing the histogram of oriented gradients (HOG) descriptor with the uniform local ternary pattern (U-LTP) descriptor. These features are extracted from the entire face image rather than from individual components of faces like eyes, nose, and mouth. The most suitable set of HOG parameters are selected after analyzing them experimentally along with the ULTP descriptor, boosting performance of the proposed technique over face images containing noise and occlusions. Face sentiments are analyzed classifying them into seven universal emotional expressions: Happy, Angry, Fear, Disgust, Sad, Surprise, and Neutral. Extracted features via HOG and ULTP are fused into a single feature vector and this feature vector is fed into a Multi-class Support Vector Machine classifier for emotion classification. Three types of experiments are conducted over three public facial image databases including JAFFE, MMI, and CK+ to evaluate the recognition rate of the proposed technique during experimental evaluation; recognition accuracy in percent, i.e., 95.71, 98.20, and 99.68 are achieved for JAFFE, MMI, and CK+, respectively.

EFL teachers’ commitment to professional ethics and their emotional intelligence: A relationship study

Emotional intelligence is the capability to realize, to create, to comprehend emotions and sentimental knowledge, and to reflectively control emotions and to improve emotional and mental growth. Th...

Sentiment Analysis and Knowledge Organization: An Overview of the International Literature

Sentiment Analysis and Knowledge Organization: An Overview of the International Literature

Domain-specific sentiment classification via fusing sentiment knowledge from multiple sources

Analyzing the sentiments in massive user-generated online data, such as product reviews and microblogs, has become a hot research topic. It can help customers, companies and expert systems make more informed decisions. Sentiment analysis is widely known as a domain dependent problem. Different domains usually have different sentiment expressions and a general sentiment classifier is not suitable for all domains. A natural solution to this problem is to train a domain-specific sentiment classifier for each target domain. However, the labeled data in target domain is usually insufficient, and it is costly and time-consuming to annotate enough samples. In order to tackle this problem, we propose a novel approach to train domain-specific sentiment classifiers by fusing the sentiment knowledge from multiple sources. Sentiment information from four sources is extracted and fused in our approach. The first source is sentiment lexicons, which contain sentiment polarities of general sentiment words. The second source is the sentiment classifiers of multiple source domains. The third source is the unlabeled data in target domain, from which we extract domain-specific sentiment relations among words. The fourth source is the labeled data in target domain. We propose a unified framework to fuse these four kinds of sentiment knowledge and train domain-specific sentiment classifier for target domain. In addition, we present an efficient optimization algorithm to solve the model of our approach. Extensive experiments are conducted on both Amazon product review dataset and Twitter dataset. Experimental results show that by fusing the sentiment information extracted from multiple sources, our approach can effectively improve the performance of sentiment classification and reduce the dependence on labeled data. For instance, our approach can achieve an accuracy of 87.22% in Kitchen domain when only 200 samples in target domain are labeled. The performance improvements of our approach compared with purely supervised sentiment classifier are 8.98% and 7.92% on Amazon and Twitter datasets respectively.

Microblog sentiment classification with heterogeneous sentiment knowledge

Microblogging services, such as Twitter, are very popular for information release and dissemination. Analyzing the sentiments in massive microblog messages is useful for sensing the public’s opinions on various topics, which has wide applications in both academic and industrial fields. However, microblog sentiment analysis is a challenging task, because microblog messages are short and noisy, and contain massive user-invented acronyms and informal words. It is expensive and time-consuming to manually annotate sufficient samples for training an accurate and robust microblog sentiment classifier. Fortunately, unlabeled microblog messages can provide a lot of useful sentiment knowledge. For example, emoticons are frequently used in microblog messages and they usually indicate sentiment orientations. In this paper, we propose to extract useful sentiment knowledge from massive unlabeled messages to enhance microblog sentiment classification. Three kinds of sentiment knowledge, i.e., contextual similarity knowledge, word-sentiment knowledge, and contextual polarity knowledge, are explored. We propose a unified framework to incorporate the heterogenous sentiment knowledge into the learning of microblog sentiment classifiers. An efficient optimization method based on ADMM is introduced to solve the model of our framework and an accelerated algorithm is proposed to tackle the most time-consuming step. Extensive experiments were conducted on three benchmark Twitter datasets. The experimental results show that our approach can improve the performance of microblog sentiment classification effectively and efficiently.