Sentiment Classification Task Research Articles

Aspect-based sentiment analysis with alternating coattention networks

Aspect-based sentiment analysis aims to predict the sentiment polarities of specific targets in a given text. Recent researches show great interest in modeling the target and context with attention network to obtain more effective feature representation for sentiment classification task. However, the use of an average vector of target for computing the attention score for context is unfair. Besides, the interaction mechanism is simple thus need to be further improved. To solve the above problems, this paper first proposes a coattention mechanism which models both target-level and context-level attention alternatively so as to focus on those key words of targets to learn more effective context representation. On this basis, we implement a Coattention-LSTM network which learns nonlinear representations of context and target simultaneously and can extracts more effective sentiment feature from coattention mechanism. Further, a Coattention-MemNet network which adopts a multiple-hops coattention mechanism is proposed to improve the sentiment classification result. Finally, we propose a new location weighted function which considers the location information to enhance the performance of coattention mechanism. Extensive experiments on two public datasets demonstrate the effectiveness of all proposed methods, and our findings in the experiments provide new insight for future developments of using attention mechanism and deep neural network for aspect-based sentiment analysis.

Identifying Implicit Polarity of Events by Using an Attention-Based Neural Network Model

Sentiment analysis has attracted extensive attention in recent years. Existing work mainly focuses on sentiment classification task in which a text or sentence usually contains sentiment word to express subjective feeling. However, little research is proposed for identifying implicit polarity of a text. Here, implicit polarity of a text means that the text does not contain sentiment words but still express a positive or negative sentiment. To address this issue, we propose an attention-based neural network model to identify implicit polarity of events. In particular, the model first learns the sentence representation by recurrent neural network with gated recurrent unit. Then multiple hops attention mechanism is used for capturing multiple aspects closely related to sentiment polarity and the event type. Experimental results on SemEval 2015 dataset show the effectiveness of the proposed model, outperforming the previous systems and strong neural baselines.

Tibetan Sentiment Classification Method Based on Semi-Supervised Recursive Autoencoders

We apply the semi-supervised recursive autoencoders (RAE) model for the sentiment classification task of Tibetan short text, and we obtain a better classification effect. The input of the semi-supervised RAE model is the word vector. We crawled a large amount of Tibetan text from the Internet, got Tibetan word vectors by using Word2vec, and verified its validity through simple experiments. The values of parameter α and word vector dimension are important to the model effect. The experiment results indicate that when α is 0.3 and the word vector dimension is 60, the model works best. Our experiment also shows the effectiveness of the semi-supervised RAE model for Tibetan sentiment classification task and suggests the validity of the Tibetan word vectors we trained.

Sentiment Analysis of Lithuanian Texts Using Traditional and Deep Learning Approaches

We describe the sentiment analysis experiments that were performed on the Lithuanian Internet comment dataset using traditional machine learning (Naïve Bayes Multinomial—NBM and Support Vector Machine—SVM) and deep learning (Long Short-Term Memory—LSTM and Convolutional Neural Network—CNN) approaches. The traditional machine learning techniques were used with the features based on the lexical, morphological, and character information. The deep learning approaches were applied on the top of two types of word embeddings (Vord2Vec continuous bag-of-words with negative sampling and FastText). Both traditional and deep learning approaches had to solve the positive/negative/neutral sentiment classification task on the balanced and full dataset versions. The best deep learning results (reaching 0.706 of accuracy) were achieved on the full dataset with CNN applied on top of the FastText embeddings, replaced emoticons, and eliminated diacritics. The traditional machine learning approaches demonstrated the best performance (0.735 of accuracy) on the full dataset with the NBM method, replaced emoticons, restored diacritics, and lemma unigrams as features. Although traditional machine learning approaches were superior when compared to the deep learning methods; deep learning demonstrated good results when applied on the small datasets.

Capsule Network With Identifying Transferable Knowledge for Cross-Domain Sentiment Classification

Domain adaptation tasks have raised much attention in recent years, especially, the task of cross-domain sentiment classification, and remarkable success has been achieved on specific domains with large amounts of labeled data. However, annotating enough data in each domain is still expensive and time-consuming, which will produce difficulty in the application of domain adaptation. In this paper, we proposed a Capsule network method with Identifying Transferable Knowledge (CITK) as common knowledge for cross-domain sentiment classification. CITK model uses capsule network to encode the intrinsic spatial part-whole relationship constituting domain invariant knowledge, which bridges the knowledge gap between the source and target domains. In addition, we use Bidirectional Encoder Representations from Transformers (BERT) to convert sentences to equal length, which is called pre-training, in order to obtain more complete semantic embedded representation, so that Significant Consistent Polarity (SCP) words can be more accurate. Extensive experiments are conducted to evaluate the effectiveness of the proposed CITK model on a real world data set of four domains. Experimental results demonstrate that CITK can significantly outperform the state-of-the-art methods for the cross-domain sentiment classification task.

Adding Prior Knowledge in Hierarchical Attention Neural Network for Cross Domain Sentiment Classification

Domain adaptation tasks have raised much attention in recent years, especially, the task of cross-domain sentiment classification (CDSC). Due to the domain discrepancy, a sentiment classifier trained in a source domain often performs less well, when directly applied to a target domain. Adversarial neural networks have been used in mainstream approaches for learning domain independent features, such as pivots, which are words with the same sentiment polarity in different domains. However, domain specific features can often determine sentiment in its context. In this paper, we propose a hierarchical attention network with prior knowledge information (HANP) for the CDSC task. Unlike other existing methods, the HANP can obtain both domain independent and domain specific features at the same time by adding prior knowledge. In addition, the HANP also includes a hierarchical representation layer with attention mechanism, so that the HANP can capture important words and sentences in relation to sentiment. Moreover, the proposed model can offer a direct visualization of the sentimental prior knowledge. The experiments on the Amazon review datasets demonstrate that the proposed HANP can significantly outperform the state-of-the-art methods.

Feature-Based Fusion Adversarial Recurrent Neural Networks for Text Sentiment Classification

Text sentiment classification is a fundamental task of natural language processing. In the past few years, many outstanding methods have attained favorable results in text sentiment classification. However, most of these methods do not make full use of the contextual information of the word embedding layer and attach less importance to the loss of information in the process of forwarding propagation. Hence, an ample room exists for further progress in enhancing the robustness of the model and the feature extraction of text. To tackle these problems, we propose a Feature-Based Fusion Adversarial Recurrent Neural Networks (FARNN-Att) integrated model with an attention mechanism. Firstly, we extract the long-term dependence of text using the BiLSTM network and put forward a novel contextual feature representation way. Subsequently, we combine the prediction results of two features vectors in the full connection layer, which can be captured through the feature connection and attention mechanism. Finally, a regularization method of adversarial training is used to improve the robustness and generalization ability of the model. Our proposed model was compared with other baseline methods such as TextCNN, BiLSTM, BiLSTM-Att, and RCNN on three different public datasets. The experimental results show that our model has state-of-the-art performance in text sentiment classification tasks in terms of accuracy, recall, and F1 score.

Analysis of Machine Learning Algorithms for Opinion Mining in Different Domains

Sentiment classification (SC) is a reference to the task of sentiment analysis (SA), which is a subfield of natural language processing (NLP) and is used to decide whether textual content implies a positive or negative review. This research focuses on the various machine learning (ML) algorithms which are utilized in the analyzation of sentiments and in the mining of reviews in different datasets. Overall, an SC task consists of two phases. The first phase deals with feature extraction (FE). Three different FE algorithms are applied in this research. The second phase covers the classification of the reviews by using various ML algorithms. These are Naïve Bayes (NB), Stochastic Gradient Descent (SGD), Support Vector Machines (SVM), Passive Aggressive (PA), Maximum Entropy (ME), Adaptive Boosting (AdaBoost), Multinomial NB (MNB), Bernoulli NB (BNB), Ridge Regression (RR) and Logistic Regression (LR). The performance of PA with a unigram is the best among other algorithms for all used datasets (IMDB, Cornell Movies, Amazon and Twitter) and provides values that range from 87% to 99.96% for all evaluation metrics.

A cost-sensitive three-way combination technique for ensemble learning in sentiment classification

Deep neural networks (DNN) have achieved remarkable results in sentiment classification. Some ensemble methods of DNN models and traditional feature-based models are proposed recently. However, to the best of our knowledge, most of the works use traditional ensemble combination techniques, e.g. voting and stacking, which are designed for weak base classifiers. So far many base classifiers, e.g. DNN, have been able to achieve good results in sentiment classification tasks, so there should be a new ensemble combination technique designed for strong base classifiers. To address this issue, we proposed a cost-sensitive combination technique using sequential three-way decisions (3WD), which is named S3WC. In S3WC, base classifiers are arranged in a linear arrangement, and a gate mechanism is constructed in each step to divide the objects into three groups, i.e., positive region, negative region and boundary region, which respectively correspond to acceptance, rejection and deferment in sequential 3WD. Each object is grouped by minimizing its total cost consisting of misclassification cost and time cost. The objects in boundary region require more information to decrease the misclassification cost, so they are reclassified by the subsequent base classifiers in order to obtain more information, while the time cost increases. In the experiment, we apply S3WC to DNN models and traditional feature-based models on five benchmark datasets, and compare its performance with traditional ensemble combination techniques. The experimental results show that S3WC outperforms any of its base classifiers in terms of classification accuracy, and the total cost of S3WC is lower than that of the existing ensemble combination techniques (e.g. majority-voting, weighted-voting, meta-learning).

Lexical based automated teaching evaluation via students’ short reviews

AbstractStudent evaluations of teaching (SET) have become a popular approach to assess faculties’ teaching. Question‐score‐based questionnaire is the most common SET measure adopted in universities. However, it fails to cover important facets of teaching process that not mentioned in the predefined questionnaire, which can be substantially obtained from students’ short reviews. In this paper, we propose two lexical‐based methods, specifically knowledge‐based and machine learning‐based, to automatically extract opinions from short reviews. Furthermore, the diversity of reviews’ themes and styles of same sentiment polarity reviews can be observed from the extracted opinion results. The experimental results show that the proposed methods are able to achieve accuracies of 78.13 and 84.78%, respectively in the task of student review sentiment classification. Further investigation on linguistic features shows that reviews with same sentiment polarity shares similar language patterns. Finally, we present an application scenario in real SET process by utilizing aforementioned methods and discoveries.

Emotion‐aware polarity lexicons for Twitter sentiment analysis

AbstractTheoretical frameworks in psychology map the relationships between emotions and sentiments. In this paper, we study the role of such mapping for computational emotion detection from text (e.g., social media) with an aim to understand the usefulness of an emotion‐rich corpus of documents (e.g., tweets) to learn polarity lexicons for sentiment analysis. We propose two different methods that leverage a corpus of emotion‐labelled tweets to learn word‐polarity lexicons. The proposed methods model the emotion corpus using a generative unigram mixture model, combined with the emotion‐sentiment mapping proposed in psychology for automated generation of word‐polarity lexicons that capture emotion‐rich vocabulary. We comparatively evaluate the quality of the proposed mixture model in learning emotion‐aware sentiment lexicons with those generated using supervised latent dirichlet allocation (sLDA) and word‐document‐frequency (WDF) statistics. Sentiment analysis experiments on benchmark Twitter data sets confirm the quality of our proposed lexicons. Further, a comparative analysis with sLDA, WDF‐based emotion‐aware lexicons, and standard sentiment lexicons that are agnostic to emotion knowledge suggests that the proposed lexicons lead to a significantly better performance in both sentiment classification and sentiment intensity prediction tasks.

Sentiment analysis based on improved pre-trained word embeddings

Sentiment analysis is a fast growing area of research in natural language processing (NLP) and text classifications. This technique has become an essential part of a wide range of applications including politics, business, advertising and marketing. There are various techniques for sentiment analysis, but recently word embeddings methods have been widely used in sentiment classification tasks. Word2Vec and GloVe are currently among the most accurate and usable word embedding methods which can convert words into meaningful vectors. However, these methods ignore sentiment information of texts and need a large corpus of texts for training and generating exact vectors. As a result, because of the small size of some corpora, researcher often have to use pre-trained word embeddings which were trained on other large text corpora such as Google News with about 100 billion words. The increasing accuracy of pre-trained word embeddings has a great impact on sentiment analysis research. In this paper, we propose a novel method, Improved Word Vectors (IWV), which increases the accuracy of pre-trained word embeddings in sentiment analysis. Our method is based on Part-of-Speech (POS) tagging techniques, lexicon-based approaches, word position algorithm and Word2Vec/GloVe methods. We tested the accuracy of our method via different deep learning models and benchmark sentiment datasets. Our experiment results show that Improved Word Vectors (IWV) are very effective for sentiment analysis.

Exploiting multiple word embeddings and one-hot character vectors for aspect-based sentiment analysis

Representing words as real-value vectors and use them as input in deep neural networks is an effective approach in many natural language processing tasks. Currently, some studies use a lower-level representation which is character-based vectors. This paper addresses on how to integrate different representations of input for the problem of aspect-based sentiment analysis. We will propose a joint model of multiple Convolutional Neural Networks (CNNs) in which each individual representation of the input is handled by one CNN. In this work we focus on three kinds of representation including word embeddings from the two methods (Word2Vec and GloVe) and the one-hot character vectors. Our experimental results demonstrate that the proposed model can achieve state-of the-art performance in aspect category detection and aspect sentiment classification tasks.

Aspect-based opinion ranking framework for product reviews using a Spearman's rank correlation coefficient method

Opinion mining (also called sentiment analysis) is a type of natural language processing for computing people's opinions and emotions. It detects opinions from structured, semi-structured, and unstructured social media contents at different levels, such as the document, word, sentence, and aspect levels. In all these levels except aspect, opinion mining identifies the overall subjectivity or sentiment polarities. An aspect level is described as a part or an attribute of an entity. It exactly describes people's likes and dislikes in social media contents. In this paper, we propose a new framework for ranking products based on aspects. First, the system identifies the aspects of products. Second, the aspects and their opinion words are identified and visualized from the products’ reviews using a Harel–Koren fast multiscale layout. Third, the network visualization is constructed and modeled, and a Spearman's rank correlation coefficient based opinion ranking method is applied to rank the products based on positive and negative ranks. Fourth, the supervised learning methods (Naïve Bayes, Maximum Entropy, and Support Vector Machine) are employed for the aspect-based sentiment classification task. Finally, the performance of the system is measured by the experimental results.

Learning Latent Opinions for Aspect-level Sentiment Classification

Aspect-level sentiment classification aims at detecting the sentiment expressed towards a particular target in a sentence. Based on the observation that the sentiment polarity is often related to specific spans in the given sentence, it is possible to make use of such information for better classification. On the other hand, such information can also serve as justifications associated with the predictions.We propose a segmentation attention based LSTM model which can effectively capture the structural dependencies between the target and the sentiment expressions with a linear-chain conditional random field (CRF) layer. The model simulates human's process of inferring sentiment information when reading: when given a target, humans tend to search for surrounding relevant text spans in the sentence before making an informed decision on the underlying sentiment information.We perform sentiment classification tasks on publicly available datasets on online reviews across different languages from SemEval tasks and social comments from Twitter. Extensive experiments show that our model achieves the state-of-the-art performance while extracting interpretable sentiment expressions.

A visual attention-based keyword extraction for document classification

Document classification plays an important role in natural language processing. Among that, keyword extraction algorithm shows its great potential in summarizing the entire document. Attention is the process of selectively concentrating on a discrete aspect of information, while ignoring other perceivable information. A new probabilistic keyword extraction algorithm is proposed, which is inspired by the visual attention mechanism. An unsupervised neural network based pre-training method is proposed for training the semantic attention based keyword extraction algorithm, which is helpful in extracting keywords with rich contextual information from the document. A bidirectional Long short-term memory network combined with the proposed semantic keyword extraction algorithm is designed for both topic and sentiment classification tasks. Experiments on four large scale datasets show that the proposed visual attention based keyword extraction algorithm gives a better performance than the baseline methods. The semantic attention based keyword extraction method is significant in summarizing the content of a document, which is very useful for large scale document classification.

Sentiment analysis of Chinese online reviews using ensemble learning framework

Unstructured online reviews are undergoing a rather rapid expansion with the development of E-commerce, and they contain sentiment information in which consumers and businesses are very interested. Therefore, effective sentiment classification has become one of the important research topics. Many studies have shown that ensemble learning methods may have great hopeful applicability in sentiment classification tasks. In this paper, we propose a new ensemble learning framework for sentiment classification of Chinese online reviews. First of all, according to the complicated characteristics of Chinese online reviews, we extract Part of Speech Combination Pattern, Frequent Word Sequence Pattern and Order Preserved Submatrix Pattern as the input features. Furthermore, we use the algorithm of Random Subspace based on Information Gain by considering the problem of massive features in the reviews, which can improve the base classifiers simultaneously. Finally, we adopt the algorithm of Constructing Base Classifiers based on Product Attributes to combine the sentiment information of each attribute in a review so as to obtain better performance on sentiment classification. The experimental results show that the proposed ensemble learning framework has significant improvement in sentiment classification of Chinese online reviews.

Automatic Approach of Sentiment Lexicon Generation for Mobile Shopping Reviews

The dramatic increase in the use of smartphones has allowed people to comment on various products at any time. The analysis of the sentiment of users’ product reviews largely depends on the quality of sentiment lexicons. Thus, the generation of high‐quality sentiment lexicons is a critical topic. In this paper, we propose an automatic approach for constructing a domain‐specific sentiment lexicon by considering the relationship between sentiment words and product features in mobile shopping reviews. The approach first selects sentiment words and product features from original reviews and mines the relationship between them using an improved pointwise mutual information algorithm. Second, sentiment words that are related to mobile shopping are clustered into categories to form sentiment dimensions. At each sentiment dimension, each sentiment word can take the value of 0 or 1, where 1 indicates that the word belongs to a particular category whereas 0 indicates that it does not belong to that category. The generated lexicon is evaluated by constructing a sentiment classification task using several product reviews written in both Chinese and English. Two popular non‐domain‐specific sentiment lexicons as well as state‐of‐the‐art machine‐learning and deep‐learning models are chosen as benchmarks, and the experimental results show that our sentiment lexicons outperform the benchmarks with statistically significant differences, thus proving the effectiveness of the proposed approach.

Cross-Domain Sentiment Classification by Capsule Network With Semantic Rules

Sentiment analysis is an important but challenging task. Remarkable success has been achieved on domains where sufficient labeled training data is available. Nevertheless, annotating sufficient data is labor-intensive and time-consuming, establishing significant barriers for adapting the sentiment classification systems to new domains. In this paper, we introduce a Capsule network for sentiment analysis in domain adaptation scenario with semantic rules (CapsuleDAR). CapsuleDAR exploits capsule network to encode the intrinsic spatial part-whole relationship constituting domain invariant knowledge that bridges the knowledge gap between the source and target domains. Furthermore, we also propose a rule network to incorporate the semantic rules into the capsule network to enhance the comprehensive sentence representation learning. Extensive experiments are conducted to evaluate the effectiveness of the proposed CapsuleDAR model on a real world data set of four domains. Experimental results demonstrate that CapsuleDAR achieves substantially better performance than the strong competitors for the cross-domain sentiment classification task.

A hybrid model for opinion mining based on domain sentiment dictionary

Sentiment classification is an application of sentiment analysis, which is a popular research field in NLP. It can classify documents into different categories according to their sentiments. For a sentiment classification task, the first step is to extract sentimental features from documents, and then classify them using some classifiers. In the first step, a traditional way to extract sentimental features is to apply sentiment dictionaries. However, sentiment words may have different sentiment tendencies in different contexts, and traditional sentiment dictionaries does not consider this situation where wrong sentiment tendencies may be selected for sentiment words. In our research, we find that sentiment words will not have diverse meanings when they associate with the nearby aspects and entities in documents. Then, we propose a three layers sentiment dictionary, which can associate sentiment words with the corresponding entities and aspects together to reduce their multiple meanings. In the second step of the sentiment classification task, many classification models, such as SVM, GBDT, can be used to classify documents according to the extracted sentiment words. However, different classifiers have different weaknesses. A Stacking-based hybrid model is applied to combine SVM and GBDT together to overcome their weaknesses and reach higher performance. This hybrid model contains two layers, and the output of the first layer will become the input of the second layer. The first layer will generate different classification results according to different classifiers, while the second layer will automatically learn how to select a probable one as the final result. The experimental results show that our hybrid model outperforms the baseline single models.