Semi-supervised Sentiment Classification Research Articles

Semi-Supervised Sentiment Classification and Emotion Distribution Learning Across Domains

In this study, sentiment classification and emotion distribution learning across domains are both formulated as a semi-supervised domain adaptation problem, which utilizes a small amount of labeled documents in the target domain for model training. By introducing a shared matrix that captures the stable association between document clusters and word clusters, non-negative matrix tri-factorization (NMTF) is robust to the labeled target domain data and has shown remarkable performance in cross-domain text classification. However, the existing NMTF-based models ignore the incompatible relationship of sentiment polarities and the relatedness among emotions. Besides, their applications on large-scale datasets are limited by the high computation complexity. To address these issues, we propose a semi-supervised NMTF framework for sentiment classification and emotion distribution learning across domains. Based on a many-to-many mapping between document clusters and sentiment polarities (or emotions), we first incorporate the prior information of label dependency to improve the model performance. Then, we develop a parallel algorithm based on message passing interface (MPI) to further enhance the model scalability. Extensive experiments on real-world datasets validate the effectiveness of our method.

Using Informative Score for Instance Selection Strategy in Semi-Supervised Sentiment Classification

Sentiment classification is a useful tool to classify reviews about sentiments and attitudes towards a product or service. Existing studies heavily rely on sentiment classification methods that require fully annotated inputs. However, there is limited labelled text available, making the acquirement process of the fully annotated input costly and labour-intensive. Lately, semi-supervised methods emerge as they require only partially labelled input but perform comparably to supervised methods. Nevertheless, some works reported that the performance of the semi-supervised model degraded after adding unlabelled instances into training. Literature also shows that not all unlabelled instances are equally useful; thus identifying the informative unlabelled instances is beneficial in training a semi-supervised model. To achieve this, an informative score is proposed and incorporated into semi-supervised sentiment classification. The evaluation is performed on a semi-supervised method without an informative score and with an informative score. By using the informative score in the instance selection strategy to identify informative unlabelled instances, semi-supervised models perform better compared to models that do not incorporate informative scores into their training. Although the performance of semi-supervised models incorporated with an informative score is not able to surpass the supervised models, the results are still found promising as the differences in performance are subtle with a small difference of 2% to 5%, but the number of labelled instances used is greatly reduced from 100% to 40%. The best finding of the proposed instance selection strategy is achieved when incorporating an informative score with a baseline confidence score at a 0.5:0.5 ratio using only 40% labelled data.

Semi-Supervised Sentiment Classification on E-Commerce Reviews Using Tripartite Graph and Clustering

Sentiment classification constitutes an important topic in the field of Natural Language Processing, whose main purpose is to extract the sentiment polarity from unstructured texts. The label propagation algorithm, as a semi-supervised learning method, has been widely used in sentiment classification due to its describing sample relation in a graph-based pattern. Whereas, current graph developing strategies fail to use the global distribution and cannot handle the issues of polysemy and synonymy properly. In this paper, a semi-supervised learning methodology, integrating the tripartite graph and the clustering, is proposed for graph construction. Experiments on E-commerce reviews demonstrate the proposed method outperform baseline methods on the whole, which enables precise sentiment classification with few labeled samples.

LeSSA: A Unified Framework based on Lexicons and Semi-Supervised Learning Approaches for Textual Sentiment Classification

Sentiment Analysis (SA) is an active research area. SA aims to classify the online unstructured user-generated contents (UUGC) into positive and negative classes. A reliable training data is vital to learn a sentiment classifier for textual sentiment classification, but due to domain heterogeneity, manually construction of reliable labeled sentiment corpora is a laborious and time-consuming task. In the absence of enough labeled data, the alternative usage of sentiment lexicons and semi-supervised learning approaches for sentiment classification have substantially attracted the attention of the research community. However, state-of-the-art techniques for semi-supervised sentiment classification present research challenges expressed in questions like the following. How to effectively utilize the concealed significant information in the unstructured data? How to learn the model while considering the most effective sentiment features? How to remove the noise and redundant features? How to refine the initial training data for initial model learning as the random selection may lead to performance degradation? Besides, mainly existing lexicons have trouble with word coverage, which may ignore key domain-specific sentiment words. Further research is required to improve the sentiment lexicons for textual sentiment classification. In order to address such research issues, in this paper, we propose a novel unified sentiment analysis framework for textual sentiment classification called LeSSA. Our main contributions are threefold. (a) lexicon construction, generating quality and wide coverage sentiment lexicon. (b) training classification models based on a high-quality training dataset generated by using k-mean clustering, active learning, self-learning, and co-training algorithms. (c) classification fusion, whereby the predictions from numerous learners are confluences to determine final sentiment polarity based on majority voting, and (d) practicality, that is, we validate our claim while applying our model on benchmark datasets. The empirical evaluation of multiple domain benchmark datasets demonstrates that the proposed framework outperforms existing semi-supervised learning techniques in terms of classification accuracy.

Context-sensitive and attribute-based sentiment classification of online consumer-generated content

Purpose Vast volumes of rich online consumer-generated content (CGC) can be used effectively to gain important insights for decision-making, product improvement and brand management. Recently, many studies have proposed semi-supervised aspect-based sentiment classification of unstructured CGC. However, most of the existing CGC mining methods rely on explicitly detecting aspect-based sentiments and overlooking the context of sentiment-bearing words. Therefore, this study aims to extract implicit context-sensitive sentiment, and handle slangs, ambiguous, informal and special words used in CGC. Design/methodology/approach A novel text mining framework is proposed to detect and evaluate implicit semantic word relations and context. First, POS (part of speech) tagging is used for detecting aspect descriptions and sentiment-bearing words. Then, LDA (latent Dirichlet allocation) is used to group similar aspects together and to form an attribute. Semantically and contextually similar words are found using the skip-gram model for distributed word vectorisation. Finally, to find context-sensitive sentiment of each attribute, cosine similarity is used along with a set of positive and negative seed words. Findings Experimental results using more than 400,000 Amazon mobile phone reviews showed that the proposed method efficiently found product attributes and corresponding context-aware sentiments. This method also outperforms the classification accuracy of the baseline model and state-of-the-art techniques using context-sensitive information on data sets from two different domains. Practical implications Extracted attributes can be easily classified into consumer issues and brand merits. A brand-based comparative study is presented to demonstrate the practical significance of the proposed approach. Originality/value This paper presents a novel method for context-sensitive attribute-based sentiment analysis of CGC, which is useful for both brand and product improvement.

Semi-Supervised Target-Dependent Sentiment Classification for Micro-Blogs

The wealth of opinions available in the social media motivated researchers to develop automatic opinion detection tools. Many such tools are currently available online for opinion mining in short text, known as micro-blogs, but their efficacies are still limited. Current tools focus on detecting sentiment polarity expressed in a micro-blog regardless of the topic (target) discussed. Little improved approaches have been proposed to detect sentiment towards a specific target, referred to as target-dependent sentiment classification. Our literature review has shown that all these target-dependent approaches use supervised learning techniques. Such techniques need a huge amount of labeled data for increasing classification accuracy. However, preparing labeled data from social media needs a lot of efforts. In this work, we address this issue by employing semisupervised learning techniques that have not been used before with target-dependent sentiment classification. To the best of our knowledge, our work is the first research that employs semisupervised learning techniques in this direction. Semi-supervised learning techniques have been known in the literature to improve classification accuracy in comparison with supervised learning techniques; however, they use same number of labeled samples plus many unlabelled ones. In this work, we propose a new semi-supervised learning technique that uses less number of labeled microblogs than that used with supervised learning techniques. Experiment results have shown that the proposed technique provides competitive accuracy.

Semi-supervised Aspect-level Sentiment Classification Model based on Variational Autoencoder

Aspect-level sentiment classification aims to predict the sentiment of a text in different aspects and it is a fine-grained sentiment analysis task. Recent work exploits an Attention-based Long Short-Term Memory Network to perform aspect-level sentiment classification. Most previous work are based on supervised learning that needs a large number of labeled samples, but the problem is that only a limited subset of data samples are labeled in practical applications. To solve this problem, we propose a novel Semi-supervised Aspect Level Sentiment Classification Model based on Variational Autoencoder (AL-SSVAE) for semi-supervised learning in the aspect-level sentiment classification. The AL-SSVAE model inputs a given aspect to an encoder a decoder based on a variational autoencoder (VAE), and it also has an aspect level sentiment classifier. It enables the attention mechanism to deal with different parts of a text when different aspects are taken as input as previous methods. Due to that the sentiment polarity of a word is usually sensitive to the given aspect, a single vector for a word is problematic. Therefore, we propose the aspect-specific word embedding learning from a topical word embeddings model to express a word and also append the corresponding sentiment vector into the word input vector. We compare our AL-SSVAE model with several recent aspect-level sentiment classification models on the SemEval 2016 dataset. The experimental results indicate that the proposed model is able to capture more accurate semantics and sentiment for the given aspect and obtain better performance on the task of the aspect level sentiment classification. Moreover, the AL-SSVAE model is able to learn with the semi-supervised mode in the aspect level sentiment classification, which enables it to learn efficiently using less labeled data.

Semi-supervised target-oriented sentiment classification

Target-oriented aspect-based sentiment analysis (TABSA) is a sentiment classification task that requires performing fine-grained semantical reasoning about a given aspect. The amount of labeled data is usually insufficient for supervised learning because the manual annotation w.r.t. the aspects is both time-consuming and laborious. In this paper, we propose a novel semi-supervised method to derive and utilize the underlying sentiment of unlabeled samples via a deep generative model. This method assumes that when given the aspect, the sentence is generated by two stochastic variables, i.e., the context variable and the sentiment variable. By explicitly disentangling the representation into the context and sentiment, the meaning of sentiment variable can be kept clean during the training phase. An additional advantage is that the proposed method uses a standalone classifier, and as such, our system is able to integrate with various supervised models. In terms of the implementation, since capturing the conditional input is non-trivial for a sequential model, special structures are put forward and investigated. We conducted experiments on SemEval 2014 task 4 and the results indicate that our method effectively handles five kinds of advanced classifiers. The proposed method outperforms two general semi-supervised methods and achieves state-of-the-art performance on this benchmark.

Semi-supervised Sentiment Classification with Dialog Data

Semi-supervised Sentiment Classification with Dialog Data

Take full advantage of unlabeled data for sentiment classification

PurposeAs a foundational issue of social mining, sentiment classification suffered from a lack of unlabeled data. To enhance accuracy of classification with few labeled data, many semi-supervised algorithms had been proposed. These algorithms improved the classification performance when the labeled data are insufficient. However, precision and efficiency are difficult to be ensured at the same time in many semi-supervised methods. This paper aims to present a novel method for using unlabeled data in a more accurate and more efficient way.Design/methodology/approachFirst, the authors designed a boosting-based method for unlabeled data selection. The improved boosting-based method can choose unlabeled data which have the same distribution with the labeled data. The authors then proposed a novel strategy which can combine weak classifiers into strong classifiers that are more rational. Finally, a semi-supervised sentiment classification algorithm is given.FindingsExperimental results demonstrate that the novel algorithm can achieve really high accuracy with low time consumption. It is helpful for achieving high-performance social network-related applications.Research limitations/implicationsThe novel method needs a small labeled data set for semi-supervised learning. Maybe someday the authors can improve it to an unsupervised method.Practical implicationsThe mentioned method can be used in text mining, image classification, audio processing and so on, and also in an unstructured data mining-related field. Overcome the problem of insufficient labeled data and achieve high precision using fewer computational time.Social implicationsSentiment mining has wide applications in public opinion management, public security, market analysis, social network and related fields. Sentiment classification is the basis of sentiment mining.Originality/valueAccording to what the authors have been informed, it is the first time transfer learning be introduced to AdaBoost for semi-supervised learning. Moreover, the improved AdaBoost uses a totally new mechanism for weighting.

Sentiment labeling for extending initial labeled data to improve semi-supervised sentiment classification

Semi-supervised framework which exploits unsupervised approach (JST) is proposed.Self-training suffers from incorrectly labeling problem with insufficient data.Confidently predicted instances are labeled and used as training data by JST.Self-training with extended training corpus incrementally improves classifier model.Sentiment labeling improves accuracy by enriching initial classifier of self-training. In recent decades, analyzing the sentiments in online customer reviews has become important to many businesses and researchers. However, insufficient amount of labeled training corpus is a bottleneck for machine learning approaches. Self-training is one of the promising semi-supervised techniques which does not require large amounts of labeled data. However, self-training also suffers from an incorrect labeling problem along with insufficient amount of labeled data. This study proposed a semi-supervised learning framework that adds only confidently predicted data to the training corpus in order to enrich the initial classifier in self-training. The experimental results indicate that the proposed method performed better than self-training.

TASC:Topic-Adaptive Sentiment Classification on Dynamic Tweets

Sentiment classification is a topic-sensitive task, i.e., a classifier trained from one topic will perform worse on another. This is especially a problem for the tweets sentiment analysis. Since the topics in Twitter are very diverse, it is impossible to train a universal classifier for all topics. Moreover, compared to product review, Twitter lacks data labeling and a rating mechanism to acquire sentiment labels. The extremely sparse text of tweets also brings down the performance of a sentiment classifier. In this paper, we propose a semi-supervised topic-adaptive sentiment classification (TASC) model, which starts with a classifier built on common features and mixed labeled data from various topics. It minimizes the hinge loss to adapt to unlabeled data and features including topic-related sentiment words, authors’ sentiments and sentiment connections derived from “@” mentions of tweets, named as topic-adaptive features. Text and non-text features are extracted and naturally split into two views for co-training. The TASC learning algorithm updates topic-adaptive features based on the collaborative selection of unlabeled data, which in turn helps to select more reliable tweets to boost the performance. We also design the adapting model along a timeline (TASC-t) for dynamic tweets. An experiment on 6 topics from published tweet corpuses demonstrates that TASC outperforms other well-known supervised and ensemble classifiers. It also beats those semi-supervised learning methods without feature adaption. Meanwhile, TASC-t can also achieve impressive accuracy and F-score. Finally, with timeline visualization of “river” graph, people can intuitively grasp the ups and downs of sentiments’ evolvement, and the intensity by color gradation.

Semi-supervised Micro-blog Sentiment Classification Method Combining Active Learning and Co-training

Semi-supervised Micro-blog Sentiment Classification Method Combining Active Learning and Co-training

Active semi-supervised learning method with hybrid deep belief networks.

In this paper, we develop a novel semi-supervised learning algorithm called active hybrid deep belief networks (AHD), to address the semi-supervised sentiment classification problem with deep learning. First, we construct the previous several hidden layers using restricted Boltzmann machines (RBM), which can reduce the dimension and abstract the information of the reviews quickly. Second, we construct the following hidden layers using convolutional restricted Boltzmann machines (CRBM), which can abstract the information of reviews effectively. Third, the constructed deep architecture is fine-tuned by gradient-descent based supervised learning with an exponential loss function. Finally, active learning method is combined based on the proposed deep architecture. We did several experiments on five sentiment classification datasets, and show that AHD is competitive with previous semi-supervised learning algorithm. Experiments are also conducted to verify the effectiveness of our proposed method with different number of labeled reviews and unlabeled reviews respectively.

Semi-supervised Sentiment Classification using Ranked Opinion Words

This work proposes a semi-supervised sentiment classification method which is based on the co-training framework. The proposed method needs to construct three sentiment classifiers. We use common text features to construct the first classifier. We extract opinion words from consumer reviews, and then we ranked these opinion words according to their importance. We also employ extracted opinion words and the ranked co-occurrence opinion words of the extracted opinion words of each review to get the second sentiment classifier. A third sentiment classifier comes into being using non-opinion text features from each review. Based on co-training semi-supervised learning framework, we use the three sentiment classifiers to iteratively get the final sentiment classifier. Experimental results show that our proposed method has better performance than the Self-learning SVM method and the Naive co-training SVM method.

Fuzzy deep belief networks for semi-supervised sentiment classification

By embedding prior knowledge into the learning structure, this paper presents a two-step semi-supervised learning method called fuzzy deep belief networks (FDBN) for sentiment classification. First, we train the general deep belief networks (DBN) by the semi-supervised learning taken on training dataset. Then, we design a fuzzy membership function for each class of reviews based on the learned deep architecture. Since the training of DBN maps each review into the DBN output space, the distribution of all training samples in the space is treated as prior knowledge and is encoded by a series of fuzzy membership functions. Second, based on the fuzzy membership functions and the DBN obtained in the first step, a novel FDBN architecture is constructed and the supervised learning stage is applied to improve the classification performance of the FDBN. FDBN not only inherits the powerful abstraction ability of DBN, but also demonstrates the attractive fuzzy classification ability for handling sentiment data. To inherit the advantages of both active learning and FDBN, we also propose an active FDBN (AFD) semi-supervised learning method. The empirical validation on five sentiment classification datasets demonstrates the effectiveness of FDBN and AFD methods.

Sentiment Classification using Subjective and Objective Views

work proposes a new semi-supervised sentiment classification method by exploiting a large number of unlabeled instances to conduct sentiment classification for Web consumer reviews. In the proposed method every consumer review has two views: subjective view and objective view. The subjective view of a consumer review reflects the opinions expressed by opinion words, while the objective view is constructed by the remaining text features. This work is trying to combine two kinds of views to carry out sentiment classification. The method is based on the co-training framework which needs three basic sentiment classifiers to iteratively get the final sentiment classifier. In the proposed method, the first sentiment classifier is constructed using the common unigram features coming from consumer reviews. The second sentiment classifier is trained on the subjective views constructed by opinion words extracted from consumer reviews. The remaining text features of these reviews are used for obtaining the objective views which can be trained for the third classifier. Experimental results show the proposed method is effective, and it has better performance than the Self-learning SVM method.

Active deep learning method for semi-supervised sentiment classification

In natural language processing community, sentiment classification based on insufficient labeled data is a well-known challenging problem. In this paper, a novel semi-supervised learning algorithm called active deep network (ADN) is proposed to address this problem. First, we propose the semi-supervised learning framework of ADN. ADN is constructed by restricted Boltzmann machines (RBM) with unsupervised learning based on labeled reviews and abundant of unlabeled reviews. Then the constructed structure is fine-tuned by gradient-descent based supervised learning with an exponential loss function. Second, in the semi-supervised learning framework, we apply active learning to identify reviews that should be labeled as training data, then using the selected labeled reviews and all unlabeled reviews to train ADN architecture. Moreover, we combine the information density with ADN, and propose information ADN (IADN) method, which can apply the information density of all unlabeled reviews in choosing the manual labeled reviews. Experiments on five sentiment classification datasets show that ADN and IADN outperform classical semi-supervised learning algorithms, and deep learning techniques applied for sentiment classification.