Cross-domain Sentiment Analysis Research Articles

Cross-Domain Sentiment Analysis Based on Small in-Domain Fine-Tuning

Significant progress has been made in sentiment analysis over the past few years, especially due to the application of deep neural language models. However, there is a problem of transferability of trained models from one domain to another, especially for less studied languages such as Russian. We propose an approach to build cross-domain sentiment analysis models based on a two-stage procedure: first, we fine-tune a pre-trained RuBERT language model on a combined non-domain corpus, and then fine-tune this model on a small domain corpus. We conducted large-scale experiments with 30 sentiment annotated corpora across 12 domains. In order to increase the representativeness of news texts with high-quality annotation, we created a novel RuNews corpus, containing 1,823 news articles annotated by sentiment. The results show that fine-tuning the model using a small number (about several hundred) of annotated domain texts can significantly improve the performance of sentiment analysis for a new domain (on average by 4.6 p.p.). We also obtained the state-of-the-art results for 7 out of 14 test corpora.

Aspect-Opinion Correlation Aware and Knowledge-Expansion Few Shot Cross-Domain Sentiment Classification

Cross-domain sentiment analysis has recently attracted significant attention, which can effectively alleviate the problem of lacking large-scale labeled data for deep neural network based methods. However, most of the existing cross-domain sentiment classification models neglect the domain-specific features, which limits their performance especially when the domain discrepancy becomes larger. Meanwhile, the relations between the aspect and opinion terms cannot be effectively modeled and thus the sentiment transfer error problem is suffered in the existing unsupervised domain-adaptation methods. To address these two issues, we propose an aspect-opinion correlation aware and knowledge-expansion few shot cross-domain sentiment classification model. Sentiment classification can be effectively conducted with only a few support instances of the target domain. Extensive experiments are conducted and the experimental results show the effectiveness of our proposed model.

Aspect-Opinion Sentiment Alignment for Cross-Domain Sentiment Analysis (Student Abstract)

Cross-domain sentiment analysis (SA) has recently attracted significant attention, which can effectively alleviate the problem of lacking large-scale labeled data for deep neural network based methods. However, exiting unsupervised cross-domain SA models ignore the relation between the aspect and opinion, which suffer from the sentiment transfer error problem. To solve this problem, we propose an aspect-opinion sentiment alignment SA model and extensive experiments are conducted to evaluate the effectiveness of our model.

Investigating the informativeness of technical indicators and news sentiment in financial market price prediction

Real-time market prediction tool tracking public opinion in specialized newsgroups and informative market data persuades investors of financial markets. Previous works mainly used lexicon-based sentiment analysis for financial markets prediction, while recently proposed transformer-based sentiment analysis promise good results for cross-domain sentiment analysis. This work considers temporal relationships between consecutive snapshots of informative market data and mood time series for market price prediction. We calculate the sentiment mood time series via the probability distribution of news embedding generated through a BERT-based transformer language model fine-tuned for financial domain sentiment analysis. We then use a deep recurrent neural network for feature extraction followed by a dense layer for price regression. We implemented our approach as an open-source API for real-time price regression. We build a corpus of financial news related to currency pairs in foreign exchange and Cryptocurrency markets. We further augment our model with informative technical indicators and news sentiment scores aligned based on news release timestamp. Results of our experiments show significant error reduction compared to the baselines. Our Financial News and Financial Sentiment Analysis RESTFul APIs are available for public use.

Joint Dilated Convolution and Self-attention for Cross-domain Sentiment Analysis

Cross-domain sentiment analysis aims to use source domain resources or models to serve the sentiment analysis tasks in the target domain, and it can effectively alleviate the problem of insufficient tagged data in the target domain. In this paper, we investigate deep transfer learning for text sentiment analysis, and focus on customer reviews for different domains. Given the shortcomings of the conventional deep neural network, such as convolutional neural network (CNN) cannot obtain the long-term dependence between features, recurrent neural network (RNN) cannot achieve parallel computing, we propose a novel neural network which joint dilated convolution and self-attention (ADC). The ADC takes residual learning as its basic framework, which can capture more distant information by introducing a dilation rate in the convolution. Self-attention is used to make the ADC pay more attention to important sentiment information. First, the ADC is pre-trained with the data of the source domain, and the weight of the pre-training is frozen. Then, the model is fine-tuned by the method of gradual unfreezing, and the weight is updated through the data of the target domain. Finally, the updated model can be utilized for the target domain sentiment analysis. By using customer review datasets, we carry out extensive experiments to demonstrate that this method has superior performance and can better achieve cross-domain sentiment analysis.

Deep Learning Based Cross Domain Sentiment Classification for Urdu Language

Sentiment analysis is a widely researched area due to its various applications in customer services, brand monitoring, and market research. Automatic sentiment classification is an important but challenging task. Contrary to the English language, sentiment analysis for low-resource languages like Urdu is an under-explored research area. Most of the work on sentiment analysis in the Urdu language is domain-dependent where models are mostly trained and tested on the same dataset on limited domains. However, sentiments in different domains are expressed differently, and manually annotating the datasets for all possible domains is unfeasible. Training a sentiment classifier using annotated data on one domain and testing it on another domain results in poor performance as the terms appearing in the source domain (training data) might not appear in the target (testing data) domain. In this paper, we present a baseline method for cross-domain sentiment analysis in the Urdu language using two different domains. Feature extraction is performed using n-grams and word embedding techniques. Sentiment classification is performed using machine learning and deep learning classifiers. The proposed method achieves an accuracy, precision, recall, and F1 scores of 0.77, 0.83, 0.68, and 0.75, respectively.

S2SAN: A sentence-to-sentence attention network for sentiment analysis of online reviews

Many existing attention-based deep learning approaches to sentiment analysis have focused on words and represent an entire review text as a word sequence. However, these approaches overlook the differences in the importance of each sentence to the complete text. To solve this problem, some work has been performed to calculate sentence-level attention, but these studies use the same approach that is applied to word-level attention, which leads to unnecessary sequential structures and increased complexity of sentence representation. Therefore, in this paper, we propose a sentence-to-sentence attention network11https://github.com/JingruiHou/S2SAN (S2SAN) using multihead self-attention. We conducted several domain-specific, cross-domain and multidomain sentiment analysis experiments with real-world datasets. The experimental results show that S2SAN outperforms other state-of-the-art models. Some classical sentiment classifiers [e.g., convolutional neural network (CNN), recurrent neural network (RNN), and long short-term memory (LSTM) models] achieve better accuracies when they are reconfigured to include sentence-to-sentence attention.

An Adaptive Hybrid Framework for Cross-domain Aspect-based Sentiment Analysis

Cross-domain aspect-based sentiment analysis aims to utilize the useful knowledge in a source domain to extract aspect terms and predict their sentiment polarities in a target domain. Recently, methods based on adversarial training have been applied to this task and achieved promising results. In such methods, both the source and target data are utilized to learn domain-invariant features through deceiving a domain discriminator. However, the task classifier is only trained on the source data, which causes the aspect and sentiment information lying in the target data can not be exploited by the task classifier. In this paper, we propose an Adaptive Hybrid Framework (AHF) for cross-domain aspect-based sentiment analysis. We integrate pseudo-label based semi-supervised learning and adversarial training in a unified network. Thus the target data can be used not only to align the features via the training of domain discriminator, but also to refine the task classifier. Furthermore, we design an adaptive mean teacher as the semi-supervised part of our network, which can mitigate the effects of noisy pseudo labels generated on the target data. We conduct experiments on four public datasets and the experimental results show that our framework significantly outperforms the state-of-the-art methods.

An attention network based on feature sequences for cross-domain sentiment classification

The difficulty of cross-domain text sentiment classification is that the data distributions in the source domain and the target domain are inconsistent. This paper proposes an attention network based on feature sequences (ANFS) for cross-domain sentiment classification, which focuses on important semantic features by using the attention mechanism. Particularly, ANFS uses a three-layer convolutional neural network (CNN) to perform deep feature extraction on the text, and then uses a bidirectional long short-term memory (BiLSTM) to capture the long-term dependency relationship among the text feature sequences. We first transfer the ANFS model trained on the source domain to the target domain and share the parameters of the convolutional layer; then we use a small amount of labeled target domain data to fine-tune the model of the BiLSTM layer and the attention layer. The experimental results on cross-domain sentiment analysis tasks demonstrate that ANFS can significantly outperform the state-of-the-art methods for cross-domain sentiment classification problems.

A Deep Neural Network Model for Cross-Domain Sentiment Analysis

Sentiment analysis is used to detect the opinion/sentiment expressed from the unstructured text. Most of the existing state-of-the-art methods are based on supervised learning, and therefore, a labelled dataset is required to build the model, and it is very difficult task to obtain a labelled dataset for every domain. Cross-domain sentiment analysis is to develop a model which is trained on labelled dataset of one domain, and the performance is evaluated on another domain. The performance of such cross-domain sentiment analysis is still very limited due to presence of many domain-related terms, and the sentiment analysis is a domain-dependent problem in which words changes their polarity depending upon the domain. In addition, cross-domain sentiment analysis model suffers with the problem of large number of out-of-the-vocabulary (unseen words) words. In this paper, the authors propose a deep learning-based approach for cross-domain sentiment analysis. Experimental results show that the proposed approach improves the performance on the benchmark dataset.

Cross-domain sentiment analysis model on Indonesian YouTube comment

A cross-domain sentiment analysis (CDSA) study in the Indonesian language and tree-based ensemble machine learning is quite interesting. CDSA is useful to support the labeling process of cross-domain sentiment and reduce any dependence on the experts; however, the mechanism in the opinion unstructured by stop word, language expressions, and Indonesian slang words is unidentified yet. This study aimed to obtain the best model of CDSA for the opinion in Indonesia language that commonly is full of stop words and slang words in the Indonesian dialect. This study was purposely to observe the benefits of the stop words cleaning and slang words conversion in CDSA in the Indonesian language form. It was also to find out which machine learning method is suitable for this model. This study started by crawling five datasets of the comments on YouTube from 5 different domains. The dataset was copied into two groups: the dataset group without any process of stop word cleaning and slang word conversion and the dataset group to stop word cleaning and slang word conversion. CDSA model was built for each dataset group and then tested using two types of tree-based ensemble machine learning, i.e., Random Forest (RF) and Extra Tree (ET) classifier, and tested using three types of non-ensemble machine learning, including Naïve Bayes (NB), SVM, and Decision Tree (DT) as the comparison. Then, It can be suggested that the accuracy of CDSA in Indonesia Language increased if it still removed the stop words and converted the slang words. The best classifier model was built using tree-based ensemble machine learning, particularly ET, as in this study, the ET model could achieve the highest accuracy by 91.19%. This model is expected to be the CDSA technique alternative in the Indonesian language.

Cross-Domain End-To-End Aspect-Based Sentiment Analysis with Domain-Dependent Embeddings

With the development of sentiment analysis, studies have been gradually classified based on different researched candidates. Among them, aspect-based sentiment analysis plays an important role in subtle opinion mining for online reviews. It used to be treated as a group of pipeline tasks but has been proved to be analysed well in an end-to-end model recently. Due to less labelled resources, the need for cross-domain aspect-based sentiment analysis has started to get attention. However, challenges exist when seeking domain-invariant features and keeping domain-dependent features to achieve domain adaptation within a fine-grained task. This paper utilizes the domain-dependent embeddings and designs the model CD-E2EABSA to achieve cross-domain aspect-based sentiment analysis in an end-to-end fashion. The proposed model utilizes the domain-dependent embeddings with a multitask learning strategy to capture both domain-invariant and domain-dependent knowledge. Various experiments are conducted and show the effectiveness of all components on two public datasets. Also, it is also proved that as a cross-domain model, CD-E2EABSA can perform better than most of the in-domain ABSA methods.

Fusion of part-of-speech vectors and attention mechanisms for cross-domain sentiment analysis

The grand challenge of cross-domain sentiment analysis is that classifiers trained in a specific domain are very sensitive to the discrepancy between domains. A sentiment classifier trained in the source domain usually have a poor performance in the target domain. One of the main strategies to solve this problem is the pivot-based strategy, which regards the feature representation as an important component. However, part-of-speech information was not considered to guide the learning of feature representation and feature mapping in previous pivot-based models. Therefore, we present a fused part-of-speech vectors and attention-based model (FAM). In our model, we fuse part-of-speech vectors and feature word embeddings as the representation of features, giving deep semantics to mapping features. And we adopt Multi-Head attention mechanism to train the cross-domain sentiment classifier to obtain the connection between different features. The results of 12 groups comparative experiments on the Amazon dataset demonstrate that our model outperforms all baseline models in this paper.

A Literature Review on Cross Domain Sentiment Analysis Using Machine learning

Sentiment analysis is the field of NLP which analyzes the sentiments of text written by users on online sites in the form of reviews. These reviews may be either in the form of a word, sentence, document, or ratings. These reviews are used as datasets when applied to train a classifier. These datasets are applied in the annotated form with the positive, negative or neutral labels as an input to train the classifier. This trained classifier is used to test other reviews, either in the same or different domains to know like or dislike of the user for the related field. Various researches have been done in single and cross domain sentiment analysis. The new methods proposed are overcoming the previous ones but according to this survey, no methods best suit the proposed work. In this article, the authors review the methods and techniques that are given by various researchers in cross domain sentiment analysis and how those are compared with the pre-existing methods for the related work.

Effect of Supervised and Unsupervised Algorithm for Cross Domain Sentiment Analysis

Today we are living in the "information age" where data is the capital of the new economy. With the rapidly growing data every day on online portals and social networking websites, today industries are collecting and analyzing more data than before. Though data is readily available but finding valuable insights out of it is a real task. With easy accessibility of the data, new technologies, and a cultural shift towards data-driven decision making drives the need for Sentiment Analysis (SA) and makes it relevant in most of the domains like politics, marketing, healthcare, etc. This rapidly increasing information on different domains has motivated researchers to develop a cross-domain sentiment analysis model. For the development of this model, we have analyzed the performance of supervised and unsupervised models on benchmark datasets for the cross-domain analysis. The models chosen for the supervised is the Support Vector Machine (SVM) and for the unsupervised approach we have used a combination of Vader wherein the testing results showed that the supervised algorithms performed well in comparison to the unsupervised algorithm.

Cross-D-vectorizers: a set of feature-spaces for cross-domain sentiment analysis from consumer review

Supervised sentiment classification approaches require labeled training (source) and testing (target) dataset. Generation of such datasets demands substantial time and effort but cross-domain classification minimizes the effort by considering two different domains for source and target datasets. In this paper, we propose Cross-D-Vectorizers i.e., a set of three sentiment n-gram feature-spaces (Lexical-TFIDF, Lex-Delta-TFIDF and SEND) for the purpose of cross-domain analysis. We construct the features by extracting sentiment unigrams combination with intensifiers and negations from the source dataset. By utilizing an existing lexicon the scores of these features are computed in three different procedures. The scores for each feature are computed by multiplying sentiment value with corresponding TFIDF rating, Delta-TFIDF rating and feature-importance-values (FIV) respectively. Importance-value for each SEND (Sentiment wEight ofN-grams inDataset) feature is calculated by multiplying the number of times the feature appears in the review and the logarithm of its inverse frequency in the corpus. We experiment by using Maximum Entropy, Support Vector Machine and K-Nearest Neighbors classifiers on three benchmark datasets and one proposed dataset for cross-domain classification. Proposed approach show improved results in comparison with existing methods. The advantage of our approach is the complexity of system reduces by considering sentiment n-grams as domain independent features instead of any n-grams.

Instance Weighting Domain Adaptation Using Distance Kernel

Domain adaptation methods aims to improve the accuracy of the target predictive classifier while using the patterns from a related source domain that has large number of labeled data. In this paper, we introduce new kernel weight domain adaptation method based on smoothness assumption of classifier. We propose new simple and intuitive method that can improve the learning of target data by adding distance kernel based cross entropy term in loss function. Distance kernel refers to a matrix which denotes distance of each instances in source and target domain. We efficiently reduced the computational cost by using the stochastic gradient descent method. We evaluated the proposed method by using synthetic data and cross domain sentiment analysis tasks of Amazon reviews in four domains. Our empirical results showed improvements in all 12 domain adaptation experiments.

Topic and Sentiment Words Extraction in Cross-Domain Product Reviews

Sentiment analysis is very popular in natural language processing and text mining. The traditional sentiment analysis methods use supervised and unsupervised classifiers in a single domain and achieve good results. When training data and test data come from different domains, these methods become poor. The problem of cross-domain opinion analysis is that it is not easy to get a large number of tagged data sets and it is impossible to tag all the data in the interesting domains. We propose an extraction method for topic and sentiment words based on conditional random field and syntactic structure to analyze the sentiment orientation of Chinese product reviews. We aim to extract topic and sentiment words from target domain and identify their sentiment orientation with one or a few topic and sentiment words being tagged in the source domain and words in the target domain without any tagged information. Our experimental results show that our method is effective in cross-domain sentiment analysis.

Approaches to Cross-Domain Sentiment Analysis: A Systematic Literature Review

A sentiment analysis has received a lot of attention from researchers working in the fields of natural language processing and text mining. However, there is a lack of annotated data sets that can be used to train a model for all domains, which is hampering the accuracy of sentiment analysis. Many research studies have attempted to tackle this issue and to improve cross-domain sentiment classification. In this paper, we present the results of a comprehensive systematic literature review of the methods and techniques employed in a cross-domain sentiment analysis. We focus on studies published during the period of 2010–2016. From our analysis of those works, it is clear that there is no perfect solution. Hence, one of the aims of this review is to create a resource in the form of an overview of the techniques, methods, and approaches that have been used to attempt to solve the problem of cross-domain sentiment analysis in order to assist researchers in developing new and more accurate techniques in the future.

Prediction Reweighting for Domain Adaptation.

There are plenty of classification methods that perform well when training and testing data are drawn from the same distribution. However, in real applications, this condition may be violated, which causes degradation of classification accuracy. Domain adaptation is an effective approach to address this problem. In this paper, we propose a general domain adaptation framework from the perspective of prediction reweighting, from which a novel approach is derived. Different from the major domain adaptation methods, our idea is to reweight predictions of the training classifier on testing data according to their signed distance to the domain separator, which is a classifier that distinguishes training data (from source domain) and testing data (from target domain). We then propagate the labels of target instances with larger weights to ones with smaller weights by introducing a manifold regularization method. It can be proved that our reweighting scheme effectively brings the source and target domains closer to each other in an appropriate sense, such that classification in target domain becomes easier. The proposed method can be implemented efficiently by a simple two-stage algorithm, and the target classifier has a closed-form solution. The effectiveness of our approach is verified by the experiments on artificial datasets and two standard benchmarks, a visual object recognition task and a cross-domain sentiment analysis of text. Experimental results demonstrate that our method is competitive with the state-of-the-art domain adaptation algorithms.