Sentiment Classification Task Research Articles

Relational Temporal Graph Reasoning for Dual-task Dialogue Language Understanding.

Dual-task dialog language understanding aims to tackle two correlative dialog language understanding tasks simultaneously via leveraging their inherent correlations. In this paper, we put forward a new framework, whose core is relational temporal graph reasoning. We propose a speaker-aware temporal graph (SATG) and a dual-task relational temporal graph (DRTG) to facilitate relational temporal modeling in dialog understanding and dual-task reasoning. Besides, different from previous works that only achieve implicit semantics-level interactions, we propose to model the explicit dependencies via integrating prediction-level interactions. To implement our framework, we first propose a novel model Dual-tAsk temporal Relational rEcurrent Reasoning network (DARER), which first generates the context-, speaker- and temporal-sensitive utterance representations through relational temporal modeling of SATG, then conducts recurrent dual-task relational temporal graph reasoning on DRTG, in which process the estimated label distributions act as key clues in prediction-level interactions. And the relational temporal modeling in DARER is achieved by relational graph convolutional networks (RGCNs). Then we further propose Relational Temporal Transformer (ReTeFormer), which achieves fine-grained relational temporal modeling via Relation- and Structure-aware Disentangled Multi-head Attention. Accordingly, we propose DARER with ReTeFormer (DARER2), which adopts two variants of ReTeFormer to achieve the relational temporal modeling of SATG and DTRG, respectively. The extensive experiments on different scenarios verify that our models outperform state-of-the-art models by a large margin. Remarkably, on the dialog sentiment classification task in the Mastodon dataset, DARER and DARER2 gain relative improvements of about 28% and 34% over the previous best model in terms of F1.

Deep learning-based sentiment classification in Amharic using multi-lingual datasets

The analysis of emotions expressed in natural language text, also known as sentiment analysis, is a key application of natural language processing (NLP). It involves assigning a positive, negative (sometimes also neutral) value to opinions expressed in various contexts such as social media, news, blogs, etc. Despite its importance, sentiment analysis for under-researched languages like Amharic has not received much attention in NLP yet due to the scarcity of resources required to train such methods. This paper examines various deep learning methods such as CNN, LSTM, FFNN, BiLSTM, and transformers, as well as memory-based methods like cosine similarity, to perform sentiment classification using the word or sentence embedding techniques. This research includes training and comparing mono-lingual or cross-lingual models using social media messages in Amharic on Twitter. The study concludes that the lack of training data in the target language is not a significant issue since the training data 1) can be machine translated from other languages using machine translation as a data augmentation technique [33], or 2) cross-lingual models can capture the semantics of the target language, even when trained on another language (e.g., English). Finally, the FFNN classifier, which combined the sentence transformer and the cosine similarity method, proved to be the best option for both 3-class and 2-class sentiment classification tasks, achieving 62.0% and 82.2% accuracy, respectively.

Adaptive Local Context and Syntactic Feature Modeling for Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis is a fine-grained sentiment analysis task that consists of two types of subtasks: aspect term extraction and aspect sentiment classification. In the aspect term extraction task, current methods suffer from the lack of fine-grained information in aspect term extraction and difficulty in identifying aspect term boundaries. In the aspect sentiment classification task, the current aspect sentiment classifier cannot adapt itself to the text and determine the local context. To address these two challenges, this work proposes an adaptive semantic relative distance approach based on dependent syntactic analysis, which uses adaptive semantic relative distance to determine the appropriate local context for each text and increase the accuracy of sentiment analysis. Meanwhile, the study also predicts the current word labels by combining local information features extracted by local convolutional neural networks and global information features to precisely locate the word labels. In two subtasks, our proposed model improves accuracy and F1 scores on the SemEval-2014 Task 4 Restaurant and Laptop datasets compared to the state-to-the-art approaches, especially in the aspect sentiment classification subtask.

An Ontology-based Sentiment Analysis Approach To Discovering Hidden Affected Objects

Nowadays, the topic of sentiment analysis has attracted a wide range of artificial intelligence technologies suchas Natural Language Processing (NLP), neural network. Long Short-Term Memory (LSTM), a deep neural network, has proven the effiency in the task of sentiment classification. However, there is a shortage of semantic knowledge from such classification results. This paper introduces a sentiment analysis approach based on a knowledge model to discover hidden affected objects, which combines ONtology and SentiMent technology, named ONSEM. In order to confirm the efficiency of the ONSEM framework, an experiment on a sentiment analysis dataset was developed and then yielded promising experimental results.

Cross‐domain sentiment classification using decoding‐enhanced bidirectional encoder representations from transformers with disentangled attention

SummaryCross‐domain sentiment classification is a significant task of sentiment analysis that objectives to predict the opinion orientation of text documents in the target domain by using the source domain's learned classifier. Most of the existing approaches of domain‐adaptation in sentiment classification focus on sharing low‐dimensional features across the domain using domain independent and specific features to mitigate the gap between domains. Earlier cross‐domain sentiment classification approaches mainly focused on document level and sentence level, they cannot consider the full impact of aspect words, position of the words, and long‐term dependencies. To address this concern, we propose a model for cross‐domain sentiment classification, which is based on decoding‐enhanced BERT with disentangled attention (DeBERTa). DeBERTa is a pretrained language model based on transformer architecture. In this article, we perform sentence and aspect embedding to mine wordpiece information from text document. DeBERTa language‐model utilize disentangled attention mechanism and an enhanced mask decoder to understand the expression features. Disentangled attention mechanism is used to encode each word into two vectors (i.e., content and position vector). In order to predict the masked tokens during model pretraining, an enhanced mask decoder is employed, which incorporates absolute positions in the decoding layer. Finally, experiments are conducted on the benchmark dataset that demonstrates the superiority of fine‐tuned DeBERTa model for cross‐domain sentiment classification tasks.

RAG-TCGCN: Aspect Sentiment Analysis Based on Residual Attention Gating and Three-Channel Graph Convolutional Networks

Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task that mainly judges the polarity of a given aspect word in a review. Current methods mainly use graph networks to do aspect-level sentiment classification tasks, most of which use syntactic or semantic graphs, and utilize attention mechanisms to interact and correlate aspect terms and contexts to obtain more useful feature representations. However, these methods may ignore some insignificant syntactic structures and some implicit information in some sentences. The attention mechanism then easily loses the original information, which eventually leads to inaccurate sentiment analysis. In order to solve this problem, this paper proposes a model based on residual attention gating and three-channel graph convolutional network (RAG-TCGCN). Firstly, the model uses a three-channel network composed of syntactic information, semantic information, and public information to simultaneously optimize and fuse through the multi-head attention mechanism to solve the problem of sentences without significant syntactic structure and with implicit information. Through the residual attention gating mechanism the problem of loss of original information is solved. Experimental verification shows that the accuracy and F1 value of the model are improved on the three public datasets.

Chinese Text Sentiment Classification Based on ERNIE and BiLSTM-AT

Chinese text sentiment classification is a sub-task of natural language processing. However, when text representation is carried out, the polysemy of a word cannot be processed when using the traditional language model to construct the word vector, and the long-distance text information cannot be fully extracted when extracting text features. To solve this problem, this paper proposes a text sentiment classification model combining ERNIE and BiLSTM-AT. First, the pre-training model ERNIE is used to obtain the word vector representation of the fused statement context. Then, the bidirectional long-short-term memory neural network is used to extract the context information and depth semantic information of the text. Then, the attention mechanism is used to assign the corresponding weights to the hidden layer vectors of each time step output by the BiLSTM layer, and the weighted summation is integrated into the sentence features. Finally, the softmax function is used to calculate the probability distribution of the emotional category of the text in the output layer. The results show that the proposed model can achieve high accuracy on both hotel reviews and takeaway reviews. Based on the pre-training model, adding bidirectional long-term and short-term memory network and attention mechanism is beneficial to improve the classification effect of the model, and has certain practicability in text sentiment classification tasks.

Network Public Opinion Risk Prediction and Judgment Based on Deep Learning: A Model of Text Sentiment Analysis.

Under the background of the gradual development and popularization of mobile Internet information technology, this paper realizes network public opinion monitoring and emotion analysis based on the deep learning method, aiming at the research needs of people's ideological changes and emotional trends. Aiming at the shortcomings of sentiment dictionaries or machine learning methods in sentiment analysis tasks, this paper builds a sentiment classification model based on deep learning methods. First, the current main text preprocessing methods are introduced, and then a sentiment classification model, BCBL, is proposed, combining BERT, CNN, and Bi LSTM. Compared with traditional models, BCBL can better complete text sentiment classification tasks on standard datasets. Next, in view of the problem that BCBL does not consider the distribution of vocabulary weights, an attention mechanism is introduced to improve BCBL, and then the BCBL-Att model is proposed. Set up multiple sets of comparative experiments again and find that the classification effect and overall performance of BCBL-Att on standard datasets are better than BCBL, indicating that BCBL-Att has more advantages in text sentiment classification tasks.

Evaluation of deep learning models for sentiment analysis

Sentiment analysis is a task that belongs to natural language processing and it is highly used in texts extracted from social networks. This task consists of assigning the labels or classes: positive, negative or neutral to the text. However, analyzing a piece of text extracted from social networks to determine if it represents a positive or negative sentiment is a difficult task, because social media texts contain slangs, typographical errors and cultural context. The shortcomings of traditional frequency based feature extraction models such as bag of words or TF-IDF affect the accuracy of sentiment classification. To improve the precision in the sentiment classification task, it is possible to use natural language modelling methods that are able to learn contextual information from words. In this work, word embedding such as Word2Vec, GloVe and Doc2VecC with different dimensions are used. The resulting word vectors will be used to train recurring neural networks such as LSTM, BiLSTM, GRU and BiGRU, to improve sentiment classification.

Attention-enabled gated spiking neural P model for aspect-level sentiment classification

Gated spiking neural P (GSNP) model is a recently developed recurrent-like network, which is abstracted by nonlinear spiking mechanism of nonlinear spiking neural P systems. In this study, a modification of GSNP is combined with attention mechanism to develop a novel model for sentiment classification, called attention-enabled GSNP model or termed as AGSNP model. The AGSNP model has two channels that process content words and aspect item respectively, where two modified GSNPs are used to obtain dependencies between content words and between aspect words. Moreover, two attention components are used to establish semantic correlation between content words and aspect item. Comparative experiments on three real data sets and several baseline models are conducted to verify the effectiveness of the AGSNP model. The comparison results demonstrate that the AGSNP model is competent for aspect-level sentiment classification tasks.

Triplet Contrastive Learning for Aspect Level Sentiment Classification

The domain of Aspect Level Sentiment Classification, in which the sentiment toward a given aspect is analyzed, attracts much attention in NLP. Recently, the state-of-the-art Aspect Level Sentiment Classification methods are devised by using the Graph Convolutional Networks to deal with both the semantics and the syntax of the sentence. Generally, the parsing of syntactic structure inevitably incorporates irrelevant information toward the aspect. Besides, the syntactic and semantic alignment and uniformity that contribute to the sentiment delivery is currently neglected during processing. In this work, a Triplet Contrastive Learning Network is developed to coordinate the syntactic information and the semantic information. To start with, the aspect-oriented sub-tree is constructed to replace the syntactic adjacency matrix. Further, a sentence-level contrastive learning scheme is proposed to highlight the features of sentiment words. Based on The Triple Contrastive Learning, the syntactic information and the semantic information are thoroughly interacted and coordinated whilst the global semantics and syntax can be exploited. Extensive experiments are performed on three benchmark datasets and achieve accuracies (BERT-based) of 87.40, 82.80, 77.55 on Rest14, Lap14, and Twitter datasets, which demonstrate that our approach achieves state-of-the-art results in Aspect Level Sentiment Classification task.

Towards solving NLP tasks with optimal transport loss

Loss functions are essential to computing the divergence of a model’s predicted distribution from the ground truth. Such functions play a vital role in machine learning algorithms as they steer the learning process. Most common loss functions in natural language processing (NLP), such as Kullback–Leibler (KL) and Jensen–Shannon (JS) divergences, do not base their computations on the properties of label coordinates. Label coordinates can help encode the inter-label relationships. For the sentiment classification task, strongly positive sentiment is closer to positive than strongly negative sentiment. Incorporating such information in the computations of the probability divergence can facilitate the model’s learning dynamics.In this work, we study an under-explored loss function in NLP — Wasserstein Optimal Transport (OT) — which takes label coordinates into account and thus allows the learning algorithm to incorporate inter-label relations. However, the limited applications of OT-based loss owe to the challenges in defining quality label coordinates. We explore the current limitations of learning with OT and provide an algorithm that jointly learns label coordinates with the model parameters. We show the efficacy of OT on several text classification tasks such as sentiment analysis and emotion recognition in conversation. We also discuss the limitations of the approach. The source codes pertaining to this work are publicly available at: https://github.com/declare-lab/NLP-OT.

Sentiment-aware multimodal pre-training for multimodal sentiment analysis

Pre-trained models, together with fine-tuning on downstream labeled datasets, have demonstrated great success in various tasks, including multimodal sentiment analysis. However, most most multimodal pre-trained models focus on learning general lexical and/or visual information, while ignoring sentiment signals. To address this problem, we propose a sentiment-aware multimodal pre-training (SMP) framework for multimodal sentiment analysis. In particular, we design a cross-modal contrastive learning module based on the interactions between visual and textual information, and introduce additional sentiment-aware pre-training objectives (e,g., fine-grained sentiment labeling) to capture fine-grained sentiment information from sentiment-rich datasets. We adopt two objectives (i.e., masked language modeling and masked auto-encoders) to capture semantic information from text and images. We conduct a series of experiments on sentence-level and target-oriented multimodal sentiment classification tasks, wherein the results of our SMP model exceeds the state-of-the-art results. Additionally, ablation studies and case studies are conducted to verify the effectiveness of our SMP model.

Aspect opinion routing network with interactive attention for aspect-based sentiment classification

Aspect-based sentiment classification task aims at discovering the sentiment polarity on a specific aspect term. Since a sentence often contains several sentiments for different aspects, it is very important to capture the correspondences between aspects and opinion words in a sentence. To this end, we propose an aspect opinion interactive attention routing network model, which adopts an interactive attention mechanism between specific aspect embeddings learned from graph convolutional network and opinion semantic embeddings learned from three bi-directional long short-term memory networks. Furthermore, dynamic routing is applied to the neural network so that its output values are constantly regulated during training to generate feature representations of specific aspects related to opinion word. Based on the above, our approach identifies opinion words feature information related to specific aspect words in context. Experimental results demonstrate that the model outperforms 16 existing methods in terms of accuracy and macro-average F1 score on most of the benchmark datasets.

Topic Sentiment Analysis for Twitter Data in Indian Languages Using Composite Kernel SVM and Deep Learning

Sentiment analysis of public opinions on social networks, such as Twitter or Facebook, can provide us with valuable information, which has a wide range of applications. But the efficiency and accuracy of the automated methods for Twitter sentiment analysis are hindered by the special characteristics of the Twitter data. The Twitter data is generally noisy, high-dimensional, and it has complex syntactic and semantic structures. Sentiment analysis of Twitter data in Indian languages is more challenging because the data is multilingual and code-mixed. In this article, we propose various composite kernel functions, each of which is used with Support Vector Machines (SVM) for developing a model for topic sentiment analysis of Twitter data in Indian languages. Each composite kernel function is constructed by taking the weighted summation of multiple single kernel functions defined by us. In addition to our proposed composite kernel SVM method, we use several state-of-the-art deep learning classifiers for topic sentiment classification. Since any suitable Twitter dataset in Indian languages is not available for conducting our experiments, we have developed our own datasets by collecting tweets related to five different Twitter trending topics in India. To prove the robustness and generalization capability of the proposed models, they are also evaluated on the US airline Twitter dataset which is a publicly available benchmark English dataset. The empirical study exhibits that the proposed composite kernel SVM method is effective for the sentiment classification task. In the case of Indian language datasets, the proposed composite kernel SVM method achieves the highest average accuracy of 74% and the highest average F-score of 0.73. On the other hand, the deep learning-based method achieves the average accuracy and the average F-score of 71.31% and 0.70, respectively. In the case of the US airline Twitter dataset, the proposed composite kernel SVM method achieves the average accuracy of 83% and the average F-score of 0.82, which are higher than that of the deep learning-based method.

Knowledge distillation for BERT unsupervised domain adaptation

A pre-trained language model, BERT, has brought significant performance improvements across a range of natural language processing tasks. Since the model is trained on a large corpus of diverse topics, it shows robust performance for domain shift problems in which data distributions at training (source data) and testing (target data) differ while sharing similarities. Despite its great improvements compared to previous models, it still suffers from performance degradation due to domain shifts. To mitigate such problems, we propose a simple but effective unsupervised domain adaptation method, adversarial adaptation with distillation (AAD), which combines the adversarial discriminative domain adaptation (ADDA) framework with knowledge distillation. We evaluate our approach in the task of cross-domain sentiment classification on 30 domain pairs, advancing the state-of-the-art performance for unsupervised domain adaptation in text sentiment classification.

Integrating Multiclass Light Weighted BiLSTM Model for Classifying Negative Emotions.

With the continuous development of social networks, Weibo has become an essential platform for people to share their opinions and feelings in daily life. Analysis of users' emotional tendencies can be effectively applied to public opinion control, public opinion surveys, and product recommendations. However, the traditional deep learning algorithm often needs a large amount of data to be retained to obtain a better accuracy when faced with new work tasks. Given this situation, a multiclassification method of microblog negative sentiment based on MAML (model-agnostic metalearning) and BiLSTM (bidirectional extended short-term memory network) is proposed to represent the microblog text word vectorization and the combination of MAML and BiLSTM is constructed. The model of BiLSTM realizes the classification of negative emotions on Weibo and updates the parameters through machine gradient descent; the metalearner in MAML calculates the sum of the losses of multiple pieces of training, performs a second gradient descent, and updates the metalearner parameters. The updated metalearner can quickly iterate when faced with a new Weibo negative sentiment classification task. The experimental results show that compared with the prepopular model, on the Weibo negative sentiment dataset, the precision rate, recall rate, and F1 value are increased by 1.68 percentage points, 2.86 percentage points, and 2.27 percentage points, respectively.

Sentiment Analysis of Animated Film Reviews Using Intelligent Machine Learning.

Film is an essential expression of a country's cultural soft power in terms of cross-cultural exchange. In addition, film is also the most direct and favourable means of communication. Along with the expansion and development of the Chinese film market, outstanding animation films have emerged in recent years. Animated films have both artistic and commercial properties and can not only have a cultural impact but can also contribute to economic growth. For this reason, our country is now paying more and more attention to the development of animated films. Specifically, animated films not only represent a country's cultural soft power and national image, but they are also a symbol of the strength of a country's cultural industry. As a reflection and extension of China's culture and ideology, animated films play an important role in enhancing cultural confidence and cultural export. In recent years, China's economy has shown a steady and sustained growth trend. At the same time, with the rapid development of internet technology, social networking has gradually penetrated into all aspects of people's lives. Various social networking forums, websites, and sites have emerged. While satisfying a wide range of needs, they also provide information on product reviews, social reviews, and service reviews. These reviews contain feedback from the reviewer about the subject of the review. Tapping into the emotions in these reviews can provide consumers with shopping references and help businesses to optimise their products and improve their business strategies. With the help of modern internet technology and information technology, the modern movie industry, such as Cat's Eye Movies and other internet entertainment service platforms, has developed a model of online ticketing, offline movie viewing, and online reviews and feedback. The content of the reviews on these movie websites fully reflects the attitudinal views of the movie-going community. These reviews play a decisive role in the box office and the further spread of culture. As a result, in order to better understand the audience's emotional tendencies and needs, it is necessary to carry out sentiment analysis and deep semantic mining of animated film reviews. As the evaluation of film works considers many factors and is complex and variable, the choice of model is crucial in the process of sentiment analysis. Machine learning models represented by deep neural networks are more tolerant of sentence noise and have strong information discrimination and feature self-learning capabilities. As a result, intelligent machine learning is more advantageous for sentiment classification tasks. This study is a combination of textual data mining and statistical analysis from the perspective of viewers' comments to study the online reviews of animation films from different countries. At the same time, this research hopes to uncover meaningful information from the film reviews and the gap between Chinese and other countries' animation films, in order to provide a little help for the rise of domestic animation films.

A Multi-Modal Convolutional Neural Network Model for Intelligent Analysis of the Influence of Music Genres on Children's Emotions.

The influence of music genres on children's emotional intelligence is one of the hot topics in the field of multi-modal emotion research. How to fuse multi-modal information has an important impact on children's emotional analysis. Most of the current research is based on transformer, in which the self-attention mechanism module is improved to achieve the fusion effect of multi-modal information. However, it is difficult for these methods to effectively capture the effective information of different modalities. Therefore, for the task of the influence of music genres on children's emotions, this paper proposes a transformer-based multi-modal convolutional neural network. The first is to use the BiLSTM sub-network model to extract the video and audio features and use the BERT sub-network to extract the text features. Secondly, this paper uses the improved transformer cross-modal fusion module to effectively fuse different types of modal information. Finally, the transformer module is used to judge the information of different modalities and analyze the emotion from the multi-modal information. At the same time, a large number of experiments prove that the model based on multi-modal convolutional neural network proposed in this paper surpasses other methods in prediction accuracy and effectively improves the accuracy of sentiment classification tasks.

A BERT-Based Aspect-Level Sentiment Analysis Algorithm for Cross-Domain Text.

Cross-domain text sentiment analysis is a text sentiment classification task that uses the existing source domain annotation data to assist the target domain, which can not only reduce the workload of new domain data annotation, but also significantly improve the utilization of source domain annotation resources. In order to effectively achieve the performance of cross-domain text sentiment classification, this paper proposes a BERT-based aspect-level sentiment analysis algorithm for cross-domain text to achieve fine-grained sentiment analysis of cross-domain text. First, the algorithm uses the BERT structure to extract sentence-level and aspect-level representation vectors, extracts local features through an improved convolutional neural network, and combines aspect-level corpus and sentence-level corpus to form a sequence sentence pair. Then, the algorithm uses domain adversarial neural network to make the feature representation extracted from different domains as indistinguishable as possible, that is, the features extracted from the source domain and the target domain have more similarity. Finally, by training the sentiment classifier on the source domain dataset with sentiment labels, it is expected that the classifier can achieve a good sentiment classification effect in both source and target domain, and achieve sentence-level and aspect-level sentiment classification. At the same time, the error pooled values of the sentiment classifier and the domain adversary are passed backwards to realize the update and optimization of the model parameters, thereby training a model with cross-domain analysis capability. Experiments are carried out on the Amazon product review dataset, and accuracy and F1 value are used as evaluation indicators. Compared with other classical algorithms, the experimental results show that the proposed algorithm has better performance.