Aspect Sentiment Classification Research Articles

An efficient sentiment analysis using topic model based optimized recurrent neural network

Abstract In recent years, topic modeling and deep neural network-based methods have attracted much attention in sentiment analysis of online reviews. This paper presents a hybrid topic model-based approach for aspect extraction and sentiment classification of textual reviews. Latent Dirichlet allocation applied for aspect extraction and two-layer bi-directional long short-term memory (LSTM) for sentiment classification. This work also proposes a hill climbing-based approach for tunning model hyperparameters. The proposed model evaluated on three different datasets. Compared to the single-layer Bi-LSTM model, the proposed model gives 95, 95, and 86% accuracy for the movie, mobile, and hotel domain, respectively.

Aspect-Level Sentiment Analysis Based on Bidirectional-GRU in SIoT

A variety of independent research activities have recently been undertaken to explore the feasibility of incorporating social networking principles into the Internet of Things solutions. The resulting model, called the Social Internet of Things, has the potential to be more powerful and competitive in supporting new IoT applications and networking services. This paper’s main contribution is in sentiment analysis, which aims to predict aspect sentiments to improve the making of automated decisions and communication between associates in the social internet of things. In recent years, to analyze sentiment polarity at a subtle level, sentiment classification has become a primetime attraction. Current approaches commonly use the Long-Short Term Memory network to figure aspects and contexts separately. Usually, they perform sentiment classification using simple attention mechanisms and avoiding the bilateral information between sentences and their corresponding aspects. Therefore, the results are not satisfactory. This manuscript intends to develop a new Bidirectional gated recurrent unit model by depending on natural language processing for fully-featured mining to perform the aspect-level sentiment classification task. Our proposed model uses the Bidirectional gated recurrent unit network to acquire the dependency-based semantic analysis of sentences and their corresponding terms compared to earlier work. At the same time, it proposes a method to learn the sentiment polarity of terms in sentences. To check out our model’s achievements, we perform several experiments on datasets, namely, (LAPTOP, RESTUARANT, and TWITTER). Our experiment results demonstrate that our model has achieved compelling performance and efficiency improvements in aspect sentiment classification compared with several existing models.

An improved aspect-category sentiment analysis model for text sentiment analysis based on RoBERTa

The aspect-category sentiment analysis can provide more and deeper information than the document-level sentiment analysis, because it aims to predict the sentiment polarities of different aspect categories in the same text. The main challenge of aspect-category sentiment analysis is that different aspect categories may present different polarities in the same text. Previous studies combine the Long Short-Term Memory (LSTM) and attention mechanism to predict the sentiment polarity of the given aspect category, but the LSTM-based methods are not really bidirectional text feature extraction methods. In this paper, we propose a multi-task aspect-category sentiment analysis model based on RoBERTa (Robustly Optimized BERT Pre-training Approach). Treating each aspect category as a subtask, we employ the RoBERTa based on deep bidirectional Transformer to extract features from both text and aspect tokens, and apply the cross-attention mechanism to guide the model to focus on the features most relevant to the given aspect category. According to the experimental results, the proposed model outperforms other models for comparison in aspect-category sentiment analysis. Furthermore, the influencing factors of our proposed model are also analyzed.

A Summary of Aspect-based Sentiment Analysis

Aspect-based sentiment analysis is a subclass of sentiment analysis tasks, focusing on judging the sentiment tendency of entities or attributes, and has received extensive attention due to fine-grained analysis results. The current research mostly summarizes the research from a single point of aspect extraction or sentiment classification, and lacks of research on the methods and problems in aspect-based sentiment analysis from the overall perspective. By using the method of literature investigation, an in-depth overview of the typical methods and solutions is given in the field of aspect-based sentiment analysis from three aspects: aspect information extraction, aspect information sentiment classification and aspect-based sentiment analysis joint modeling. Then, we discuss some of the most challenging problems existing in the field in terms of model adaptability, error transmission, knowledge and reasoning, and proposes future research opportunities in terms of strengthening knowledge representation, multi-task learning and integrating knowledge and reasoning from the overall perspective of practical application.

A span-based model for aspect terms extraction and aspect sentiment classification

Sentiment analysis is a field of natural language processing, which is used to identify and extract opinions and attitudes from text. Aspect-based sentiment analysis aims to extract aspect terms and predict sentiment categories of the opinion aspects. It includes two subtasks: aspect terms extraction and aspect sentiment classification. However, previous studies regarded them as two independent tasks and solve them, respectively, which has limitations for practical application. In this paper, we combine the requirements of two subtasks to propose a new aspect-based sentiment analysis framework based on span, which is a simple and effective joint model to generate all aspects and corresponding sentiment polarities of the input sentences. Specifically, dual gated recurrent units which are used to extract the respective representation of each task can process sequence information better, and an interaction layer which is used to consider the relationship between the representations. Experiments on three benchmark datasets show that the proposed framework outperforms the state-of-the-art baseline models.

Convolutional multi-head self-attention on memory for aspect sentiment classification

This paper presents a method for aspect based sentiment classification tasks, named convolutional multi-head self-attention memory network (CMA-MemNet). This is an improved model based on memory networks, and makes it possible to extract more rich and complex semantic information from sequences and aspects. In order to fix the memory network's inability to capture context-related information on a word-level, we propose utilizing convolution to capture n-gram grammatical information. We use multi-head self-attention to make up for the problem where the memory network ignores the semantic information of the sequence itself. Meanwhile, unlike most recurrent neural network (RNN) long short term memory (LSTM), gated recurrent unit (GRU) models, we retain the parallelism of the network. We experiment on the open datasets SemEval-2014 Task 4 and SemEval-2016 Task 6. Compared with some popular baseline methods, our model performs excellently.

Knowing What, How and Why: A Near Complete Solution for Aspect-Based Sentiment Analysis

Target-based sentiment analysis or aspect-based sentiment analysis (ABSA) refers to addressing various sentiment analysis tasks at a fine-grained level, which includes but is not limited to aspect extraction, aspect sentiment classification, and opinion extraction. There exist many solvers of the above individual subtasks or a combination of two subtasks, and they can work together to tell a complete story, i.e. the discussed aspect, the sentiment on it, and the cause of the sentiment. However, no previous ABSA research tried to provide a complete solution in one shot. In this paper, we introduce a new subtask under ABSA, named aspect sentiment triplet extraction (ASTE). Particularly, a solver of this task needs to extract triplets (What, How, Why) from the inputs, which show WHAT the targeted aspects are, HOW their sentiment polarities are and WHY they have such polarities (i.e. opinion reasons). For instance, one triplet from “Waiters are very friendly and the pasta is simply average” could be (‘Waiters’, positive, ‘friendly’). We propose a two-stage framework to address this task. The first stage predicts what, how and why in a unified model, and then the second stage pairs up the predicted what (how) and why from the first stage to output triplets. In the experiments, our framework has set a benchmark performance in this novel triplet extraction task. Meanwhile, it outperforms a few strong baselines adapted from state-of-the-art related methods.

Multi-task learning for aspect term extraction and aspect sentiment classification

Aspect sentiment classification has a dependency over the aspect term extraction. The majority of the existing studies tackle these two problems independently, i.e., while performing aspect sentiment classification, it is assumed that the aspect terms are pre-identified. However, such assumptions are neither practical nor appropriate. In this paper, we address these impractical limitations and propose a multi-task learning framework for the identification and classification of aspect terms in a unified model. At first, the proposed approach employs a BiLSTM followed by a self-attention mechanism to identify the aspect terms in a given sentence. Subsequently, the architecture utilizes a CNN framework to predict the sentiments of the identified aspect terms. We evaluate our proposed approach for the three benchmark datasets across two languages, i.e., English and Hindi. Experimental results suggest that the proposed multi-task model achieves competitive performance with reduced complexity (i.e., a single model for the two tasks compared to two separate models for each task) for both the languages.

SEML: A Semi-Supervised Multi-Task Learning Framework for Aspect-Based Sentiment Analysis

Aspect-Based Sentiment Analysis (ABSA) involves two sub-tasks, namely Aspect Mining (AM) and Aspect Sentiment Classification (ASC), which aims to extract the words describing aspects of a reviewed entity (e.g., a product or service) and analyze the expressed sentiments on the aspects. As AM and ASC can be formulated as a sequence labeling problem to predict the aspect or sentiment labels of each word in the review, supervised deep sequence learning models have recently achieved the best performance. However, these supervised models require a large number of labeled reviews which are very costly or unavailable, and they usually perform only one of the two sub-tasks, which limits their practical use. To this end, this paper proposes a SEmi-supervised Multi-task Learning framework (called SEML) for ABSA. SEML has three key features. (1) SEML applies Cross-View Training (CVT) to enable semi-supervised sequence learning over a small set of labeled reviews and a large set of unlabeled reviews from the same domain in a unified end-to-end architecture. (2) SEML solves the two sub-tasks simultaneously by employing three stacked bidirectional recurrent neural layers to learn the representations of reviews, in which the representations learned from different layers are fed into CVT, AM and ASC, respectively. (3) SEML develops a Moving-window Attentive Gated Recurrent Unit (MAGRU) for the three recurrent neural layers to enhance representation learning and prediction accuracy, as nearby contexts within a moving-window in a review can provide important semantic information for the prediction task in ABSA. Finally, we conduct extensive experiments on ABSA over four review datasets from the SemEval workshops. Experimental results show that SEML significantly outperforms the state-of-the-art models.

Weakly Supervised Framework for Aspect-Based Sentiment Analysis on Students’ Reviews of MOOCs

Students’ feedback is an effective mechanism that provides valuable insights about teaching-learning process. Handling opinions of students expressed in reviews is a quite labour-intensive and tedious task as it is typically performed manually by the human intervention. While this task may be viable for small-scale courses that involve just a few students’ feedback, it is unpractical for large-scale cases as it applies to online courses in general, and MOOCs, in particular. Therefore, to address this issue, we propose in this paper a framework to automatically analyzing opinions of students expressed in reviews. Specifically, the framework relies on aspect-level sentiment analysis and aims to automatically identify sentiment or opinion polarity expressed towards a given aspect related to the MOOC. The proposed framework takes advantage of weakly supervised annotation of MOOC-related aspects and propagates the weak supervision signal to effectively identify the aspect categories discussed in the unlabeled students’ reviews. Consequently, it significantly reduces the need for manually annotated data which is the main bottleneck for all deep learning techniques. A large-scale real-world education dataset containing around 105k students’ reviews collected from Coursera and a dataset comprising of 5989 students’ feedback in traditional classroom settings are used to perform experiments. The experimental results indicate that our proposed framework attains inspiring performance with respect to both the aspect category identification and the aspect sentiment classification. Moreover, the results suggest that the framework leads to more accurate results than the expensive and labour-intensive sentiment analysis techniques relying heavily on manually labelled data.

基于自注意力的门控卷积神经网络的要素类情感分类研究

基于自注意力的门控卷积神经网络的要素类情感分类研究

Improving aspect-based sentiment analysis via aligning aspect embedding

Aspect-Based Sentiment Analysis (ABSA) is a fine-grained sentiment analysis task, which aims to predict sentiment polarities of given aspects or target terms in text. ABSA contains two subtasks: Aspect-Category Sentiment Analysis (ACSA) and Aspect-Term Sentiment Analysis (ATSA). Aspect embeddings have been extensively used for representing aspect-categories on ACSA task. Based on our observations, existing aspect embeddings cannot properly represent the relation between aspect-categories and aspect-terms. To address this limitation, this paper presents a learning method which trains aspect embeddings according to the relation between aspect-categories and aspect-terms. According to the cosine measure metric we proposed in this paper, the limitation is successfully alleviated in the aspect embeddings which are trained by our method. The trained aspect embeddings can be used as initialization in existing models to solve ACSA task. We conduct experiments on SemEval datasets for ACSA task, and the results indicate that our pre-trained aspect embeddings are capable of improving the performance of sentiment analysis.

Aspect Aware Learning for Aspect Category Sentiment Analysis

Aspect category sentiment analysis (ACSA) is an underexploited subtask in aspect level sentiment analysis. It aims to identify the sentiment of predefined aspect categories. The main challenge in ACSA comes from the fact that the aspect category may not occur in the sentence in most of the cases. For example, the review “ they have delicious sandwiches ” positively talks about the aspect category “ food ” in an implicit manner. In this article, we propose a novel aspect aware learning (AAL) framework for ACSA tasks. Our key idea is to exploit the interaction between the aspect category and the contents under the guidance of both sentiment polarity and predefined categories. To this end, we design a two-way memory network for integrating AAL into the framework of sentiment classification. We further present two algorithms to incorporate the potential impacts of aspect categories. One is to capture the correlations between aspect terms and the aspect category like “sandwiches” and “food.” The other is to recognize the aspect category for sentiment representations like “food” for “delicious.” We conduct extensive experiments on four SemEval datasets. The results reveal the essential role of AAL in ACSA by achieving the state-of-the-art performance.

Enhanced Twofold-LDA Model for Aspect Discovery and Sentiment Classification

There is a need to automatically classify information from online reviews. Customers want to know useful information about different aspects of a product or service and also the sentiment expressed towards each aspect. This article proposes an Enhanced Twofold-LDA model (Latent Dirichlet Allocation), in which one LDA is used for aspect assignment and another is used for sentiment classification, aiming to automatically determine aspect and sentiment. The enhanced model incorporates domain knowledge (i.e., seed words) to produce more focused topics and has the ability to handle two aspects in at the sentence level simultaneously. The experiment results show that the Enhanced Twofold-LDA model is able to produce topics more related to aspects in comparison to the state of arts method ASUM (Aspect and Sentiment Unification Model), whereas comparable with ASUM on sentiment classification performance.

Towards Autoencoding Variational Inference for Aspect-Based Opinion Summary

ABSTRACTAspect-based Opinion Summary (AOS), consisting of aspect discovery and sentiment classification steps, has recently been emerging as one of the most crucial data mining tasks in e-commerce systems. Along this direction, the LDA-based model is considered as a notably suitable approach, since this model offers both topic modeling and sentiment classification. However, unlike traditional topic modeling, in the context of aspect discovery, it is often required some initial seed words, whose prior knowledge is not easy to be incorporated into LDA models. Moreover, LDA approaches rely on sampling methods, which need to load the whole corpus into memory, making them hardly scalable. In this research, we study an alternative approach for AOS problem, based on Autoencoding Variational Inference (AVI). Firstly, we introduce the Autoencoding Variational Inference for Aspect Discovery (AVIAD) model, which extends the previous work of Autoencoding Variational Inference for Topic Models (AVITM) to embed prior knowledge of seed words. This work includes enhancement of the previous AVI architecture and also modification of the loss function. Ultimately, we present the Autoencoding Variational Inference for Joint Sentiment/Topic (AVIJST) model. In this model, we substantially extend the AVI model to support the JST model, which performs topic modeling for corresponding sentiment. The experimental results show that our proposed models enjoy higher topic coherent, faster convergence time and better accuracy on sentiment classification, as compared to their LDA-based counterparts.

Improving Word Embedding Coverage in Less-Resourced Languages Through Multi-Linguality and Cross-Linguality

In the era of deep learning-based systems, efficient input representation is one of the primary requisites in solving various problems related to Natural Language Processing (NLP), data mining, text mining, and the like. Absence of adequate representation for an input introduces the problem of data sparsity, and it poses a great challenge to solve the underlying problem. The problem is more intensified with resource-poor languages due to the absence of a sufficiently large corpus required to train a word embedding model. In this work, we propose an effective method to improve the word embedding coverage in less-resourced languages by leveraging bilingual word embeddings learned from different corpora. We train and evaluate deep Long Short Term Memory (LSTM)-based architecture and show the effectiveness of the proposed approach for two aspect-level sentiment analysis tasks (i.e., aspect term extraction and sentiment classification). The neural network architecture is further assisted by hand-crafted features for prediction. We apply the proposed model in two experimental setups: multi-lingual and cross-lingual. Experimental results show the effectiveness of the proposed approach against the state-of-the-art methods.

Exploiting multiple word embeddings and one-hot character vectors for aspect-based sentiment analysis

Representing words as real-value vectors and use them as input in deep neural networks is an effective approach in many natural language processing tasks. Currently, some studies use a lower-level representation which is character-based vectors. This paper addresses on how to integrate different representations of input for the problem of aspect-based sentiment analysis. We will propose a joint model of multiple Convolutional Neural Networks (CNNs) in which each individual representation of the input is handled by one CNN. In this work we focus on three kinds of representation including word embeddings from the two methods (Word2Vec and GloVe) and the one-hot character vectors. Our experimental results demonstrate that the proposed model can achieve state-of the-art performance in aspect category detection and aspect sentiment classification tasks.

Joint Aspect Discovery, Sentiment Classification, Aspect-Level Ratings and Weights Approximation for Recommender Systems by Rating Supervised Latent Topic Model

Textual product reviews posted by previous shoppers have been serving as an important source of information that helps on-line shoppers to make their decisions. However, reading through all the reviews of a product is usually a time-demanding and frustrating task, especially when those reviews deliver conflicting information. Therefore, it is of great practical value to develop techniques to automatically generate brief but accurate summaries for the numerous reviews on shopping websites. There are currently two main research streams in review mining: one is joint aspect discovery and sentiment classification, the other one is aspect-level ratings and weights approximation. There exist a number of models in each of the two areas. However, no previous work that aims to solve the two problems simultaneously has been proposed. In this paper we propose Rating Supervised Latent Topic Model to integrate the two problems into an unified optimisation problem. In the proposed model, we employ a latent topic model for aspect discovery and sentiment classification and use a regression model to approximate aspect-level ratings and weights based on the output of the topic model. We test the proposed model on a review dataset crawled from Amazon.com. The preliminary experiment results show that the proposed model outperforms a number of state-of-the-art models by a considerable margin.

Feature selection and ensemble construction: A two-step method for aspect based sentiment analysis

In this paper we present a cascaded framework of feature selection and classifier ensemble using particle swarm optimization (PSO) for aspect based sentiment analysis. Aspect based sentiment analysis is performed in two steps, viz. aspect term extraction and sentiment classification. The pruned, compact set of features performs better compared to the baseline model that makes use of the complete set of features for aspect term extraction and sentiment classification. We further construct an ensemble based on PSO, and put it in cascade after the feature selection module. We use the features that are identified based on the properties of different classifiers and domains. As base learning algorithms we use three classifiers, namely Maximum Entropy (ME), Conditional Random Field (CRF) and Support Vector Machine (SVM). Experiments for aspect term extraction and sentiment analysis on two different kinds of domains show the effectiveness of our proposed approach.

Cascaded Convolutional Neural Networks for Aspect-Based Opinion Summary

This paper studies aspect-based opinion summary (AOS) of reviews on particular products. In practice, an AOS system needs to address two core subtasks, aspect extraction and sentiment classification. Most existing approaches to aspect extraction, using linguistic analysis or topic modeling, are general across different products but not precise enough or suitable for particular products. Instead we take a less general but more precise scheme, which directly maps each review sentence into pre-defined aspects. To tackle aspect mapping and sentiment classification, we propose a convolutional neural network (CNN) based method, cascaded CNN (C-CNN). C-CNN contains two levels of convolutional networks. Multiple CNNs at level 1 deal with aspect mapping task. If a review sentence belongs to pre-defined aspect categories, a single CNN at level 2 determines its sentiment polarity. Experimental results show that C-CNN with pre-trained word embedding outperform cascaded SVM with feature engineering. We also build a system called OpiSum with C-CNN. The demo of OpiSum can be found at http://114.215.167.42.