Aspect Term Extraction Research Articles

Understanding Fine-Grained Sentiments of Super-Priority Destination Visitors using Multi-task Learning for Extraction of Aspect Terms and Polarity Classification on Reviews

One sector that plays an important role in the national economy is the Tourism sector. To maintain economic stability, the Indonesian Government has made various efforts to mitigate the impact of the Covid-19 pandemic on the tourism sector. The National Economic Recovery (PEN) is one of its programs, specifically focused on the recovery of tourism, particularly in the 5 super-priority destinations (Lake Toba, Borobudur, Mandalika, Labuan Bajo, Likupang). To ensure the effective recovery of these 5 super-priority destinations, in addition to information from stakeholders, the government can conduct studies based on the aspirations of the people. One task that can assist the government in summarizing user reviews is sentiment analysis. Previous studies have focused on comments from the local Indonesian population, which have not reached the majority of tourists who comment in English. Therefore, there is a need for Sentiment Analysis (SA) at the aspect level on TripAdvisor reviews of the Super Priority Destinations (SPDs) using the English language. The method used for sentiment analysis of these tourist reviews is the Local Context Focus - Aspect Term Extraction & Aspect Polarization Classification (LCF-ATEPC). This method employs multi-task learning, domain Bidirectional Encoder Representations from Transformers (BERT), and information about the aspect position within a sentence. Based on the research, the LCF-ATEPC model has proven to be capable of performing sentiment analysis of English aspects effectively. The best-performing LCF-ATEPC model was obtained using a dataset without outliers (95%) with APC-ACC, APC-F1, and ATE-F1 accuracies of 83.33%, 69.5%, and 66.83%, respectively.

Context-Aware Dynamic Word Embeddings for Aspect Term Extraction

The aspect term extraction (ATE) task aims to extract aspect terms describing a part or an attribute of a product from review sentences. Most existing works rely on either general or domain embedding to address this problem. Despite the promising results, the importance of general and domain embeddings is still ignored by most methods, resulting in degraded performances. Besides, word embedding is also related to downstream tasks, and how to regularize word embeddings to capture context-aware information is an unresolved problem. To solve these issues, we first propose context-aware dynamic word embedding (CDWE), which could simultaneously consider general meanings, domain-specific meanings, and the context information of words. Based on CDWE, we propose an attention-based convolution neural network, called ADWE-CNN for ATE, which could adaptively capture the previous meanings of words by utilizing an attention mechanism to assign different importance to the respective embeddings. The experimental results show that ADWE-CNN achieves a comparable performance with the state-of-the-art approaches. Various ablation studies have been conducted to explore the benefit of each component. Our code is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">http://github.com/xiejiajia2018/ADWE-CNN</uri> .

Part-of-speech based label update network for aspect sentiment triplet extraction

Aspect sentiment triplet analysis (ASTE) is a nuanced task that entails the extraction of all triplets from a user comment, where each triplet consist of an aspect term, an opinion term, and the associated sentiment polarity of the aspect term. Recent research has brought forth a boundary-driven table-filling approach that adeptly tackles the persistent issues of inconsistent relationships and boundary insensitivity found in prior methods, but the improvement in performance is somewhat limited because this method overlook the wealth of information encapsulated within each word present in the sentence. To overcome these limitations, this study proposes a novel Part-of-speech Based Label Update Network (PBLUN) for aspect sentiment triplet extraction. Specifically, a POS-based label update module integrated with aspect term extraction (ATE) and opinion term extraction (OTE) tasks is devised to discern the existence of aspect or opinion words within the set adjacent search domain and update their labels. In addition, the proposed model leverages biaffine attention network to extract probability distribution denoting various relationships between words and effectively combine them with relation-level representation. Experiments conducted on four benchmark datasets have conclusively demonstrated the advantages of our proposed method when compared to strong baseline models.

IIM: an information interaction mechanism for aspect-based sentiment analysis

ABSTRACT Term polarity co-extraction is an aspect-based sentiment analysis task, which has been widely used in the fields of user opinions extraction. It consists of two subtasks: aspect term extraction and aspect sentiment classification. Most existing studies solve aforesaid subtasks as independent tasks or simply unify the two subtasks without making full use of the relationship between tasks to mine the interaction of text information, which leads to low performance for practical applications. Meanwhile, the learning framework for these studies has a label drift phenomenon (LDP) in the process of predictive learning, increasing the learning error rate. To address the above problems, this study unifies subtasks and proposes a Unified framework based on the information interaction mechanism framework, called IIM. Specifically, we design an Information Interaction Channel (IIC) to construct closer semantic features to extract preliminary term-polarity unified labels from the perspective of basic semantics. For label inconsistency between aspect terms, a Position-aware Module (SAM) is proposed to alleviate the Label Drift Phenomenon (LDP). Moreover, we introduce a syntax-attention graph neural network (Syn-AttGCN) to model the syntactic structure of text and strengthen the emotional connection between aspect terms. The experimental results show that IIM outperforms most baselines. Meanwhile, the SAM module has a certain slowing effect on LDP.

Soft prompt enhanced joint learning for cross-domain aspect-based sentiment analysis

Aspect term extraction is a fundamental task in fine-grained sentiment analysis, aiming to detect customer's opinion targets from reviews about products or services. The traditional supervised models have achieved promising results with annotated datasets. However, their performance dramatically decreases in cross-domain aspect term extraction tasks. Existing cross-domain transfer learning methods face two common limitations: (1) these works directly inject linguistic features into language models, making it challenging to transfer linguistic knowledge to the target domain; (2) they rely on the fixed predefined prompts, which is time-consuming to construct the prompts for all potential aspect term spans. To address the limitations, we propose a soft prompt-based joint learning method for cross-domain aspect term extraction in this paper. Specifically, by incorporating external linguistic features, the proposed method learns domain-invariant representations between source and target domains via multiple objectives, which bridges the gap between domains with varied distributions of aspect terms. Furthermore, the proposed method interpolates a set of transferable soft prompts consisting of multiple learnable vectors that are beneficial to detect aspect terms in the target domain. Extensive experiments are conducted on two groups of datasets and the experimental results demonstrate the effectiveness of the proposed method for cross-domain aspect terms extraction.

Aspect term extraction via adaptive fusion of sequential and hierarchical representation

In aspect-based sentiment analysis, a fundamental task is extracting aspect terms from opinionated sentences. Aspect term extraction (ATE) has been found to play a critical role among several scenarios, such as service quality improvement and recommendation systems. While deep learning-based methods have achieved great progress in ATE, they mainly consider sequential semantic information and generally ignore the utilisation of syntactic relations of the whole sentence on overall meanings. Furthermore, performances of these methods may also be diminished by poor handling of relation and text noises. To address these issues, we propose a fused sequential and hierarchical representation (FSHR) model, wherein both sequential and hierarchical representations are generated, which facilitates not only the capture of linear semantic information for predicting meaning-related aspect terms but also the utilisation of syntactic relations over the entire sentence to better identify structure-related aspect terms. Moreover, to refine the aspect representation, we incorporate relation-gate mechanism which selectively activates meaningful syntactic dependency paths and design the multi-way aspect attention which prompts the model to focus on relevant text segments about particular aspects. Eventually, sequential and hierarchical representations are adaptively fused for aspect prediction. Experiment results on four datasets demonstrate that FSHR outperforms competitive baselines, and further extensive analyses reveal the effectiveness of our model.

Syntax-enhanced aspect-based sentiment analysis with multi-layer attention

As a key task of fine-grained sentiment analysis, aspect-based sentiment analysis aims to analyse people’s opinions at the aspect level from user-generated texts. Various sub-tasks have been defined according to different scenarios, extracting aspect terms, opinion terms, and the corresponding sentiment. However, most existing studies merely focus on a specific sub-task or a subset of sub-tasks, having many complicated models designed and developed. This hinders the practical applications of aspect-based sentiment analysis. Therefore, some unified frameworks are proposed to handle all the subtasks, but most of them suffer from two limitations. First, the syntactic features are neglected, but such features have been proven effective for aspect-based sentiment analysis. Second, very few efficient mechanisms are developed to leverage important syntactic features, e.g., dependency relations, dependency relation types, and part-of-speech tags. To address these challenges, in this paper, we propose a novel unified framework to handle all defined sub-tasks for aspect-based sentiment analysis. Specifically, based on the graph convolutional network, a multi-layer semantic model is designed to capture the semantic relations between aspect and opinion terms. Moreover, a multi-layer syntax model is proposed to learn explicit dependency relations from different layers. To facilitate the sub-tasks, the learned semantic features are propagated to the syntax model with better semantic guidance to learn the syntactic representations comprehensively. Different from the conventional syntactic model, the proposed framework introduces two attention mechanisms. One is to model dependency relation and type, and the other is to encode part-of-speech tags for detecting aspect and opinion term boundaries. Extensive experiments are conducted to evaluate the proposed novel unified framework, and the experimental results on four groups of real-world datasets explicitly demonstrate the superiority of the proposed framework over a range of baselines.

An Interactive Learning Network That Maintains Sentiment Consistency in End-to-End Aspect-Based Sentiment Analysis

Most of the aspect-based sentiment analysis research completes the two subtasks (aspect terms extraction and aspect sentiment classification) separately, and it cannot see the full picture and actual effect of the complete aspect-based sentiment analysis. The purpose of end-to-end aspect-based sentiment analysis is to complete the two subtasks of aspect terms extraction and aspect sentiment classification at the same time, and the current research in this area focuses on the connection between the two subtasks and uses the connection between them to construct the model. However, they rarely pay attention to the connection between different aspects and ignore the sentiment inconsistency within the aspects caused by the end-to-end model. Therefore, we propose an interactive learning network to maintain sentiment consistency, first using the multi-head attention mechanism to achieve the interaction between aspects and subtasks and then using the gate mechanism to design an auxiliary module to maintain sentiment consistency within aspect items. The experimental results on the datasets Laptop14, Restaurant14, and Twitter showed that, compared with the optimal benchmark method, the F1 values of the proposed method increased by 0.4%, 1.21%, and 5.22%, respectively. This indicates that the proposed method can effectively consider the relationships between aspect items and maintain emotional consistency within the aspect items.

An efficient LSTM based cross domain aspect based sentiment analysis (CD-ABSA)

This research study focuses the cross-domain aspect-based sentiment analysis (CD-ABSA) for existing source domain annotation data. The CD-ABSA tries to use the valuable information in a source domain to extract aspect terms and evaluate their sentiment polarities in a target domain. It can considerably increase the usage of the source domain annotation resources while also reducing the workload of newer domain data annotation. one of the main components of the CD-ABSA is aspect extraction. In this paper, we utilized the most common topic modelling techniques: LDA and LSA to extract aspects from the reviews as it does not require labelled data. The topics are extracted from the education domain of the Course and Teacher Performance Evaluation (CTPE) dataset. In this paper, we also evaluated the different hyper-parameters on the CD-ABSA model and selected the best and optimal combination. The proposed methodology train on domain-dependent and independent word embedding that achieves CD-ABSA, in particularly end-to-end fashion. The experiment carries out on Academica dataset, which consists of students’ comments/feedback and SemEval-2014 dataset, which includes laptops and restaurants reviews. The evaluation metrics such as (precision, recall, F1 score and validation Accuracy) is considered while judging the LSTM classifier performance for CD-ABSA as a result.

Aspect-Based Sentiment Analysis

Aspect-Based Sentiment Analysis (ABSA) is an advanced NLP application that aims to identify aspect terms present in the given review and predict the sentiment associated with those aspect terms. ABSA is better than sentence-based sentiment classification because it considers the aspect terms present in the reviews to determine the sentiment rather than considering the individual sentence. Entrepreneurs could make use of ABSA to understand the customers' opinions about different aspects of their products or services. The task of Aspect-based sentiment analysis can be divided into two subtasks: Aspect Term Extraction (ATE) and Aspect Term Sentiment Classification (ATSC). In this paper, an SVM model is proposed for the task of ATE, and an Attention based LSTM model is proposed for the task of ATSC. The proposed models will be trained and tested on SemEval-2014 dataset.

Block-level dependency syntax based model for end-to-end aspect-based sentiment analysis

End-to-End aspect-based sentiment analysis (E2E-ABSA) aims to jointly extract aspect terms and identify their sentiment polarities. Although previous research has demonstrated that syntax knowledge can be beneficial for E2E-ABSA, standard syntax dependency parsing struggles to capture the block-level relation between aspect and opinion terms, which hinders the role of syntax in E2E-ABSA. To address this issue, this paper proposes a block-level dependency syntax parsing (BDEP) based model to enhance the performance of E2E-ABSA. BDEP is constructed by incorporating routine dependency syntax parsing and part-of-speech tagging, which enables the capture of block-level relations. Subsequently. the BDEP-guided interactive attention module (BDEP-IAM) is used to obtain the aspect-aware representation of each word. Finally the adaptive fusion module is leveraged to combine the semantic-syntactic representation to simultaneously extract the aspect term and identify aspect-orient sentiment polarity. The model is evaluated on five benchmark datasets, including Laptop14, Rest _ALL, Restaurant14, Restaurant15, and TWITTER, with F1 scores of 62.67%, 76.53%, 75.42%, 62.21%, and 58.03%, respectively. The results show that our model outperforms the other compared state-of-the-art (SOTA) methods on all datasets. Additionally, ablation experiments confirm the efficacy of BDEP and IAM in improving aspect-level sentiment analysis.

Combining BERT with TCN-BiGRU for enhancing Arabic aspect category detection

Aspect-based sentiment analysis (ABSA) is a challenging task of sentiment analysis that aims at extracting the discussed aspects and identifying the sentiment corresponding to each aspect. We can distinguish three main ABSA tasks: aspect term extraction, aspect category detection (ACD), and aspect sentiment classification. Most Arabic ABSA research has relied on rule-based or machine learning-based methods, with little attention to deep learning techniques. Moreover, most existing Arabic deep learning models are initialized using context-free word embedding models, which cannot handle polysemy. Therefore, this paper aims at overcoming the limitations mentioned above by exploiting the contextualized embeddings from pre-trained language models, specifically the BERT model. Besides, we combine BERT with a temporal convolutional network and a bidirectional gated recurrent unit network in order to enhance the extracted semantic and contextual features. The evaluation results show that the proposed method has outperformed the baseline and other models by achieving an F1-score of 84.58% for the Arabic ACD task. Furthermore, a set of methods are examined to handle the class imbalance in the used dataset. Data augmentation based on back-translation has shown its effectiveness through enhancing the first results by an overall improvement of more than 3% in terms of F1-score.

Aspect-based sentiment analysis via multitask learning for online reviews

Aspect based sentiment analysis(ABSA) aims to identify aspect terms in online reviews and predict their corresponding sentiment polarity. Sentiment analysis poses a challenging fine-grained task. Two typical subtasks are involved: Aspect Term Extraction (ATE) and Aspect Polarity Classification (APC). These two subtasks are usually trained discretely, which ignores the connection between ATE and APC. Concretely, we can relate ATE to APC through aspects and train them concurrently. We mainly use the ATE task as an auxiliary task, allowing the APC to focus more on relevant aspects to facilitate aspect polarity classification. In addition, previous studies have shown that utilizing dependency syntax information with a graph neural network (GNN) also contributes to the performance of the APC task. However, most studies directly input sentence dependency relations into graph neural networks without considering the influence of aspects, which do not emphasize the important dependency relationships. To address these issues, we propose a multitask learning model combining APC and ATE tasks that can extract aspect terms as well as classify aspect polarity simultaneously. Moreover, we exploit multihead attention(MHA) to associate dependency sequences with aspect extraction, which not only combines both ATE and APC tasks but also stresses the significant dependency relations, enabling the model to focus more on words closely related to aspects. According to our experiments on three benchmark datasets, we demonstrate that the connection between ATE and APC can be better established by our model, which enhances aspect polarity classification performance significantly. The source code has been released on GitHub https://github.com/winder-source/MTABSA.

BERT-Based Joint Model for Aspect Term Extraction and Aspect Polarity Detection in Arabic Text

Aspect-based sentiment analysis (ABSA) is a method used to identify the aspects discussed in a given text and determine the sentiment expressed towards each aspect. This can help provide a more fine-grained understanding of the opinions expressed in the text. The majority of Arabic ABSA techniques in use today significantly rely on repeated pre-processing and feature-engineering operations, as well as the use of outside resources (e.g., lexicons). In essence, there is a significant research gap in NLP with regard to the use of transfer learning (TL) techniques and language models for aspect term extraction (ATE) and aspect polarity detection (APD) in Arabic text. While TL has proven to be an effective approach for a variety of NLP tasks in other languages, its use in the context of Arabic has been relatively under-explored. This paper aims to address this gap by presenting a TL-based approach for ATE and APD in Arabic, leveraging the knowledge and capabilities of previously trained language models. The Arabic base (Arabic version) of the BERT model serves as the foundation for the suggested models. Different BERT implementations are also contrasted. A reference ABSA dataset was used for the experiments (HAAD dataset). The experimental results demonstrate that our models surpass the baseline model and previously proposed approaches.

Adaptive Local Context and Syntactic Feature Modeling for Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis is a fine-grained sentiment analysis task that consists of two types of subtasks: aspect term extraction and aspect sentiment classification. In the aspect term extraction task, current methods suffer from the lack of fine-grained information in aspect term extraction and difficulty in identifying aspect term boundaries. In the aspect sentiment classification task, the current aspect sentiment classifier cannot adapt itself to the text and determine the local context. To address these two challenges, this work proposes an adaptive semantic relative distance approach based on dependent syntactic analysis, which uses adaptive semantic relative distance to determine the appropriate local context for each text and increase the accuracy of sentiment analysis. Meanwhile, the study also predicts the current word labels by combining local information features extracted by local convolutional neural networks and global information features to precisely locate the word labels. In two subtasks, our proposed model improves accuracy and F1 scores on the SemEval-2014 Task 4 Restaurant and Laptop datasets compared to the state-to-the-art approaches, especially in the aspect sentiment classification subtask.

Multi-Task Learning Model with Data Augmentation for Arabic Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) is a fine-grained process. Its fundamental subtasks are aspect term extraction (ATE) and aspect polarity classification (APC), and these subtasks are dependent and closely related. However, most existing works on Arabic ABSA content separately address them, assume that aspect terms are preidentified, or use a pipeline model. Pipeline solutions design different models for each task, and the output from the ATE model is used as the input to the APC model, which may result in error propagation among different steps because APC is affected by ATE error. These methods are impractical for real-world scenarios where the ATE task is the base task for APC, and its result impacts the accuracy of APC. Thus, in this study, we focused on a multi-task learning model for Arabic ATE and APC in which the model is jointly trained on two subtasks simultaneously in a single model. This paper integrates the multi-task model, namely Local Cotext Foucse-Aspect Term Extraction and Polarity classification (LCF-ATEPC) and Arabic Bidirectional Encoder Representation from Transformers (AraBERT) as a shred layer for Arabic contextual text representation. The LCF-ATEPC model is based on a multi-head self-attention and local context focus mechanism (LCF) to capture the interactive information between an aspect and its context. Moreover, data augmentation techniques are proposed based on state-of-the-art augmentation techniques (word embedding substitution with constraints and contextual embedding (AraBERT)) to increase the diversity of the training dataset. This paper examined the effect of data augmentation on the multi-task model for Arabic ABSA. Extensive experiments were conducted on the original and combined datasets (merging the original and augmented datasets). Experimental results demonstrate that the proposed Multi-task model outperformed existing APC techniques. Superior results were obtained by AraBERT and LCF-ATEPC with fusion layer (AR-LCF-ATEPC-Fusion) and the proposed data augmentation word embedding-based method (FastText) on the combined dataset.

A Semi-Supervised Approach for Aspect Category Detection and Aspect Term Extraction from Opinionated Text

A Semi-Supervised Approach for Aspect Category Detection and Aspect Term Extraction from Opinionated Text

A Deep-Learning Framework for Analysing Students' Review in Higher Education.

As part of continuous process improvements to teaching and learning, the management of tertiary institutions requests students to review modules towards the end of each semester. These reviews capture students' perceptions about various aspects of their learning experience. Considering the large volume of textual feedback, it is not feasible to manually analyze all the comments, hence the need for automated approaches. This study presents a framework for analyzing students' qualitative reviews. The framework consists of four distinct components: aspect-term extraction, aspect-category identification, sentiment polarity determination, and grades' prediction. We evaluated the framework with the dataset from the Lilongwe University of Agriculture and Natural Resources (LUANAR). A sample size of 1,111 reviews was used. A microaverage F1-score of 0.67 was achieved using Bi- LSTM-CRF and BIO tagging scheme for aspect-term extraction. Twelve aspect categories were then defined for the education domain and four variants of RNNs models (GRU, LSTM, Bi-LSTM, and Bi-GRU) were compared. A Bi-GRU model was developed for sentiment polarity determination and the model achieved a weighted F1-score of 0.96 for sentiment analysis. Finally, a Bi-LSTM-ANN model which combined textual and numerical features was implemented to predict students' grades based on the reviews. A weighted F1-score of 0.59 was obtained, and out of 29 students with "F" grade, 20 were correctly identified by the model.

Self-adaptive attention fusion for multimodal aspect-based sentiment analysis.

Multimodal aspect term extraction (MATE) and multimodal aspect-oriented sentiment classification (MASC) are two crucial subtasks in multimodal sentiment analysis. The use of pretrained generative models has attracted increasing attention in aspect-based sentiment analysis (ABSA). However, the inherent semantic gap between textual and visual modalities poses a challenge in transferring text-based generative pretraining models to image-text multimodal sentiment analysis tasks. To tackle this issue, this paper proposes a self-adaptive cross-modal attention fusion architecture for joint multimodal aspect-based sentiment analysis (JMABSA), which is a generative model based on an image-text selective fusion mechanism that aims to bridge the semantic gap between text and image representations and adaptively transfer a textual-based pretraining model to the multimodal JMASA task. We conducted extensive experiments on two benchmark datasets, and the experimental results show that our model significantly outperforms other state of the art approaches by a significant margin.

Reinforcement Learning-based Aspect Term Extraction using Dilated Convolutions and Differential Equation Initialization

Reinforcement Learning-based Aspect Term Extraction using Dilated Convolutions and Differential Equation Initialization