Opinion Target Extraction Research Articles

A false emotion opinion target extraction model with two stage BERT and background information fusion

As social media has gradually become an indispensable part of people’s life, more and more users begin to express their opinions on social media. These opinions contain rich emotional information, as well as many abnormal false emotion. And the opinion target rich in false emotion will have great potential value in the field of movie box office prediction, public opinion guidance and so on. Therefore, we propose a false emotion opinion target extraction task and two stage BERT+CRF model based on the concept of false emotion. Firstly, two stage BERT network includes C-BERT and W-BERT. We use C-BERT to learn the users comment feature, and W-BERT to learn the background content features of microblog. Then, we design five feature interaction fusion methods to fuse the features learned by C-BERT and W-BERT. At last, we transform the false emotion opinion target extraction task into a sequence labeling task, and use the CRF method to learn the label dependency of the context and output the optimal sequence among all candidate sequences. Due to the lack of false emotion opinion target datasets, we collate a false emotion opinion target dataset Weibo23. Comparing with the advanced models used in related research, including some sequence labeling models and named entity recognition models, two stage BERT+CRF model achieves an F1 score of 0.8834, which is about 10%–20% higher.

O2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^2$$\\end{document}-Bert: Two-Stage Target-Based Sentiment Analysis

Target-based sentiment analysis (TBSA) is one of the most important NLP research topics for widespread applications. However, the task is challenging, especially when the targets contain multiple words or do not exist in the sequences. Conventional approaches cannot accurately extract the (target, sentiment) pairs due to the limitations of the fixed end-to-end architecture design. In this paper, we propose a framework named O2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^2$$\\end{document}-Bert, which consists of Opinion target extraction (OTE-Bert) and Opinion sentiment classification (OSC-Bert) to complete the task in two stages. More specifically, we divide the OTE-Bert into three modules. First, an entity number prediction module predicts the number of entities in a sequence, even in an extreme situation where no entities are contained. Afterwards, with predicted number of entities, an entity starting annotation module is responsible for predicting their starting positions. Finally, an entity length prediction module predicts the lengths of these entities, and thus, accomplishes target extraction. In OSC-Bert, the sentiment polarities of extracted targets from OTE-Bert. According to the characteristics of BERT encoders, our framework can be adapted to short English sequences without domain limitations. For other languages, our approach might work through altering the tokenization. Experimental results on the SemEval 2014-16 benchmarks show that the proposed model achieves competitive performances on both domains (restaurants and laptops) and both tasks (target extraction and sentiment classification), with F1-score as evaluated metrics. Specifically, OTE-Bert achieves 84.63%, 89.20%, 83.16%, and 86.88% F1 scores for target extraction, while OSC-Bert achieves 82.90%, 80.73%, 76.94%, and 83.58% F1 scores for sentiment classification, on the chosen benchmarks. The statistics validate the effectiveness and robustness of our approach and the new “two-stage paradigm”. In future work, we will explore more possibilities of the new paradigm on other NLP tasks.

A Novel Cascade Model for End-to-End Aspect-Based Social Comment Sentiment Analysis

The end-to-end aspect-based social comment sentiment analysis (E2E-ABSA) task aims to discover human’s fine-grained sentimental polarity, which can be refined to determine the attitude in response to an object revealed in a social user’s textual description. The E2E-ABSA problem includes two sub-tasks, i.e., opinion target extraction and target sentiment identification. However, most previous methods always tend to model these two tasks independently, which inevitably hinders the overall practical performance. This paper investigates the critical collaborative signals between these two sub-tasks and thus proposes a novel cascade social comment sentiment analysis model for jointly tackling the E2E-ABSA problem, namely CasNSA. Instead of treating the opinion target extraction and target sentiment identification as discrete procedures in previous works, our new framework takes the contextualized target semantic encoding into consideration to yield better sentimental polarity judgment. Additionally, extensive empirical results show that the proposed approach effectively achieves a 68.13% F1-score on SemEval-2014, 62.34% F1-Score on SemEval-2015, 56.40% F1-Score on SemEval-2016, and 50.05% F1-score on a Twitter dataset, which is higher than the existing approaches. Ablated experiments demonstrate that the CasNSA model substantially outperforms state-of-the-art methods, even when using fixed words embedding rather than pre-trained BERT fine tuning. Moreover, in-depth performance analysis on the social comment datasets further validates that our work gains superior performance and reliability effectively and efficiently in realistic scenarios.

Extended rule-based opinion target extraction with a novel text pre-processing method and ensemble learning

Opinion target extraction (OTE) is the extraction of explicit expressions related to entity aspects interpreted with subjective attributive words in the review sentences using supervised or rule-based approaches. Despite the constraints of syntactic-based relation rules, rule-based approaches can be domain-independently implemented. Although supervised approaches yield better results, more costly due to requiring a large number of labeled samples. This study proposes an unsupervised (rule-based) OTE approach with novel methods and extended rule-based techniques to overcome the aforementioned issues. In this study, first, a novel pattern-based text pre-processing method is proposed to eliminate punctuations that are incompatible with determinative group rules patterns. Then, implemented syntactic-based relation rules on the dependency relation graph are extended with new auxiliary features to extract multi-word expressions which modify each other. The majority voting method is used for optimizing the performance of outputs. Finally, the effectiveness of the proposed approach was tested on a restaurant review dataset. The experimental results show that the proposed approach outperforms all unsupervised approaches. Additionally, it gives comparable results with the supervised approaches, revealing the effectiveness of the proposed approach.

Negative-emotion Opinion Target Extraction Based on Attention and BiLSTM-CRF

Negative-emotion Opinion Target Extraction Based on Attention and BiLSTM-CRF

A Model for Cross-Domain Opinion Target Extraction in Sentiment Analysis

Opinion target extraction is one of the core tasks in sentiment analysis on text data. In recent years, dependency parser–based approaches have been commonly studied for opinion target extraction. However, dependency parsers are limited by language and grammatical constraints. Therefore, in this work, a sequential pattern-based rule mining model, which does not have such constraints, is proposed for cross-domain opinion target extraction from product reviews in unknown domains. Thus, knowing the domain of reviews while extracting opinion targets becomes no longer a requirement. The proposed model also reveals the difference between the concepts of opinion target and aspect, which are commonly confused in the literature. The model consists of two stages. In the first stage, the aspects of reviews are extracted from the target domain using the rules automatically generated from source domains. The aspects are also transferred from the source domains to a target domain. Moreover, aspect pruning is applied to further improve the performance of aspect extraction. In the second stage, the opinion target is extracted among the aspects extracted at the former stage using the rules automatically generated for opinion target extraction. The proposed model was evaluated on several benchmark datasets in different domains and compared against the literature. The experimental results revealed that the opinion targets of the reviews in unknown domains can be extracted with higher accuracy than those of the previous works.

ALSEE: a framework for attribute-level sentiment element extraction towards product reviews

Attribute-level sentiment element extraction aims to obtain the word pair < opinion target, opinion word > from texts, which mainly obtain fine-grained evaluation information in the attribute level. Due to the information fragmentation and semantic sparseness of product reviews, it is difficult to capture more comprehensive local information from unstructured texts, which leads to the incorrect extraction of some word pairs. Aimed at the problem, this paper proposes a framework for Attribute-Level Sentiment Element Extraction (ALSEE) towards product reviews. Firstly, a small amount of sample data is selected by random sampling, and multiple features (including part of speech, word distance, dependency relationship and semantic role) which are labelled. The labelled data are used as the training set. Then, the Condition Random Field (CRF) model is applied to extract opinion targets (OT) and opinion words (OW). The self-training strategy is used to achieve the semi-supervised learning of CRF model through iterative training. Finally, target-opinion word pairs with modifying relationship are obtained by dependency parsing. Compared with the existing methods, the proposed framework can effectively extract attribute-level sentiment elements though experimental results.

Opinion Target Extraction Using a Shallow Semantic Parsing Framework

In this paper, we present a simplified shallow semantic parsing approach to extracting opinion targets. This is done by formulating opinion target extraction (OTE) as a shallow semantic parsing problem with the opinion expression as the predicate and the corresponding targets as its arguments. In principle, our parsing approach to OTE differs from the state-of-the-art sequence labeling one in two aspects. First, we model OTE from parse tree level, where abundant structured syntactic information is available for use, instead of word sequence level, where only lexical information is available. Second, we focus on determining whether a constituent, rather than a word, is an opinion target or not, via a simplified shallow semantic parsing framework. Evaluation on two datasets shows that structured syntactic information plays a critical role in capturing the domination relationship between an opinion expression and its targets. It also shows that our parsing approach much outperforms the state-of-the-art sequence labeling one.

Arabic aspect based sentiment analysis using bidirectional GRU based models

Aspect-based Sentiment analysis (ABSA) accomplishes a fine-grained analysis that defines the aspects of a given document or sentence and the sentiments conveyed regarding each aspect. This level of analysis is the most detailed version that is capable of exploring the nuanced viewpoints of the reviews. The bulk of study in ABSA focuses on English with very little work available in Arabic. Most previous work in Arabic has been based on regular methods of machine learning that mainly depends on a group of rare resources and tools for analyzing and processing Arabic content such as lexicons, but the lack of those resources presents another challenge. In order to address these challenges, Deep Learning (DL)-based methods are proposed using two models based on Gated Recurrent Units (GRU) neural networks for ABSA. The first is a DL model that takes advantage of word and character representations by combining bidirectional GRU, Convolutional Neural Network (CNN), and Conditional Random Field (CRF) making up the (BGRU-CNN-CRF) model to extract the main opinionated aspects (OTE). The second is an interactive attention network based on bidirectional GRU (IAN-BGRU) to identify sentiment polarity toward extracted aspects. We evaluated our models using the benchmarked Arabic hotel reviews dataset. The results indicate that the proposed methods are better than baseline research on both tasks having 39.7% enhancement in F1-score for opinion target extraction (T2) and 7.58% in accuracy for aspect-based sentiment polarity classification (T3). Achieving F1 score of 70.67% for T2, and accuracy of 83.98% for T3.

Research on Opinion Target Extraction and System Information Collection Analysis for Reviews

Though the study of opinion target extraction of movie review, this paper proposed a method for opinion target extraction of movie review based on deep learning, and analyzed its emotional polarity. Finally, the experimental results were displayed in a visual manner to help users judge the details of the movie content. In this paper, by establishing a bidirectional short-term and short-term memory network model, the opinion target extraction task is translated into a sequence labeling task, which improved the accuracy of opinion object extraction. This paper crawled and organized the Chinese movie review corpus from the Internet and the experimental results proved the effectiveness of the proposed method.

Intelligent Aspect based Model for Efficient Sentiment Analysis of User Reviews

Mining online reviews to extract opinion targets and words are substantial tasks in fine-grained opinion mining. The main aim is to mine sensible multi-grain aspects and opinion words from unlabeled reviews. In this paper, Combined Aspect based Sentiment Model (CASM) is propounded to cooperatively mine multi-grain features and opinions. CASM deals with aspects, opinions, sentiment polarity and granularity concurrently. Support Vector Machine-Radial Basis Function kernel (SVM-RBF) classifier is applied to improve CASM by splitting aspects and opinion words. CASM-SVM-RBF deals with two different kinds of aspects and opinions: general and particular aspects, overall opinions and aspect-precise opinions. Candidates with improved confidence are mined as opinion targets or words. Index Terms:Aspects, Opinions, Sentiment, Support Vector Machine, Radial Basis Function, Support Vector Machine (SVM), Radial Basis Function (RBF)

A novel evolutionary approach for learning syntactic features for cross domain opinion target extraction

Aspect-Based Sentiment Analysis (ABSA) is the problem of mining textual user reviews to gauge the orientation of a user towards the various characteristics of a product. Identifying words in the review towards which the user holds a sentiment, known as Aspect Target Extraction (ATE) or Opinion Target Extraction (OTE), is a crucial step in ABSA. Several start-of-the-art techniques, mostly employing Deep Learning methods, have been proposed to tackle this problem. However, when attempting ATE in a new domain, most of the supervised methods are crippled by a lack of labeled data within the domain and do not perform well with models built on other unrelated domains. Other unsupervised methods based on linguistic rules framed by experts have been proposed to mitigate the problem of the lack of data but this requires manual engineering of rules, which may not capture all aspect terms. Different from the previous approaches, we propose evolutionary methods to generate pattern graphs, which encode rules to identify cross-domain aspect terms. The proposed methods aim to mine linguistic patterns, which are quality indicators of the presence of aspect words. The automated pattern learning technique using the evolutionary approach enables automated learning of rules. This approach eliminates the need for human experts to coin the same, while at the same time enabling the production of complicated rules, which can identify cross-domain aspects effectively. Two slightly different approaches are proposed — the first based on identifying opinion targets by matching the learned pattern graphs against existing sentences and the second based on using the learned pattern graphs to construct features for building classification models for aspect identification. We conduct extensive experiments using 10 real-life datasets of varying sizes and different characteristics, and demonstrate the superiority of the proposed methods for cross-domain ATE. We find that the proposed approaches perform better as compared to the existing methods in 69 of the 90 cases, or in other words, 76% of the cases.

Extraction of Opinion Target Using Syntactic Rules in Urdu Text

Opinion target or aspect extraction is the key task of aspect-based sentiment analysis. This task focuses on the extraction of targeted words or phrases against which a user has expressed his/her opinion. Although, opinion target extraction has been studied extensively in the English language domain, with notable work in other languages such as Chinese, Arabic etc., other regional languages have been neglected. One of the reasons is the lack of resources and available texts for these languages. Urdu is one, with millions of native and non-native speakers across the globe. In this paper, the Urdu language domain is focused on to identify opinion targets from written Urdu texts. To accomplish this task, several syntactic rules are crafted to identify users’ opinions and associated target words. These rules are crafted using the grammatical and linguistic context of the words in the sentence. To the best of our knowledge, there is no existing work available in the Urdu domain for opinion target extraction. The proposed methodology is evaluated on an Urdu language dataset and compared with an existing approach for the English language by applying the same technique. The experiments have demonstrated that the proposed approach achieves promising performance as compared to the applied English language domain approach.

MF-COTE: A chinese opinion target extraction model based on multiple features

Recently, deep learning methods have been applied to deal with the opinion target extraction (OTE) task with fruitful achievements. On the other hand, since the features captured by the embedding layer can make a multiple-perspective analysis from a sentence, an embedding layer that can grasp the high-level semantics of the sentences is of essence for processing the OTE task and can improve the performance of model into a more efficient manner. However, most of the existing studies focused on the network structure rather than the significant embedded layer, which may be the fundamental reason for the problem of relatively poor performance in this field, not mention the Chinese extraction model. To compensate these shortcomings, this paper proposes a model using multiple effective features and Bidirectional Encoder Representations from Transformers (BERT) on the architecture of Bidirectional Long Short-Term Memory (BiLSTM) and Conditional Random Field (CRF) for Chinese opinion target extraction task, namely MF-COTE, which can construct features from different perspectives to capture the context and local features of the sentences. Besides, to handle the difficult case of multiple nouns in one sentence, we innovatively propose noting words feature to regulate the model emphasize on the noun near the transition or contrast word, thus leading a better opinion target location. Moreover, to demonstrate the superiorities of the proposed model, extensive comparison experiments are systematically conducted compared with other existing state-of-the-art methods, with the F1-score of 90.76%, 92.10%, 89.63% on the Baidu, the Dianping, and the Mafengwo dataset, respectively.

Extracting Opinion Targets Using Attention-Based Neural Model

Extracting opinion-target expression is a core subtask to perform aspect-based sentiment analysis which aims to identify the discussed aspects within a text associated with their opinion targets and classify the sentiment as positive, negative, or neutral. This paper proposes a deep learning model to tackle the opinion-target expression extraction task. The proposed model is composed of bidirectional long short-term memory as an encoder, long short-term memory as a decoder with an attention mechanism, and conditional random fields. This model, which operates at the sentence level, is designed to extract opinion targets for the Arabic language. The proposed model’s performance is evaluated using SemEval-2016 annotated dataset for the hotels’ domain. Experimental results show that the proposed model outperforms the baseline and the prior works, where it achieved an F1 measure of 72.83%.

An Emotion Expression Extraction Method for Chinese Microblog Sentences

With the rapid spread of Chinese microblog, a large number of microblog topics are being generated in real-time. More and more users pay attention to emotion expressions of these opinionated sentences in different topics. It is challenging to label the emotion expressions of opinionated sentences manually. For this endeavor, an emotion expression extraction method is proposed to process millions of user-generated opinionated sentences automatically in this paper. Specifically, the proposed method mainly contains two tasks: emotion classification and opinion target extraction. We first use a lexicon-based emotion classification method to compute different emotion values in emotion label vectors of opinionated sentences. Then emotion label vectors of opinionated sentences are revised by an unsupervised emotion label propagation algorithm. After extracting candidate opinion targets of opinionated sentences, the opinion target extraction task is performed on a random walk-based ranking algorithm, which considers the connection between candidate opinion targets and the textual similarity between opinionated sentences, ranks candidate opinion targets of opinionated sentences. Experimental results demonstrate the effectiveness of algorithms in the proposed method.

CHOpinionMiner: An unsupervised system for Chinese opinion target extraction

SummaryOpinion target extraction (OTE) is an important task in fine‐grained sentiment analysis field, which focuses on the identification of the targets of users' opinions or sentiments from online reviews. Existing approaches to OTE are mainly based on unsupervised rule‐based methods or supervised machine learning methods. However, the latter needs a large number of labeled samples to train their models and lack of labeled corpus limits the research progress on OTE of Chinese. In this paper, we proposed a novel unsupervised and domain independent system called CHOpinionMiner. First, noun phrases are extracted as candidate opinion targets based on phrase structure grammar, then the syntactic paths of the candidate opinion targets and the opinion words in one opinionated sentence are extracted. After that, the legal paths, or rather, the opinion targets‐opinion lexicon pairs that are syntactically associated are selected according to the syntactic rules. Finally, not only the formal targets but also the orientations are obtained. Experiments on data sets consisting of microblog topics and product reviews demonstrate that our approach outperforms the existing state‐of‐the‐art methods.

A Unified Model for Opinion Target Extraction and Target Sentiment Prediction

Target-based sentiment analysis involves opinion target extraction and target sentiment classification. However, most of the existing works usually studied one of these two sub-tasks alone, which hinders their practical use. This paper aims to solve the complete task of target-based sentiment analysis in an end-to-end fashion, and presents a novel unified model which applies a unified tagging scheme. Our framework involves two stacked recurrent neural networks: The upper one predicts the unified tags to produce the final output results of the primary target-based sentiment analysis; The lower one performs an auxiliary target boundary prediction aiming at guiding the upper network to improve the performance of the primary task. To explore the inter-task dependency, we propose to explicitly model the constrained transitions from target boundaries to target sentiment polarities. We also propose to maintain the sentiment consistency within an opinion target via a gate mechanism which models the relation between the features for the current word and the previous word. We conduct extensive experiments on three benchmark datasets and our framework achieves consistently superior results.

Language independent sequence labelling for Opinion Target Extraction

In this research note we present a language independent system to model Opinion Target Extraction (OTE) as a sequence labelling task. The system consists of a combination of clustering features implemented on top of a simple set of shallow local features. Experiments on the well known Aspect Based Sentiment Analysis (ABSA) benchmarks show that our approach is very competitive across languages, obtaining best results for six languages in seven different datasets. Furthermore, the results provide further insights into the behaviour of clustering features for sequence labelling tasks. The system and models generated in this work are available for public use and to facilitate reproducibility of results.

Sentiment Analysis of Arabic Tweets: Opinion Target Extraction

Sentiment Analysis of Arabic Tweets: Opinion Target Extraction