Fine-grained Opinion Research Articles

DNet: A lightweight and efficient model for aspect based sentiment analysis

Aspect based sentiment analysis (ABSA) is the task of identifying fine-grained opinion polarity towards a specific target in a sentence, which is empowering experts and intelligent systems with enriched interaction capabilities. Most of approaches to date usually capture semantic relations between target and context words based on RNNs (Recurrent Neural Networks) or pre-trained models (e.g. BERT). However, due to computational complexity and size constraints, these models are often hosted in the cloud. Enabling ABSA models to run on resource-constrained end-devices with quick response time is still challenging and not yet well studied. This paper presents distillation network (DNet), a lightweight and efficient sentiment analysis model based on gated convolutional neural networks for on-device inference. Through combining stacked gated convolution with attention mechanism, DNet can distill aspect-aware context information from unstructured text progressively, achieving high performance with less inference latency and reduced model size. Experiments on SemEval 2014 Task 4 and ACL14 Twitter datasets demonstrate that our approach achieves the state-of-the-art performance. Furthermore, compared with the BERT-based model, DNet reduces the model size by more than 50 times and improves the responsiveness by 24 times.

An association-constrained LDA model for joint extraction of product aspects and opinions

The Latent Dirichlet Allocation (LDA) model, which is a document-level probabilistic model, has been widely used in topic modeling. However, an essential issue of the LDA is its shortage in identifying co-occurrence relationships (e.g., aspect-aspect, aspect-opinion, etc.) in sentences. To address the problem, we propose an association constrained LDA (AC-LDA) for effectively capturing the co-occurrence relationships. Specifically, based on the basic features of the syntactic structure in product reviews, we formalize three major types of word association combinations and then carefully design corresponding identifications. For reducing the influence of global aspect words on the local distribution, we apply an important constraint on global aspects. Finally, the constraint and related association combinations are merged into the LDA to guide the topic-words allocation in the learning process. Based on the experiments on real-world product review data, we demonstrate that our model can effectively capture the relationships hidden in local sentences and further increase the extraction rate of fine-grained aspects and opinion words. Our results confirm the superiority of the AC-LDA over the state-of-the-art methods in terms of the extraction accuracy. We also verify the strength of our method in identifying irregularly appeared terms, such as non-aspect opinions, low-frequency words, and secondary aspects.

Deep Learning Model for Fine-Grained Aspect-Based Opinion Mining

Despite the great manufactures’ efforts to achieve customer satisfaction and improve their performance, social media opinion mining is still on the fly a big challenge. Current opinion mining requires sophisticated feature engineering and syntactic word embedding without considering semantic interaction between aspect term and opinionated features, which degrade the performance of most of opinion mining tasks, especially those that are designed for smart manufacturing. Research on intelligent aspect level opinion mining (AOM) follows the fast proliferation of user-generated data through social media for industrial manufacturing purposes. Google’s pre-trained language model, Bidirectional Encoder Representations from Transformers (BERT) widely overcomes existing methods in eleven natural language processing (NLP) tasks, which makes it the standard way for semantic text representation. In this paper, we introduce a novel deep learning model for fine-grained aspect-based opinion mining, named as FGAOM. First, we train the BERT model on three specific domain corpora for domain adaption, then use adjusted BERT as embedding layer for concurrent extraction of local and global context features. Then, we propose Multi-head Self-Attention (MSHA) to effectively fuse internal semantic text representation and take advantage of convolutional layers to model aspect term interaction with surrounding sentiment features. Finally, the performance of the proposed model is evaluated via extensive experiments on three public datasets. Results show that performance of the proposed model outperforms performances of recent the-of-the-art models.

Syntactically Meaningful and Transferable Recursive Neural Networks for Aspect and Opinion Extraction

In fine-grained opinion mining, extracting aspect terms (a.k.a. opinion targets) and opinion terms (a.k.a. opinion expressions) from user-generated texts is the most fundamental task in order to generate structured opinion summarization. Existing studies have shown that the syntactic relations between aspect and opinion words play an important role for aspect and opinion terms extraction. However, most of the works either relied on predefined rules or separated relation mining with feature learning. Moreover, these works only focused on single-domain extraction, which failed to adapt well to other domains of interest where only unlabeled data are available. In real-world scenarios, annotated resources are extremely scarce for many domains, motivating knowledge transfer strategies from labeled source domain(s) to any unlabeled target domain. We observe that syntactic relations among target words to be extracted are not only crucial for single-domain extraction, but also serve as invariant “pivot” information to bridge the gap between different domains. In this article, we explore the constructions of recursive neural networks based on the dependency tree of each sentence for associating syntactic structure with feature learning. Furthermore, we construct transferable recursive neural networks to automatically learn the domain-invariant fine-grained interactions among aspect words and opinion words. The transferability is built on an auxiliary task and a conditional domain adversarial network to reduce domain distribution difference in the hidden spaces effectively in word level through syntactic relations. Specifically, the auxiliary task builds structural correspondences across domains by predicting the dependency relation for each path of the dependency tree in the recursive neural network. The conditional domain adversarial network helps to learn domain-invariant hidden representation for each word conditioned on the syntactic structure. In the end, we integrate the recursive neural network with a sequence labeling classifier on top that models contextual influence in the final predictions. Extensive experiments and analysis are conducted to demonstrate the effectiveness of the proposed model and each component on three benchmark data sets.

Transferable Interactive Memory Network for Domain Adaptation in Fine-Grained Opinion Extraction

In fine-grained opinion mining, aspect and opinion terms extraction has become a fundamental task that provides key information for user-generated texts. Despite its importance, a lack of annotated resources in many domains impede the ability to train a precise model. Very few attempts have applied unsupervised domain adaptation methods to transfer fine-grained knowledge (in the word level) from some labeled source domain(s) to any unlabeled target domain. Existing methods depend on the construction of “pivot” knowledge, e.g., common opinion terms or syntactic relations between aspect and opinion words. In this work, we propose an interactive memory network that consists of local and global memory units. The model could exploit both local and global memory interactions to capture intra-correlations among aspect words or opinion words themselves, as well as the interconnections between aspect and opinion words. The source space and the target space are aligned through these domaininvariant interactions by incorporating an auxiliary task and domain adversarial networks. The proposed model does not require any external resources and demonstrates promising results on 3 benchmark datasets.

Research on Intelligence Computing Models of Fine-Grained Opinion Mining in Online Reviews

In order to obtain evaluation information about the various aspects of products or services, the Fine-grained Topic Sentiment Unification (FG-TSU) model is proposed based on the improvement of LDA (Latent Dirichlet Allocation) model. Firstly, the topics are divided into local and global topics. And the sliding window is introduced to lower co-occurrence information from document to sentence level, to implement fine-grained extraction of local topics. Secondly, the indicator variables are used to distinguish aspects and opinions. Finally, we incorporate the sentiment layer into LDA model to obtain the sentiment polarity of the whole review and specific aspects. The datasets of hotel and mobile phone are selected to verify the domain adaptability of this model. The experimental results verified the feasibility of FG-TSU model in the realization of opinion mining.

Memory networks for fine-grained opinion mining

Fine-grained opinion mining has attracted increasing attention recently because of its benefits for providing richer information compared with coarse-grained sentiment analysis. Under this problem, there are several existing works focusing on aspect (or opinion) terms extraction which utilize the syntactic relations among the words given by a dependency parser. These approaches, however, require additional information and highly depend on the quality of the parsing results. As a result, they may perform poorly on user-generated texts, such as product reviews, tweets, etc., whose syntactic structure is not precise. In this work, we offer an end-to-end deep learning model without any preprocessing. The model consists of a memory network that automatically learns the complicated interactions among aspect words and opinion words. Moreover, we extend the network with a multi-task manner to solve a finer-grained opinion mining problem, which is more challenging than the traditional fine-grained opinion mining problem. To be specific, the finer-grained problem involves identification of aspect and opinion terms within each sentence, as well as categorization of the identified terms at the same time. To this end, we develop an end-to-end multi-task memory network, where aspect/opinion terms extraction for a specific category is considered as a task, and all the tasks are learned jointly by exploring commonalities and relationships among them. We demonstrate state-of-the-art performance of our proposed model on several benchmark datasets.

End-to-end neural opinion extraction with a transition-based model

Fine-grained opinion extraction has received increasing interests in the natural language processing community. It usually involves several subtasks. Recently, joint methods and neural models have been investigated by several studies, achieving promising performance by using graph-based models such as conditional random field. In this work, we propose a novel end-to-end neural model alternatively for joint opinion extraction, by using a transition-based framework. First, we exploit multi-layer bi-directional long short term memory (LSTM) networks to encode the input sentences, and then decode incrementally based on partial output results dominated by a transition system. We use global normalization and beam search for training and decoding. Experiments on a standard benchmark show that the proposed end-to-end model can achieve competitive results compared with the state-of-the-art neural models of opinion extraction.

Fine-Grained Opinion Extraction from Chinese Car Reviews with an Integrated Strategy

With rapid development of E-commerce, a large amount of data including reviews about different types of products can be accessed within short time. On top of this, opinion mining is becoming increasingly effective to extract valuable information for product design, improvement and brand marketing, especially with fine-grained opinion mining. However, limited by the unstructured and causal expression of opinions, one cannot extract valuable information conveniently. In this paper, we propose an integrated strategy to automatically extract feature-based information, with which one can easily acquire detailed opinion about certain products. For adaptation to the reviews’ characteristics, our strategy is made up of a multi-label classification (MLC) for reviews, a binary classification (BC) for sentences and a sentence-level sequence labelling with a deep learning method. During experiment, our approach achieves 82% accuracy in the final sequence labelling task under the setting of a 20-fold cross validation. In addition, the strategy can be expediently employed in other reviews as long as there is an according amount of labelled data for startup.

Fine-grained Opinion Extraction with Mixed Network Model

With the development of opinion mining, the relevant areas are being valued more than before. As the foundation of opinion mining, the performance of opinion extraction is significant for the opinion mining results. Among the present methods of opinion extraction, many of them take advantages of the relation features. Unlike other models which use relations in an independent way, in this paper we propose a mixed network model which interactively uses two relations as well as other lexical and syntactic features to solve the extraction problem. A series of experiments show that our model outperforms other baseline models and has a further potential of improvement.

Fine-grained opinion mining of product review using sentiment and semantic orientation

Fine-grained opinion mining of product review using sentiment and semantic orientation

Fine-grained opinion mining of product review using sentiment and semantic orientation

The reviews or feedback about a product or service has become quite significant in marketing, promoting, or improvising the product or service, since e-commerce, or purchasing of online products, has recently become a trend. The availability of product reviews is online and in the form of text but these reviews are very much in an unstructured form, which does not help either the new consumer, or the organisation, to take any decision further. In this paper, we have proposed an approach based on opinion mining and sentiment analysis. We have explored the sentiment orientation and sentiment classification to evaluate the customers' review. The reviews of various mobiles were converted from unstructured to structure to extract the summarised knowledge from online reviews. The number of user reviews was explored and the empirical results found that the sentiment orientation and classification provides the effective methods for better decision-making and benchmarking.

Product Aspect Clustering by Incorporating Background Knowledge for Opinion Mining.

Product aspect recognition is a key task in fine-grained opinion mining. Current methods primarily focus on the extraction of aspects from the product reviews. However, it is also important to cluster synonymous extracted aspects into the same category. In this paper, we focus on the problem of product aspect clustering. The primary challenge is to properly cluster and generalize aspects that have similar meanings but different representations. To address this problem, we learn two types of background knowledge for each extracted aspect based on two types of effective aspect relations: relevant aspect relations and irrelevant aspect relations, which describe two different types of relationships between two aspects. Based on these two types of relationships, we can assign many relevant and irrelevant aspects into two different sets as the background knowledge to describe each product aspect. To obtain abundant background knowledge for each product aspect, we can enrich the available information with background knowledge from the Web. Then, we design a hierarchical clustering algorithm to cluster these aspects into different groups, in which aspect similarity is computed using the relevant and irrelevant aspect sets for each product aspect. Experimental results obtained in both camera and mobile phone domains demonstrate that the proposed product aspect clustering method based on two types of background knowledge performs better than the baseline approach without the use of background knowledge. Moreover, the experimental results also indicate that expanding the available background knowledge using the Web is feasible.

Automated rule selection for opinion target extraction

Opinion target extraction, also called aspect extraction, aims to extract fine-grained opinion targets from opinion texts, such as customer reviews of products and services. This task is important because opinions without targets are of limited use. It is one of the core tasks of the popular aspect-oriented opinion mining, and is also among the most challenging tasks tackled by opinion mining researchers. Previous work has shown that the syntactic-based approach, which employs extraction rules about grammar dependency relations between opinion words and aspects (or targets), performs quite well. This approach is highly desirable in practice because it is unsupervised and domain independent. The problem of this approach is that the extraction rules should be carefully selected and tuned manually so as not to produce too many errors. Although it is easy to evaluate the accuracy of each rule automatically, it is not easy to select a set of rules that produces the best overall result due to the overlapping coverage of the rules. In this paper, we propose two approaches to select an effective set of rules. The first approach employs a greedy algorithm, and the second one employs a local search algorithm, specifically, simulated annealing. Our experiment results show that the proposed approaches can select a subset of a given rule set to achieve significantly better results than the full rule set and the existing state-of-the-art CRF-based supervised method.

Improving Opinion Aspect Extraction Using Semantic Similarity and Aspect Associations

Aspect extraction is a key task of fine-grained opinion mining. Although it has been studied by many researchers, it remains to be highly challenging. This paper proposes a novel unsupervised approach to make a major improvement. The approach is based on the framework of lifelong learning and is implemented with two forms of recommendations that are based on semantic similarity and aspect associations respectively. Experimental results using eight review datasets show the effectiveness of the proposed approach.

Review mining for estimating users’ ratings and weights for product aspects

Fine-grained opinions are often buried in user reviews. The opinionated aspects may also be associated with different weights by reviewers to represent the aspects’ relative importance. As the opinions and weights provide valuable information about users’ preferences for products, they can facilitate the generation of personalised recommendations. However, few studies to date have investigated the three inter-connected tasks in a unified framework: aspect identification, aspect-based rating inference and weight estimation. In this paper, we propose a unified framework for performing the three tasks, which involves 1) identifying the product aspects mentioned in a review, 2) inferring the reviewer’s ratings for these aspects from the opinions s/he expressed in a review, and 3) estimating the reviewer’s weights for these aspects. The relationship among these three tasks is inherently dependent in that the output of one task adjusts the accuracy of another task. We particularly develop an unsupervised model to Collectively estimate Aspect Ratings and Weights (shorted as CARW), which performs all of the three tasks by enhancing each other mutually. We conduct experiments on three real-life datasets to evaluate the CARW model. Experimental results show that the proposed model can achieve better performance than the related methods regarding each task.

A novel localization and coverage framework for real-time participatory urban monitoring

Participatory sensing is a powerful paradigm in which users participate in the sensing campaign by collecting and crowdsourcing fine-grained information and opinions about events of interest (such as weather or environment monitoring, traffic conditions or accidents, crime scenes, emergency response, healthcare and wellness management), thus leading to actionable inferences and decisions. Based on the nature of user involvement, participatory sensing applications can be of two types—automated and user manipulated. The first type of applications automatically collects data samples from smartphone sensors and sends them to the server. On the other hand, the second type of applications depends on the users to manually collect data samples and upload them at their convenience. Because of the high density of smartphone users in urban population and ease of participation, the automated participatory sensing paradigm can be effectively applied to continuous monitoring of various phenomena in urban scenarios (e.g., fine-grained temperature monitoring, noise or air pollution), leading to what is called participatory urban sensing. However, for creating a fine-grained and real-time map of the monitored area, the data samples need to be collected continuously (at a high frequency) which poses several research challenges. First, how to ensure coverage of the collected data that reflects how well the targeted area is monitored? Second, how to localize the smartphones since continuous usage of the location sensor (e.g., GPS) can drain the battery in few hours? Third, how to provide energy efficiency in the data collection process by collecting necessary data samples in each data collection round? In this article, we propose a novel framework called PLUS to address three major issues in real-time participatory urban monitoring applications, namely, ensuring coverage of the collected data, localization of the participating smartphones, and overall energy efficiency of the data collection process. Specifically the PLUS framework can guarantee a specified requirement of partial data coverage of the monitored area in an energy efficient manner. Additionally we devised a Markov-Predictor based energy efficient outdoor localization scheme for the mobile devices to participate in the data collection process. Simulation studies and real life experiments exhibit that PLUS can significantly reduce energy consumption of the mobile devices for urban monitoring applications as compared to traditional approaches.

An Association-Based Unified Framework for Mining Features and Opinion Words

Mining features and opinion words is essential for fine-grained opinion analysis of customer reviews. It is observed that semantic dependencies naturally exist between features and opinion words, even among features or opinion words themselves. In this article, we employ a corpus statistics association measure to quantify the pairwise word dependencies and propose a generalized association-based unified framework to identify features, including explicit and implicit features, and opinion words from reviews. We first extract explicit features and opinion words via an association-based bootstrapping method (ABOOT). ABOOT starts with a small list of annotated feature seeds and then iteratively recognizes a large number of domain-specific features and opinion words by discovering the corpus statistics association between each pair of words on a given review domain. Two instances of this ABOOT method are evaluated based on two particular association models, likelihood ratio tests (LRTs) and latent semantic analysis (LSA). Next, we introduce a natural extension to identify implicit features by employing the recognized known semantic correlations between features and opinion words. Experimental results illustrate the benefits of the proposed association-based methods for identifying features and opinion words versus benchmark methods.

Co-Extracting Opinion Targets and Opinion Words from Online Reviews Based on the Word Alignment Model

Mining opinion targets and opinion words from online reviews are important tasks for fine-grained opinion mining, the key component of which involves detecting opinion relations among words. To this end, this paper proposes a novel approach based on the partially-supervised alignment model, which regards identifying opinion relations as an alignment process. Then, a graph-based co-ranking algorithm is exploited to estimate the confidence of each candidate. Finally, candidates with higher confidence are extracted as opinion targets or opinion words. Compared to previous methods based on the nearest-neighbor rules, our model captures opinion relations more precisely, especially for long-span relations. Compared to syntax-based methods, our word alignment model effectively alleviates the negative effects of parsing errors when dealing with informal online texts. In particular, compared to the traditional unsupervised alignment model, the proposed model obtains better precision because of the usage of partial supervision. In addition, when estimating candidate confidence, we penalize higher-degree vertices in our graph-based co-ranking algorithm to decrease the probability of error generation. Our experimental results on three corpora with different sizes and languages show that our approach effectively outperforms state-of-the-art methods.

Joint Modeling of Opinion Expression Extraction and Attribute Classification

In this paper, we study the problems of opinion expression extraction and expression-level polarity and intensity classification. Traditional fine-grained opinion analysis systems address these problems in isolation and thus cannot capture interactions among the textual spans of opinion expressions and their opinion-related properties. We present two types of joint approaches that can account for such interactions during 1) both learning and inference or 2) only during inference. Extensive experiments on a standard dataset demonstrate that our approaches provide substantial improvements over previously published results. By analyzing the results, we gain some insight into the advantages of different joint models.