Document-level Sentiment Analysis Research Articles

A survey on personalized document-level sentiment analysis

Personalized document-level sentiment analysis (PDSA) plays an important role in many real-world applications. So far, various deep learning models for PDSA have been proposed. However, there has been no systematic survey in this area. To address this issue, in this paper, we overview the existing methods of PDSA, and propose a novel two-dimensional PDSA taxonomy. Specifically, in the dimension of attribute usage type, PDSA works are divided into two groups: (1) user-based and (2) user and product-based models. In the dimension of attribute processing method, PDSA works are divided into two groups: (1) feature-based and (2) relation-based models. To fill in the research blank indicated by the taxonomy, we further propose an architecture named User Correlation Mining (UCM) for PDSA. Specifically, UCM contains two components, namely Similar User Cluster Module (SUCM) and Triple Attributes BERT Model (TABM). SUCM is responsible for user clustering, and TABM aims to classify the user’s sentiment based on the information of users, products, user clusters and user reviews. To evaluate the performances of the existing works as well as UCM, we conduct extensive experiments on three real-world datasets. The experiment results show that our proposed architecture UCM outperforms the other state-of-the-art methods.

A hybrid multi-source data fusion for word, sentence, aspect, and document-level sentiment analysis on real-time databases

Sentiment analysis (SA)at the sentence, aspect, and document levels determines the sentiment of particular aspect phrases in a given sentence. Due to their capacity to extract sentiment information from text in aspect-level sentiment classification, neural networks (NNs) have achieved significant success. Generally speaking, sufficiently sizable training corpora are necessary for NNs to be effective. The performance of NN-based systems is reduced by the small size of the aspect-level corpora currently available. In this research, we suggest a gated bilateral recurrent neural network (G-Bi-RNN) as a foundation for multi-source data fusion, their system offers sentiment information that several sources. We develop a uniform architecture specifically to include information from sentimental lexicons, including aspect- and sentence-level corpora. To further provide aspect-specific phrase representations for SA, we use G-Bi-RNN, a deep bilateral Transformer-based pre-trained language model. We assess our methods using SemEval 2014 datasets for laptops and restaurants. According to experimental findings, our method consistently outperforms cutting-edge techniques on all datasets. We use a number of well-known aspect-level SA datasets to assess the efficacy of our model. Experiments show that when compared to baseline models, the suggested model can produce state-of-the-art results.

Deep learning-based application for multilevel sentiment analysis of Indonesian hotel reviews

PurposeIn this study, we present a web-based application that retrieves hotel review documents in Indonesian languages from an online travel agent (OTA) and analyses their sentiments from the coarse-grained document to the fine-grained aspect level. Design/Methodology/Approach: There are four main stages in this study: development of sentiment analysis model at the document level based on a convolutional neural network (CNN), development of sentiment analysis model at the aspect level based on an improved long short-term memory (LSTM), model deployment for multilevel sentiment analysis in a web-based application, and its performance evaluation. The developed application uses several sentiment visualizations types at coarse-grained and fine-grained levels, such as pie charts, line charts, and bar charts. FindingThe application's functionality was demonstrated in practice based on three datasets from three OTA websites, which were analyzed and evaluated based on several matrices, namely, the precision, recall, and F1-score. The results revealed that the performance for the F1-score was 0.95 ± 0.03, 0.87 ± 0.02, and 0.92 ± 0.07 for document-level sentiment analysis, aspect-level sentiment analysis, and aspect-polarity detection, respectively. OriginalityThe developed application (Sentilytics 1.0) can analyze sentiment at document and aspect levels. The two levels of sentiment analysis are based on two models generated by fine-tuning CNN and LSTM models using specific architectures and domain data (Indonesian hotel reviews).

Multimodal Interaction and Fused Graph Convolution Network for Sentiment Classification of Online Reviews

An increasing number of people tend to convey their opinions in different modalities. For the purpose of opinion mining, sentiment classification based on multimodal data becomes a major focus. In this work, we propose a novel Multimodal Interactive and Fusion Graph Convolutional Network to deal with both texts and images on the task of document-level multimodal sentiment analysis. The image caption is introduced as an auxiliary, which is aligned with the image to enhance the semantics delivery. Then, a graph is constructed with the sentences and images generated as nodes. In line with the graph learning, the long-distance dependencies can be captured while the visual noise can be filtered. Specifically, a cross-modal graph convolutional network is built for multimodal information fusion. Extensive experiments are conducted on a multimodal dataset from Yelp. Experimental results reveal that our model obtains a satisfying working performance in DLMSA tasks.

Aspect-based Sentiment Analysis on Car Reviews Using SpaCy Dependency Parsing and VADER

All businesses, including car manufacturers, need to understand what aspects of their products are perceived as positive and negative based on user reviews so that they can make improvements for the negative aspects and maintain the already positive aspects of their products. One of the available tools for this task is Sentiment Analysis. The traditional document-level and sentence-level sentiment analysis will only classify each document / sentence into a class. This approach is incapable of finding the more fine-grained sentiment for a specific aspect of interest, for example, comfort, price, engine, paint, etc. Therefore, in this case, Aspect-based Sentiment Analysis is used. A total of 22.702 rows of car review data are scraped from the Edmunds website (www.edmunds.com) for a specific car manufacturer. Dependency Parsing and noun phrase extraction were carried out using the SpaCy module in Python, and VADER sentiment analysis was used to determine the polarity of the sentiment for each noun phrase. Results showed that the vast majority of the sentiments are on the positive aspects: comfortable to drive, good fuel economy / mileage, reliability, spaciousness, value for money, helpful rear camera, quiet ride, good acceleration, well-designed, good sound system, and solid build. The results for the negative aspects have some similar aspects with those in the positive class but has a very low frequency. This finding means that the vast majority of the users are satisfied with multiple aspects of the produced cars. The limitation of this research and future research direction are discussed.

Towards Reliable Baselines for Document-Level Sentiment Analysis in the Czech and Slovak Languages

This article helps establish reliable baselines for document-level sentiment analysis in highly inflected languages like Czech and Slovak. We revisit an earlier study representing the first comprehensive formulation of such baselines in Czech and show that some of its reported results need to be significantly revised. More specifically, we show that its online product review dataset contained more than 18% of non-trivial duplicates, which incorrectly inflated its macro F1-measure results by more than 19 percentage points. We also establish that part-of-speech-related features have no damaging effect on machine learning algorithms (contrary to the claim made in the study) and rehabilitate the Chi-squared metric for feature selection as being on par with the best performing metrics such as Information Gain. We demonstrate that in feature selection experiments with Information Gain and Chi-squared metrics, the top 10% of ranked unigram and bigram features suffice for the best results regarding online product and movie reviews, while the top 5% of ranked unigram and bigram features are optimal for the Facebook dataset. Finally, we reiterate an important but often ignored warning by George Forman and Martin Scholz that different possible ways of averaging the F1-measure in cross-validation studies of highly unbalanced datasets can lead to results differing by more than 10 percentage points. This can invalidate the comparisons of F1-measure results across different studies if incompatible ways of averaging F1 are used.

Re-Engineered Word Embeddings for Improved Document-Level Sentiment Analysis

In this paper, a novel re-engineering mechanism for the generation of word embeddings is proposed for document-level sentiment analysis. Current approaches to sentiment analysis often integrate feature engineering with classification, without optimizing the feature vectors explicitly. Engineering feature vectors to match the data between the training set and query sample as proposed in this paper could be a promising way for boosting the classification performance in machine learning applications. The proposed mechanism is designed to re-engineer the feature components from a set of embedding vectors for greatly increased between-class separation, hence better leveraging the informative content of the documents. The proposed mechanism was evaluated using four public benchmarking datasets for both two-way and five-way semantic classifications. The resulting embeddings have demonstrated substantially improved performance for a range of sentiment analysis tasks. Tests using all the four datasets achieved by far the best classification results compared with the state-of-the-art.

A Joint Multi-task Architecture for Document-level Aspect-based Sentiment Analysis in Vietnamese

A Joint Multi-task Architecture for Document-level Aspect-based Sentiment Analysis in Vietnamese

Supervised Machine Learning Algorithms for Sentiment Analysis of Bangla Newspaper

We can state undoubtedly that Bangla language is rich enough to work with and implement various Natural Language Processing (NLP) tasks. Though it needs proper attention, hardly NLP field has been explored with it. In this age of digitalization, large amount of Bangla news contents are generated in online platforms. Some of the contents are inappropriate for the children or aged people. With the motivation to filter out news contents easily, the aim of this work is to perform document level sentiment analysis (SA) on Bangla online news. In this respect, the dataset is created by collecting news from online Bangla newspaper archive. Further, the documents are manually annotated into positive and negative classes. Composite process technique of “Pipeline” class including Count Vectorizer, transformer (TF-IDF) and machine learning (ML) classifiers are employed to extract features and to train the dataset. Six supervised ML classifiers (i.e. Multinomial Naive Bayes (MNB), K-Nearest Neighbor (K-NN), Random Forest (RF), (C4.5) Decision Tree (DT), Logistic Regression (LR) and Linear Support Vector Machine (LSVM)) are used to analyze the best classifier for the proposed model. There has been very few works on SA of Bangla news. So, this work is a small attempt to contribute in this field. This model showed remarkable efficiency through better results in both the validation process of percentage split method and 10-fold cross validation. Among all six classifiers, RF has outperformed others by 99% accuracy. Even though LSVM has shown lowest accuracy of 80%, it is also considered as good output. However, this work has also exhibited surpassing outcome for recent and critical Bangla news indicating proper feature extraction to build up the model.

Topic-Based Document-Level Sentiment Analysis Using Contextual Cues

Document-level Sentiment Analysis is a complex task that implies the analysis of large textual content that can incorporate multiple contradictory polarities at the phrase and word levels. Most of the current approaches either represent textual data using pre-trained word embeddings without considering the local context that can be extracted from the dataset, or they detect the overall topic polarity without considering both the local and global context. In this paper, we propose a novel document-topic embedding model, DocTopic2Vec, for document-level polarity detection in large texts by employing general and specific contextual cues obtained through the use of document embeddings (Doc2Vec) and Topic Modeling. In our approach, (1) we use a large dataset with game reviews to create different word embeddings by applying Word2Vec, FastText, and GloVe, (2) we create Doc2Vecs enriched with the local context given by the word embeddings for each review, (3) we construct topic embeddings Topic2Vec using three Topic Modeling algorithms, i.e., LDA, NMF, and LSI, to enhance the global context of the Sentiment Analysis task, (4) for each document and its dominant topic, we build the new DocTopic2Vec by concatenating the Doc2Vec with the Topic2Vec created with the same word embedding. We also design six new Convolutional-based (Bidirectional) Recurrent Deep Neural Network Architectures that show promising results for this task. The proposed DocTopic2Vecs are used to benchmark multiple Machine and Deep Learning models, i.e., a Logistic Regression model, used as a baseline, and 18 Deep Neural Networks Architectures. The experimental results show that the new embedding and the new Deep Neural Network Architectures achieve better results than the baseline, i.e., Logistic Regression and Doc2Vec.

Conciseness is better: Recurrent attention LSTM model for document-level sentiment analysis

Long short-term memory (LSTM) or gated recurrent units (GRUs) are usually employed to recurrently learn variable-length sentence representations with long-range dependency in document-level sentiment analysis. However, LSTM and GRUs are biased models in which the words in the tail of sentences are dominant over the words at the beginning. Although some reweighting methods, such as self-attention, are useful for reweighting each word according to its importance, when processing a document with many tokens, they tend to assign equally smaller weights to each word, leading to the keywords being covered by nonsentiment words. In this paper, a recurrent attention LSTM neural network is presented to iteratively locate an attention region covering the key sentiment words. By gradually reducing the range of attention and the number of tokens, the model can leverage the weight of the key sentiment words for final classification. Additionally, a joint loss function is implemented to highlight both keywords and appropriate attention regions. The comparative experiments are conducted on the IMDB, Yelp and Amazon document-level corpora. The results show that the proposed model outperforms several state-of-the-art methods in document-level sentiment classification.

Exploring BERT for Aspect Extraction in Portuguese Language

Sentiment Analysis is the computer science field that comprises techniques that aim to automatically extract opinions from texts. Usually, these techniques assign a Sentiment Orientation to the whole document (Document Level Sentiment Analysis). But a document can express sentiment about several aspects of an entity. Methods that extract those aspects, paired with the sentiment about them, operate in the Aspect Level. Aspect-Based Sentiment Analysis approaches can be split into two stages: Aspect Extraction and Aspect Sentiment Classification. The literature presents works mainly focused on reviews about hotels, smartphones, or restaurants. In this work, we present an approach for Aspect Extraction based on Multilingual (Google's) and Portuguese (BERTimbau) BERT pre-trained models. Our experiments show that Aspect Extraction based on BERT pre-trained for Portuguese achieved Balanced Accuracy of up to 93% on a corpus of reviews about the accommodation sector.

DSC_NEXT: Document-Level Sentiment ClassificatioN with EXTensional-Information

Document-level sentiment classification is to assign an overall sentiment polarity to an opinion document. Some researchers have already realized that, in addition to document texts, extensional-information such as product features and user preferences can be quite useful. Many previous studies represent them as ID-type extensional-information and incorporate them into deep learning models. However, they ignore the descriptive extensional information that is also useful for document representations. This paper covers the following aspects: (1) introduces the Description of Opinion Target (DOT), a new extensional-information for document-level sentiment classification, (2) builds the Document-level Sentiment ClassificatioN with EXTensional-information (DSC_NEXT) dataset which consists of three datasets: IMDB_NEXT, Yelp_NEXT and CMRDB_NEXT and (3) validates the effectiveness of DOT by performing experiments based on current state-of-the-art (SOTA) document-level sentiment analysis methods. Implications for using extensional-information in neural network models are also considered.

TensSent: a tensor based sentimental word embedding method

The representation of words as vectors, conventionally known as word embeddings, has drawn considerable attention in recent years as feature learning techniques for natural language processing. The majority of these methods operate solely on the semantic and the syntactic aspects of a text, remaining oblivious to sentimental information. However, as numerous words with opposite polarities may appear in similar contexts, such as “interesting” and “boring”, the exclusive use of context-oriented information lacks the required particulars for generating word embeddings as features in sentiment analysis. Along this thread, the present study proposes two novel unsupervised models to integrating word polarity information and word co-occurrences as more tailored features for sentiment analysis. Word polarity and co-occurrence come together in the form of a tensor and tensor factorization is employed for generating the word embeddings. The experimental results on IMDB and SemEval-task2 datasets demonstrate the relatively higher performance of the proposed method compared to baseline approaches on the tasks of document-level sentiment analysis by 4.

An Overview on Fine-grained Text Sentiment Analysis: Survey and Challenges

Among various natural language process tasks, sentiment analysis has always been a research hotspot. From the initial sentence-level and document-level coarse-grained sentiment analysis to recent fine-grained sentiment analysis on the aspect word level, researchers are committed to applying diverse methods to obtain better sentiment analysis results, ranging from lexicon-based, statistical machine learning methods to deep learning models. In the change of technology, several benchmark datasets that can be used for model performance comparison are gradually yielded. This article summarizes the current research status of aspect-level text sentiment analysis from multiple dimensions such as dataset, mainstream methods, and evaluation indicators, finally, it puts forward the challenges facing and potential research directions from a unique perspective.

An improved aspect-category sentiment analysis model for text sentiment analysis based on RoBERTa

The aspect-category sentiment analysis can provide more and deeper information than the document-level sentiment analysis, because it aims to predict the sentiment polarities of different aspect categories in the same text. The main challenge of aspect-category sentiment analysis is that different aspect categories may present different polarities in the same text. Previous studies combine the Long Short-Term Memory (LSTM) and attention mechanism to predict the sentiment polarity of the given aspect category, but the LSTM-based methods are not really bidirectional text feature extraction methods. In this paper, we propose a multi-task aspect-category sentiment analysis model based on RoBERTa (Robustly Optimized BERT Pre-training Approach). Treating each aspect category as a subtask, we employ the RoBERTa based on deep bidirectional Transformer to extract features from both text and aspect tokens, and apply the cross-attention mechanism to guide the model to focus on the features most relevant to the given aspect category. According to the experimental results, the proposed model outperforms other models for comparison in aspect-category sentiment analysis. Furthermore, the influencing factors of our proposed model are also analyzed.

Tens-embedding: A Tensor-based document embedding method

A human is capable of understanding and classifying a text but a computer can understand the underlying semantics of a text when texts are represented in a way comprehensible by computers. The text representation is a fundamental stage in natural language processing (NLP). One of the main drawbacks of existing text representation approaches is that they only utilize one aspect or view of a text e.g. They only consider texts by their words while the topic information can be extracted from text as well. The term-document and document-topic matrix are two views of a text and contain complementary information. We use the strength of both views to extract a richer representation. In this paper, we propose three different text representation methods with the help of these two matrices and tensor factorization to utilize the power of both views. The proposed approach (Tens-Embedding) was applied in the tasks of text classification, sentence-level and document-level sentiment analysis and text clustering wherein the conducted experiments on 20newsgroups, R52, R8, MR and IMDB datasets indicated the superiority of the proposed method in comparison with other document embedding techniques.

Contextual Sentiment Neural Network for Document Sentiment Analysis

Although deep neural networks are excellent for text sentiment analysis, their applications in real-world practice are occasionally limited owing to their black-box property. In this study, we propose a novel neural network model called contextual sentiment neural network (CSNN) model that can explain the process of its sentiment analysis prediction in a way that humans find natural and agreeable and can catch up the summary of the contents. The CSNN has the following interpretable layers: the word-level original sentiment layer, word-level sentiment shift layer, word-level global importance layer, word-level contextual sentiment layer, and concept-level contextual sentiment layer. Because of these layers, this network can explain the process of its document-level sentiment analysis results in a human-like way using these layers. Realizing the interpretability of each layer in the CSNN is a crucial problem in the development of this CSNN because the general back-propagation method cannot realize such interpretability. To realize this interpretability, we propose a novel learning strategy called initialization propagation (IP) learning. Using real textual datasets, we experimentally demonstrate that the proposed IP learning is effective for improving the interpretability of each layer in CSNN. We then experimentally demonstrate that the CSNN has both the high predictability and high explanation ability.

Extracting Online Product Review Patterns and Causes: A New Aspect/Cause Based Heuristic for Designers

Identifying and analysing large amounts of online product reviews (OPR) have become a critical challenge for product design due to their valuable and insightful content. Despite the potential of the OPRs, designers cannot benefit from this opportunity because the analysis of the interpretations requires knowledge of different areas. This study proposes a novel framework for designers by utilizing online reviews for product design depending on the real-world OPRs of a sample product in terms of clarity, comparability, and validity. The framework contains five steps – retrieval of customer text, mining text polarity and product aspects by Document-Level Sentiment Analysis (DLSA) and Aspect-Based Sentiment Analysis (ABSA), summarizing and visualizing review candidate patterns by Correspondence Analysis (CA) and discovering and predicting possible causes of candidate patterns by correlation analysis.

ScrAnViz: a tool for analytics and visualisation of unstructured data

Existing big data visualisation tools are meant for visualising structured data. But survey shows that about 80-90% of potentially usable business information is in unstructured format. Analysing unstructured data is challenging due to lack of structure and relational form. In this paper, we have proposed a tool called ScrAnViz that can structure data, perform analysis and provide visualisation thereby helps in decision making for business people and end users. An attribute based opinion mining algorithm has been developed and implemented. Performance analysis shows that the algorithm has reduced the search time by three times than the traditional document level sentiment analysis systems.