Sentiment Words Research Articles

Integrating external knowledge into aspect-based sentiment analysis using graph neural network

Aspect-based sentiment analysis aims to analyze the sentiment polarity of a given aspect. The graph convolutional neural network model is widely used. However, most existing research focuses on mining the context-word-to-aspect-word dependencies of dependency trees based on the sentence itself without using much text-related external knowledge. In addition, the problem of reasonably capturing words outside the multihop grammatical distance and edge label hinders the effect of GCN. This paper proposes a graph convolutional network that fuses external knowledge (sentiment lexicon and part-of-speech information) (EK-GCN). Specifically, we conduct a statistical study on part-of-speech and construct a part-of-speech matrix to fully consider the influence of denying words, degree words, and other words that affect sentiment expression in sentences on sentiment classification. Then, an external sentiment lexicon is used to assign sentiment scores to each word in the sentence to construct a sentiment score matrix to highlight the weight of sentiment words, which to a certain extent, compensates for the fact that the syntactic dependency tree cannot capture edge labels. In addition, we design a Word–Sentence Interaction Network (WSIN), which can fully consider the information of the current aspect word and interact with the context information of the reviews to filter useful sentence information. We conduct experiments on four benchmark datasets, and the excellent experimental results demonstrate the effectiveness of our model. The results also verify that fully integrating external knowledge can assist in completing aspect-based sentiment analysis tasks.

Correlation of Narrative Evaluations to Clerkship Grades Using Statistical Sentiment Analysis.

Narrative evaluations are essential components of medical student assessment. This study evaluated how well narrative clerkship evaluation word choice correlated with an assigned letter grade. One hundred clerkship evaluations, 50 from family medicine (FM) and 50 from internal medicine (IM), with even distribution of "Honors" and "Near-Honors" among medical students that graduated in 2020 from the Oregon Health and Science University (OHSU) were examined. A textual sentiment analysis, which evaluates positive and negative word choice, was used to determine each evaluation's collective sentiment. An average sentiment score and character count were calculated for Honors and Near-Honors evaluations from both clerkship disciplines. Sentiment word totals were used to form "word clouds" that highlight the most frequent word selections. While sentiment scores positively correlated with the assigned grade, there was no statistically significant difference between the average sentiment scores among Honors and Near Honors graded evaluations within the FM or IM clerkship evaluation sets. There was no significant difference in evaluation character length among the assigned grades. Among FM evaluations, "outstanding" and "excellent" were the two most common sentiment words used in both Honors and Near-Honors. Among IM evaluations, outstanding and excellent were most commonly used in Honors evaluations, while "excellent" and "good" were most common in Near-Honors. This study outlines a novel text analysis method for analyzing narrative evaluation association with assigned grade that other institutions can utilize. Sentiment word choices are not significantly different among Honors and Near Honors clerkship narrative evaluations. The online version contains supplementary material available at 10.1007/s40670-022-01654-2.

Opinion analysis and aspect understanding during covid-19 pandemic using BERT-Bi-LSTM ensemble method

Social media platforms significantly increase general information about disease severity and inform preventive measures among community members. To identify public opinion through tweets on the subject of Covid-19 and investigate public sentiment in the country over the period. This article proposed a novel method for sentiment analysis of coronavirus-related tweets using bidirectional encoder representations from transformers (BERT) bi-directional long short-term memory (Bi-LSTM) ensemble learning model. The proposed approach consists of two stages. In the first stage, the BERT model gains the domain knowledge with Covid-19 data and fine-tunes with sentiment word dictionary. The second stage is the Bi-LSTM model, which is used to process the data in a bi-directional way with context sequence dependency preserving to process the data and classify the sentiment. Finally, the ensemble technique combines both models to classify the sentiment into positive and negative categories. The result obtained by the proposed method is better than the state-of-the-art methods. Moreover, the proposed model efficiently understands the public opinion on the Twitter platform, which can aid in formulating, monitoring and regulating public health policies during a pandemic.

A New Sentiment-Enhanced Word Embedding Method for Sentiment Analysis

Since some sentiment words have similar syntactic and semantic features in the corpus, existing pre-trained word embeddings always perform poorly in sentiment analysis tasks. This paper proposes a new sentiment-enhanced word embedding (S-EWE) method to improve the effectiveness of sentence-level sentiment classification. This sentiment enhancement method takes full advantage of the mapping relationship between word embeddings and their corresponding sentiment orientations. This method first converts words to word embeddings and assigns sentiment mapping vectors to all word embeddings. Then, word embeddings and their corresponding sentiment mapping vectors are fused to S-EWEs. After reducing the dimensions of S-EWEs through a fully connected layer, the predicted sentiment orientations are obtained. The S-EWE method adopts the cross-entropy function to calculate the loss between predicted and true sentiment orientations, and backpropagates the loss to train the sentiment mapping vectors. Experiments show that the accuracy and macro-F1 values of six sentiment classification models using Word2Vec and GloVe with the S-EWEs are on average 1.07% and 1.58% higher than those without the S-EWEs on the SemEval-2013 dataset, and on average 1.23% and 1.26% higher than those without the S-EWEs on the SST-2 dataset. In all baseline models with S-EWEs, the convergence time of the attention-based bidirectional CNN-RNN deep model (ABCDM) with S-EWEs was significantly decreased by 51.21% of ABCDM on the SemEval-2013 dataset. The convergence time of CNN-LSTM with S-EWEs was vastly reduced by 41.34% of CNN-LSTM on the SST-2 dataset. In addition, the S-EWE method is not valid for contextualized word embedding models. The main reasons are that the S-EWE method only enhances the embedding layer of the models and has no effect on the models themselves.

Fast-Track Product Evaluation From Text Reviews in M-Commerce

The popularity of mobile commerce has offered many new challenges for investigating public sentiments. With an uncountable number of stores and products available on the marketplace, customers heavily relied on the comments or reviews posted by others to support their buying decisions. For the online retailer's side, these text reviews are valuable resources to understand the latest customer expectations and devise a better product plan for launching suitable products to customers. Sentiment analysis is then developed for the evaluation of a significant amount of text data by searching the sentiment words. Nevertheless, different writers may have different perceptions on the sentiment words, and hence, this inconsistency would be amplified. In this connection, a novel approach to obtain public sentiment by combining the topic modeling, fuzzy set, and multi-criteria decision-making approaches is proposed. The uncertainty of different perceptions on the sentiment words is remedied by fuzzy-set.

Parameter Optimization of Educational Network Ecosystem Based on BERT Deep Learning Model

The key sentimental words in the text cannot be paid attention to effectively, and language knowledge such as the text information and the sentimental resources are relied on. Therefore, it is necessary to make full use of this unique sentimental information to achieve the best performance of the model. In order to solve the problems, a method based on the fusion of the convolutional neural network and the bidirectional GRU network text sentiment analysis capsule model to analyze the ideological and political education of public opinion is put forward. In this model, each sentiment category is combined with the attention mechanism to generate feature vectors to construct sentiment capsules. Finally, the text sentiment categories are judged according to the attributes of the capsules. The model is tested on MR, IMDB, SST-5, and the data set of the ideological and political education review. Experimental results show that compared with MC-CNN-LSTM, the readiness rate of the proposed model is improved by 5.1%, 2.8%, 2.8%, and 1.6% on four public Chinese and English data sets, respectively. Compared with LR-Bi-LSTM, NSCL, and multi-Bi-LSTM models, the accuracy of the proposed MC-BiGRU-Capsule model on MR and SST-5 data are 3.2%, 2.4%, and 3.4% higher than that of the LR-Bi-LSTM, NSCL, and multi-Bi-LSTM models, respectively. It also shows a better classification effect on multiclassification data sets. It is concluded that compared with other baseline models, this method has a better classification effect.

TaskDrop: A competitive baseline for continual learning of sentiment classification

In this paper, we study the multi-task sentiment classification problem in the continual learning setting, i.e., a model is sequentially trained to classify the sentiment of reviews of products in a particular category. The use of common sentiment words in reviews of different product categories leads to large cross-task similarity, which differentiates it from continual learning in other domains. This knowledge sharing nature renders forgetting reduction focused approaches less effective for the problem under consideration. Unlike existing approaches, where task-specific masks are learned with specifically presumed training objectives, we propose an approach called Task-aware Dropout (TaskDrop) to randomly sample a binary mask for each task. While the standard dropout generates and applies random masks for each training instance per epoch for regularization, random masks in TaskDrop are used for model capacity allocation and reuse to each coming task. We conducted experimental studies on Amazon review data and made comparison to various baselines and state-of-the-art approaches. Our empirical results show that regardless of simplicity, TaskDrop overall achieved competitive performance, especially after relatively long term learning. This demonstrates that the proposed random capacity allocation mechanism works well for continual sentiment classification.

Text‐based soybean futures price forecasting: A two‐stage deep learning approach

AbstractThis paper investigates the soybean futures price prediction problem from a new perspective and proposes an effective prediction model named Two‐Stage Hybrid Long Short‐Term Memory (TSH‐LSTM) by using text data from social media. First, the unstructured text is transformed into structured data by sentiment analysis and text classification methods. The improved sentiment score is computed by combining the degree centrality of sentiment words based on the sentiment dictionary method, and the characteristics of price fluctuations in texts are learned through the text Recurrent Convolutional Neural Networks. Second, the significant relationship between social media features and soybean futures price is assessed through stepwise regression, and the results of such an assessment are used as a basis for the identification of significant factors as input variables of the prediction model. Finally, the TSH‐LSTM prediction model is designed, and the final prediction result is acquired through the combination of prediction results of each stage using the error reciprocal method. The empirical results indicate that the incorporation of the social media text feature helps improve forecasting performances. Specifically, the proposed TSH‐LSTM is more accurate than univariate LSTM, multivariate LSTM, and eXtreme Gradient Boosting.

How has airport service quality changed in the context of COVID-19: A data-driven crowdsourcing approach based on sentiment analysis

Airport service quality (ASQ) is a competitive advantage for airport management in today's airport market. Since the COVID-19 health crisis has unprecedentedly influenced airport regulations and operations, effective measurement of ASQ has become crucial for airport administrations. Surveying travelers' attitudes is useful for ASQ assessment but collecting responses could be time-consuming and costly. Therefore, this paper adopts a data-driven crowdsourcing approach to study ASQ during the COVID-19 pandemic by investigating Google Maps reviews from the 98 busiest U.S. airports. To do so, this study develops a topical ontology of keywords regarding ASQ attributes and uses a sentiment tool to derive passengers' attitudes. Through sentiment analysis, Google Maps reviews show more positive sentiment toward environment and personnel but remain constant about facilities during COVID-19. The lexical salience-valence analysis (LSVA) is then applied to explain such changes by tracking the sentiment of frequent words in reviews. Through correlation and regression analysis, this study demonstrates that rating is significantly related to check-in, environment, and personnel in pre-and post-COVID periods. Additionally, the effect of access, wayfinding, facilities, and environment on rating significantly differs between the two periods. The findings illustrate the effectiveness of leveraging online reviews and offer practical implications for what matters to air travelers, especially in the COVID-19 context.

Sentiment Analysis of Chinese Reviews Based on BiTCN-Attention Model

It is of great significance for individuals, enterprises, and government departments to analyze and excavate the sentiment in the comments. Many deep learning models are used for text sentiment analysis, and the BiTCN model has good efficacy on sentiment analysis. However, in the actual semantic expression, the contribution of each word to the sentiment tendency is different, BiTCN treats it fairly and does not pay more attention to the key sentiment words. For this problem, a sentiment analysis model based on the BiTCN-Attention is proposed in this paper. The Self-Attention mechanism and Multi-Head Self-Attention mechanism are added to BiTCN respectively to form BiTCN-SA and BiTCN-MHSA, which improve the weight of sentiment words and the accuracy of feature extraction, to increase the effect of sentiment analysis. The experimental results show that the model accuracies of BiTCN-SA and BiTCN-MHSA in the JingDong commodity review data set are 3.96% and 2.41% higher than that of BiTCN, respectively. In the comment data set of DianPing, the accuracy of BiTCN-SA and BiTCN-MHSA improved by 4.62% and 3.49%, respectively, compared with that of BiTCN.

Refining Word Embeddings with Sentiment Information for Sentiment Analysis

Natural Language Processing problems generally require the use of pre-trained distributed word representations to be solved with deep learning models. However, distributed representations usually rely on contextual information which prevents them from learning all the important word characteristics. The task of sentiment analysis suffers from such a problem because sentiment information is ignored during the process of learning word embeddings. The performance of sentiment analysis can be affected since two words with similar vectors may have opposite sentiment orientations. The present paper introduces a novel model called Continuous Sentiment Contextualized Vectors (CSCV) to address this problem. The proposed model can learn word sentiment embedding using its surrounding context words. It uses Continuous Bag-of-Words (CBOW) model to deal with the context and sentiment lexicons to identify sentiment. Existing pre-trained vectors are combined then with the obtained sentiment vectors using Principal component analysis (PCA) to enhance their quality. The experiments show that: (1) CSCV vectors can be used to enhance any pre-trained word vectors; (2) The result vectors strongly alleviate the problem of similar words with opposite polarities; (3) The performance of sentiment classification is improved by applying this approach.

How Incongruence Between Managers’ Tone of Voice and Word Sentiment Influences Security Analysts

Maintaining favorable analyst recommendations is important to organizations and their management, as these recommendations provide stakeholders with expert opinions about organizations’ well-being. Hence, there is a strong impetus for managers to prevent analysts downgrading their organization. In verbal communication, such as in earnings calls, managers not only try to influence analysts via the words that they use, but also via the tone in which they speak (i.e., paraverbal communication). Paraverbal communication is both an extra channel of information for analysts, and a strategic tool managers may use to exert influence. On the one hand, management wants to signal trustworthiness through congruency in their communication, but on the other hand is inclined to offset negative information by speaking positively, or downplay expectations by using a negative tone when discussing positive information. Leveraging several state-of-the-art Natural Language Processing algorithms, using earnings calls’ audio and textual data, we analyze the incongruence in managers’ communication. We find that paraverbal communication affects analysts’ recommendations and show which paraverbal strategies characterize organizations that are least likely to get downgraded. Our study therefore contributes to the literature on corporate communication, demonstrating the importance of verbal tone in managing the relationship with market intermediaries (analysts). Furthermore, this is one of the first studies in management that utilizes audio data quantitatively.

ASK-RoBERTa: A pretraining model for aspect-based sentiment classification via sentiment knowledge mining

The main objective of aspect-based sentiment classification (ABSC) is to predict sentiment polarities of different aspects from sentences or documents. Recent research integrates sentiment terms into pretraining models whose accuracy impacts the ABSC performance. This paper introduces a sentiment knowledge-adaptive pretraining model (ASK-RoBERTa). A sentiment word dictionary is first built from general and field sentiment words. We develop a series of term and sentiment mining rules based on part-of-speech tagging and sentence dependency grammar. These mining rules consider word dependencies, compounding, and conjunctions. The pretraining model optimizes the mining rules to capture the dependency between aspects and sentiment words. Experimental results on multiple public benchmark datasets demonstrate the satisfactory performance of ASK-RoBERTa.

Linguistic features and psychological states: A machine-learning based approach.

Previous research mostly used simplistic measures and limited linguistic features (e.g., personal pronouns, absolutist words, and sentiment words) in a text to identify its author’s psychological states. In this study, we proposed using additional linguistic features, that is, sentiments polarities and emotions, to classify texts of various psychological states. A large dataset of forum posts including texts of anxiety, depression, suicide ideation, and normal states were experimented with machine-learning algorithms. The results showed that the proposed linguistic features with machine-learning algorithms, namely Support Vector Machine and Deep Learning achieved a high level of performance in the detection of psychological state. The study represents one of the first attempts that uses sentiment polarities and emotions to detect texts of psychological states, and the findings may contribute to our understanding of how accuracy may be enhanced in the detection of various psychological states. Significance and suggestions of the study are also offered.

Sentiment Analysis in British and American Literature Teaching under Formative Assessment and Machine Learning

In English education, British and American literature is a new type of course. The teaching of British and American literature has also undergone many reforms. In the practice of teaching reform, artificial intelligence- (AI-) assisted teaching such as machine learning (ML) has a long history. The performance is continuously improved by studying the mechanism of computer simulation of the human brain learning British and American literature. Then, computer intelligence can be realized. Based on this, this paper mainly discusses two aspects. One is the sentiment tendency analysis method based on the sentiment dictionary, and the other is the sentiment tendency analysis method based on ML. It mainly introduces the judgment of different emotional tendencies by the sentiment analysis model, which is an automatic review analysis and ensemble classification approach. The improvement of sentiment analysis improves the recognition range of text sentiment words in British and American literature teaching to optimize the process of text analysis. Its main feature is that the sentiment analysis of text directly acts on the tendency of words, with fine granularity and accurate analysis. Finally, it is concluded that the maximum value of the algorithm proposed here is 0.9, which has higher accuracy than the maximum value of 0.81 of other analysis models. The results indicate that the integrated classification model combining British and American literature teaching with the dimensions hidden Markov model has relatively reasonable text analysis and high sentiment classification accuracy. In terms of British and American literature teaching, using ML algorithms can effectively help teachers teach British and American literature through sentiment analysis.

Sentiment Analysis of Chinese E-Commerce Product Reviews Using ERNIE Word Embedding and Attention Mechanism

The development of e-commerce has ushered in a golden age. E-commerce product reviews are remarks initiated by online shopping users to evaluate the quality and service of the products they purchase; these reviews help consumers learn the reality of the product. The sentiment polarity of e-commerce product reviews is the best way to obtain customer feedback on products. Therefore, sentiment analysis of product reviews on e-commerce platforms is greatly significant. However, the challenges of sentiment analysis of Chinese e-commerce product reviews lie in dimension mapping, disambiguation of sentiment words, and polysemy of Chinese words. To solve the above problems, this paper proposes a sentiment analysis model ERNIE-BiLSTM-Att (EBLA). Here, the dynamic word vector generated using the Enhanced Representation through Knowledge Integration (ERNIE) word embedding model is input into the Bidirectional Long Short-term Memory (BiLSTM) to extract text features. Then, the Attention Mechanism (Att) is used to optimize the weight of the hidden layer. Finally, softmax is used as the output layer for sentiment classification. The experimental results on the JD.com Chinese e-commerce product review dataset show that the proposed model achieves more than 0.87 in precision, recall, and F1 values, which is superior to classic deep learning models proposed by other researchers; it has strong practicability in sentiment analysis of Chinese e-commerce product reviews.

The language of happiness in self-reported descriptions of happy moments: Words, concepts, and entities

This article attempts to study the language of happiness from a double perspective. First, the impact and relevance of sentiment words and expressions in self-reported descriptions of happiness are examined. Second, the sources of happiness that are mentioned in such descriptions are identified. A large sample of “happy moments” from the HappyDB corpus is processed employing advanced text analytics techniques. The sentiment analysis results reveal that positive lexical items have a limited role in the description of happy moments. For the second objective, unsupervised machine learning algorithms are used to extract and cluster keywords and manually label the resulting semantic classes. Results indicate that these classes, linguistically materialized in compact lexical families, accurately describe the sources of happiness, a result that is reinforced by our named entities analysis, which also reveals the important role that commercial products and services play as a source of happiness. Thus, this study attempts to provide methodological underpinnings for the automatic processing of self-reported happy moments, and contributes to a better understanding of the linguistic expression of happiness, with interdisciplinary implications for fields such as affective content analysis, sentiment analysis, and cultural, social and behavioural studies.

Stepwise Multi-Task Learning Model for Holder Extraction in Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis is a text analysis technique that categorizes data by aspect and identifies the sentiment attributed to each one and a task for a fine-grained sentiment analysis. In order to accurately perform a fine-grained sentiment analysis, a sentiment word within a text, a target it modifies, and a holder who represents the sentiment word are required; however, they should be extracted in sequence because the sentiment word is an important clue for extracting the target, which is key evidence of the holder. Namely, the three types of information sequentially become an important clue. Therefore, in this paper, we propose a stepwise multi-task learning model for holder extraction with RoBERTa and Bi-LSTM. The tasks are sentiment word extraction, target extraction, and holder extraction. The proposed model was trained and evaluated under Laptop and Restaurant datasets in SemEval 2014 through 2016. We have observed that the performance of the proposed model was improved by using stepwised features that are the output of the previous task. Furthermore, the generalization effect has been observed by making the final output format of the model a BIO tagging scheme. This can avoid overfitting to a specific domain of the review text by outputting BIO tags instead of the words.

Analisis Sentimen Pengungkapan Informasi Manajemen: Text Mining Berbasis Metode VADER

Financial statements present information related to the firm’s financial condition. However, there is various types of data are provided to help us in assessing and understanding the firm’s business condition. The most abundant firm-related data that are available are in the form of text. It can include annual reports, official websites, or even social media posts that may contain non-financial data. Nonfinancial information is also important to help the interpretation of financial information. The research aims to analyze the sentiment of management discussion and analysis included in the manufacturing company's annual report. The research sample is manufacturing companies listed on the Indonesia Stock Exchange (IDX) during the 2016-2020 period in a row. There are 102 companies or 510 research observations include in this research. This research uses a sentiment analysis technique based on a lexicon-based approach using the VADER method. Meanwhile, the sentiment analysis process will be assisted by the Orange Data Mining application. The results of the research show that positive disclosure is greater than negative sentiments. The pattern of sentiment for companies that are classified based on sector, company size (total assets and total sales), profitability (ROI), and liquidity (current ratio) shows relatively the same results. These results illustrate that company characteristics do not make a difference in the choice of word sentiment in manufacturing companies.

Lexicon-Based Sentiment Convolutional Neural Networks for Online Review Analysis

With the growing availability and popularity of sentiment-rich resources like blogs and online reviews, new opportunities and challenges have emerged regarding the identification, extraction, and organization of sentiments from user-generated documents or sentences. Recently, many studies have exploited lexicon-based methods or supervised learning algorithms to conduct sentiment analysis tasks separately; however, the former approaches ignore contextual information of sentences and the latter ones do not take sentiment information embedded in sentiment words into consideration. To tackle these limitations, we propose a new model named Sentiment Convolutional Neural Network (SentiCNN) to analyze the sentiments of sentences with both contextual and sentiment information of sentiment words, in which, contextual information is captured from word embeddings and sentiment information is identified using existing lexicons. We incorporate a Highway Network into our model to adaptively combine sentiment and contextual information from sentences by strengthening the connection between features of both sentences and their sentiment words. Furthermore, we propose three lexicon-based attention mechanisms (LBAMs) for our SentiCNN model to find the most important indicators of sentiments and make predictions more effectively. Experiments over two well-known datasets indicate that sentiment words, the Highway Network, and LBAMs contribute to sentiment analysis.