Sentiment Classification Results Research Articles

Augmenting Multimodal Content Representation with Transformers for Misinformation Detection

Information sharing on social media has become a common practice for people around the world. Since it is difficult to check user-generated content on social media, huge amounts of rumors and misinformation are being spread with authentic information. On the one hand, most of the social platforms identify rumors through manual fact-checking, which is very inefficient. On the other hand, with an emerging form of misinformation that contains inconsistent image–text pairs, it would be beneficial if we could compare the meaning of multimodal content within the same post for detecting image–text inconsistency. In this paper, we propose a novel approach to misinformation detection by multimodal feature fusion with transformers and credibility assessment with self-attention-based Bi-RNN networks. Firstly, captions are derived from images using an image captioning module to obtain their semantic descriptions. These are compared with surrounding text by fine-tuning transformers for consistency check in semantics. Then, to further aggregate sentiment features into text representation, we fine-tune a separate transformer for text sentiment classification, where the output is concatenated to augment text embeddings. Finally, Multi-Cell Bi-GRUs with self-attention are used to train the credibility assessment model for misinformation detection. From the experimental results on tweets, the best performance with an accuracy of 0.904 and an F1-score of 0.921 can be obtained when applying feature fusion of augmented embeddings with sentiment classification results. This shows the potential of the innovative way of applying transformers in our proposed approach to misinformation detection. Further investigation is needed to validate the performance on various types of multimodal discrepancies.

Sentiment Analysis Using Improved CT-BERT_CONVLayer Fusion Model for COVID-19 Vaccine Recommendation

COVID-19 has significantly impacted individuals, communities, and countries worldwide. These effects include health impacts, economics impacts, social impacts, educational, political and environmental impacts. The COVID-19 vaccine development was crucial for disease control and monitoring, yet the threat still looms large. Vaccine recommender systems can help the health practitioners in combating COVID-19 by providing the information and guidance on the benefits and risks of COVID-19 vaccines to individuals based on their preferences and medical history. In this paper, we have proposed sentiment analysis based recommender system for COVID-19 vaccines. We used Twitter data of 10,000 tweets about COVID-19 vaccines and applied pre-processing steps. We propose an ensemble of random forest with CT-BERT_CONVLayerFusion model, a novel algorithm, for classifying the tweets into seven different categories of sentiments. We also performed aspect-based review categorization which works on the queries given by a user. We compared the results of sentiment classification with the state-of-the-art with metrics including accuracy, recall, precision, and F1-score, and found out that our proposed approach outperformed all other state-of-the-art model by achieving maximum accuracy, recall, precision and F1-measure. Hence, such advanced methods can help somehow to fight COVID-19 as well as reducing the vaccine hesitancy by suggesting proper vaccines to patients based on the their specific concerns and questions.

A Sentiment Analysis Approach for Exploring Customer Reviews of Online Food Delivery Services: A Greek Case

The unprecedented production and sharing of data, opinions, and comments among people on social media and the Internet in general has highlighted sentiment analysis (SA) as a key machine learning approach in scientific and market research. Sentiment analysis can extract sentiments and opinions from user-generated text, providing useful evidence for new product decision-making and effective customer relationship management. However, there are concerns about existing standard sentiment analysis tools regarding the generation of inaccurate sentiment classification results. The objective of this paper is to determine the efficiency of off-the-shelf sentiment analysis APIs in recognizing low-resource languages, such as Greek. Specifically, we examined whether sentiment analysis performed on 300 online ordering customer reviews using the Meaning Cloud web-based tool produced meaningful results with high accuracy. According to the results of this study, we found low agreement between the web-based and the actual raters in the food delivery services related data. However, the low accuracy of the results highlights the need for specialized sentiment analysis tools capable of recognizing only one low-resource language. Finally, the results highlight the necessity of developing specialized lexicons tailored not only to a specific language but also to a particular field, such as a specific type of restaurant or shop.

Text Emotion Classification System Integrating Visual Communication and Deep Learning for Social Platform

With the development of modern information technology, social networks have become an im-portant platform for people to express, and at the same time, a large number of texts have been produced. However, the comment text has the characteristics of randomness and colloquialism in the way of expression, and also contains a lot of non-text data. Therefore, manually analyzing the emotional information in the text will consume a lot of time and the accuracy will be limited. To solve emotion classification, this study proposes a knowledge enhanced double loop emotion classification neural network model with attention mechanism. The study first preprocesses text data using a full sentence vector word vector model, then uses convolutional neural networks to recognize emotions in emoticons and emoticons in the text. Finally, the classification results are integrated using algorithms such as a dual loop sentiment classification neural network with pool-ing layers and attention mechanisms, K-nearest neighbors, and decision trees. The final compre-hensive expression recognition and text recognition results are used to obtain the text sentiment classification results. The experimental data shows that the model proposed in this study has an accuracy of 0.947 in the training set test, which is significantly better than other models. In da-tasets A and B, the accuracy of the research design model was 0.958 and 0.924, and the recall was 0.964 and 0.986, respectively. Compared to the baseline method or existing research models, the values of each indicator were significantly higher. The recall rate is the proportion of instances correctly identified as positive by the model to all actual positive instances, which can reflect the emotional classification performance of the research and design model. The higher the value, the better the performance of the model. In practical applications, the positive review rate of this model is above 0.9, which has obvious advantages compared to other models. This study utilizes deep learning techniques to classify sentiment in comment texts, providing reference for the field of text sentiment classification. In the e-commerce industry, it is possible to identify the emotions in user comments on products, further understand the product situation on the platform, and make targeted planning for product reserves, specifications, and so on.

Linking negative herding effect to online public opinion: A new BiGADG approach for sentiment classification

AbstractThe negative herding effect tends to cause the spread of negative emotions among netizens, resulting in significant public opinion crises. Therefore, linking the negative herding effect and online public opinion, we aim to exploring the connection between netizens' emotions, opinions, and the negative herding effect. To better distinguish the emotions in texts, a new BiGADG (Bidirectional Gated recurrent unit based on Attention with Distance‐based dependency syntax Graph convolutional network) model is proposed in this article. This model includes primarily the proposed dependency syntax based on dependency distance, Bidirectional Gated Recurrent Unit (BiGRU), multi‐head self‐attention and Graph Convolutional Network (GCN). Particularly, the dependency syntax based on dependency distances incorporates word‐to‐word dependency distances into the traditional dependency syntax, which enriches the syntactic structural features of the text. After the experiments, the accuracy of BiGADG model is 86.53%, which is 3.11% higher compared to BERT model. It is evident that our model for sentiment classification has been improved. Based on the results of sentiment classification, we investigate the existence of negative herding behaviour and its underlying motivation in online public opinion. We have found that the negative herding effect, if not effectively managed, may trigger large‐scale and uncontrollable public opinion crises.

DRGAT: Dual-relational graph attention networks for aspect-based sentiment classification

Aspect-based sentiment classification has become a popular topic in natural language processing. Exploiting dependency syntactic information with graph neural networks has recently become a popular trend. Despite their success, methods that rely heavily on a dependency tree face major challenges. This concerns the alignment of aspects and their word sentiments due to the richness of the language and the fact that a dependency tree might produce noisy signals from unrelated associations. This paper introduces a Dual-Relational Graph Attention Network (DRGAT) that fully exploits syntactic structural information and then the sentiment-aware context (e.g., phrase segmentation and hierarchical structure) of the constituent tree of a sentence. Additional constituency and dependency attention mechanisms provide comprehensive syntactic information across words, thereby enabling an accurate connection between aspect words and corresponding sentiment words. Considering that the original parsed constituency tree may have a large depth, this could lead to words being far apart increasing the computational overhead. The constituency tree of each sentence is dynamically reconstructed by determining the importance of each relational node. Extensive experimental results on six English datasets demonstrated that fully exploiting syntactic information can achieve excellent sentiment classification results.

Sentiment Analysis of DANA Application Reviews on Google Play Store Using Naïve Bayes Classifier Algorithm Based on Information Gain

DANA is a digital payment platform that provides various features to make it easier for users to make payments, transfers, and balance replenishment online. DANA application users provide a variety of reviews that include both constructive and critical opinions, which can be valuable input for DANA application developers. The purpose of this research is to evaluate the results of sentiment classification of DANA application user reviews on the Google Play Store service using the Naïve Bayes Classifier method and Information Gain feature selection. In addition, this study aims to assess the effect of applying IG feature selection on the performance of the resulting model. In this study, reviews are divided into two categories, namely positive and negative based on lexicon-based labeling. Furthermore, data weighting, feature selection, and data division are carried out with a proportion of 80% train data and 20% test data before model building. There are two models, namely a model without feature selection (NBC model) and a model with feature selection (NBC-IG model). The evaluation results showed that the NBC model with 1106 features performed well, with 82.91% accuracy, 83.96% precision, and 90.23% recall. Meanwhile, the NBC-IG model with 536 features showed higher performance, with 85.09% accuracy, 85.79% precision, and 92.09% recall. The application of IG feature selection with the IG value limit parameter > 0.01 in the NBC model successfully reduced the number of features by 570, and improved model performance with an increase in accuracy by 2.18%, precision by 1.83%, and recall by 1.86%.

Spillover Effect of Network Public Opinion on Market Prices of Small-Scale Agricultural Products

Network public opinion plays a crucial role in the behavior and decision making of various stakeholders, including farmers, middlemen, and consumers. It also affects the price fluctuations of small-scale agricultural products. Understanding the transmission path and spillover effect of network public opinion on the price fluctuations of these products is essential for ensuring their sustainable development and price stability. This paper selects the monthly data of network public opinion and related market prices of small-scale agricultural products from January 2014 to December 2021, constructs a network public opinion value through the sentiment classification results of deep learning models, and uses the trivariate VAR-BEKK-GARCH(1,1) model and spillover index model to study the spillover effect and spillover index of network public opinion on the market prices of small-scale agricultural products (national average price and origin price). The results show that: (1) There is a bidirectional volatility spillover effect between public opinion sentiment and the market prices of small-scale agricultural products. Additionally, this two-way volatility spillover effect is also evident between the average market prices and the origin prices of these commodities. (2) The influence of network public opinion on the market prices of small-scale agricultural products is substantial, with the spillover index being more pronounced for origin prices than for national average prices and reaching its zenith earlier. Consequently, based on these results, recommendations are provided to adapt planting and inventory strategies, enhance vigilance towards price risk transmission amongst small-scale agricultural product markets, and improve the comprehensive information platform encompassing the entire industry chain.

Design of cultural and creative experience products in scenic spots based on CORE model

Abstract Under the background of “cultural and tourism integration”, scenic cultural and creative experience products are an important means to enhance the communication power of scenic spots and generate income. In this paper, the CORE model is used as the design concept, and the CORE design process for scenic cultural and creative products is constructed. The plain Bayesian classification algorithm is used to extract review feedback from the product, and the vectorization of the extracted reviews is achieved using the FastText model. The vectorized reviews are input into the logistic regression classification model to obtain sentiment classification results. A fused sentiment feature vector is constructed by combining Apirior and a sentiment dictionary aimed at separating sentiment vector classification. Finally, the experience effect of cultural and creative products is explored as an example of X scenic spot, and the optimization strategy of product design is explored. The attribute feature of X Scenic Spot’s cultural and creative experience product, “color” embraces four emotional attributes, and the corresponding adaptations are 4.77, 5.42, 5.40, and 5.38, respectively. The SDs of the emotional attributes of the existing product are The SDs of the emotional attributes of the existing products are 1.141, 1.025, and 0.184, respectively, and the differences between the product descriptions and the users’ opinions on the perceptual attributes are large, which provides the evaluation correlation in the CORE model. Under the guidance of the CORE model, four aspects of cultural and creative marketing, scenic area IP, consumer needs and product experience dimensions are proposed.

Computational Intelligence-based Data Analytics for Sentiment Classification on Product Reviews

Abstract Computational Intelligence (CI) is a sub-field of Artificial Intelligence (AI) that concentrates on the development of intelligent techniques and models to solve complex problems. When it comes to data analytics, CI techniques can be highly effective in handling large volumes of data, extracting meaningful patterns, and making accurate predictions. An increasing number of online reviews were being posted on the Internet every day with the rapid growth of Electronic Commerce (E-Commerce) and social networks. In the fastest growing research area, Sentiment Analysis (SA) is one among them that helps consumers in making better decisions relating to purchases through proper analysis and understanding of shared sentiments from social media and the web. In recent times, several approaches were modelled for acquiring insights from such datasets. But still, there comes a problem in managing text of large size; hence, precise polarity recognition of consumer reviews was an exciting and ongoing issue. This article introduces a novel Sentiment Analysis on Product Reviews using Enhanced Grasshopper Optimization with Deep Learning (SAPR-EGODL) approach. The objective of the SAPR-EGODL approach lies in the identification and classification of different sentiment types that exist in product reviews. At the initial stage, data preprocessing takes place which transforms the product review data into meaningful data. Next, the SAPREGODL technique employs Multi-Head Attention-based Bidirectional Long Short Term (MHABLSTM) technique for sentiment classification. In this study, the EGO model is exploited for improving the classification accomplishment of the MHABLSTM technique. A sequence of simulations was accomplished on different datasets for examining the advanced sentiment classification results of the SAPR-EGODL technique. The comprehensive relative research exhibited the promising accomplishment of the SAPR-EGODL model compared to current techniques.

Impact of ByteDance crisis communication strategies on different social media users

The impact of corporate crisis communication strategies on users’ attitudes across different platforms has emerged as a new focal point in crisis management. This study is rooted in Social Media Crisis Communication (SMCC) and employs the case of Trump’s sanctions on TikTok to analyze variations in the effects of ByteDance’s crisis communication strategies on different social media platforms. We initally identified five announcements that generated significant discussions on Toutiao and Weibo and collected the corresponding user comments (a total of 50,702). Subsequently, we utilized two approaches, machine learning and deep learning, to conduct sentiment classification tests on the text to identify the best-performing model. This model was then applied on the entire dataset for sentiment classification, followed by semantic network analysis based on the sentiment classification results. The results demostrated that the pre-trained ERNIE model outperformed the other tested models (F1 = 82.40%). Following the fourth crisis communication event, users on Toutiao and Weibo exhibited contrasting sentimental tendencies. Theoretically, we observed that users on different social media platforms relying on distinct information sources, expressed different sentimental responses to the same crisis. Social media users have a tendency to anthropomorphize corporate personality traits. In practical terms, we recommend that companies engage in crisis communication on multiple social media platforms and do not overlook the most influential platforms in the market.

Sentiment Analysis Using PSEUDO Nearest Neighbor and TF-IDF TEXT Vectorizer

Twitter is one of the social media that is often used by researchers as an object of research to conduct sentiment analysis. Twitter is also a good indicator in influencing research, problems that often arise in research in the field of sentiment analysis are the many factors such as the use of colloquial or informal language and other factors that can affect sentiment results. To improve the results of sentiment classification, it is necessary to carry out a good information extraction process. One of the word weighting methods resulting from the information extraction process is the TF-IDF Vectorizer. This study examines the effect of the TF-IDF Vectorizer weighting results in sentiment analysis using the Pseudo Nearest Neighbor method. The results of the f-measure classification of sentiment using the TF-IDF Vectorizer at parameters k-2 = 89%, k-3 = 89%, k-4 = 71% and k-5 = 75% while without using the TF-IDF Vectorizer on the parameters k-2 = 90%, k-3 = 92%, k-4 = 84% and k-5 = 89%. From the results of the classification of sentiment analysis that does not use the TF-IDF Vectorizer, the f-measure value is slightly better than using it.

Pelican Optimization Algorithm with Deep Learning for Aspect based Sentiment Analysis on Asian Low Resource Languages

Recently, sentiment analysis (SA) has become more popular as it is crucial to moderate and examine the data from the internet. It contains several applications, such as social media monitoring, market research, and opinion mining. Aspect Based Sentiment Analysis (ABSA) is a domain of SA that manages sentiment at a better level. ABSA classifies sentiment in terms of all the aspects for obtaining superior insights as sentiment expressed. A major contribution has been developed in ABSA, and then this progress can be restricted only to some languages with suitable resources. One common method is to utilize machine learning (ML) approaches, namely Neural Networks (NN), Support Vector Machines (SVM), and Naive Bayes (NB), together with Asian and low language-specific resources. These resources offer data on the sentiment polarity (neutral, positive, or negative) of phrases and words that are generally utilized in low-resource languages. In this aspect, this study develops a new Pelican Optimization Algorithm with Deep Learning for ABSA (POADL-ABSA) on Asian and Low Resource Languages. The proposed POADL-ABSA technique focuses on the detection and classification of sentiments. To accomplish this, the POADL-ABSA technique encompasses various levels of operations such as pre-processed, feature vector conversion, and classification. In addition, the POADL-ABSA technique employs the BERT model for feature vector extraction. Besides, attention-based bi-directional long short-term memory (ABiLSTM) system was used for the recognition and classification of sentiments. Finally, the POA was utilized for optimum hyperparameter selection of the ABiLSTM model, and it helps in attaining enhanced sentiment classification results. To ensure the improvised performance of the IAOADL-ABSA technique, an extensive experimental outcome the IAOADL-ABSA technique surpassed other models with acc{u}_y , pre{c}_n , rec{a}_l , and {F}_{score} of 98.72%, 98.71%, 98.72%, and 98.71%, respectively. Therefore, the IAOADL-ABSA technique can be employed for accurate classification results.

Sentiment Analysis of the Consumer Review Text Based on BERT-BiLSTM in a Social Media Environment

In this paper, a BERT-BiLSTM-based consumer review text sentiment analysis method in the e-commerce big data field is proposed. First, the unlabeled text is trained using the BERT training model for the language introduced in the deep learning, and then the pre-training model of the text data is delivered by the learning textual features and data to extract deeper vectors. Second, the BiLSTM model is applied to simultaneously obtain contextual information so as to illustrate optimal textual features. Finally, a corresponding sentiment analysis model relative to the consumer review text is constructed by combining the BERT model with BiLSTM to better merge the context for classifying sentiment and improving the final feature vector accuracy for the sentiment classification results. Simulated by experiments, the method proposed in this paper was compared with another three methods using the same data set. The results obtained indicate that the proposed method has the highest precision, recall, and F1-Measure, and the values reach 92.64%, 90.32%, and 91.46%, respectively.

SENTIMENT ANALYSIS OF MERDEKA BELAJAR KAMPUS MERDEKA POLICY USING SUPPORT VECTOR MACHINE WITH WORD2VEC

Sentiment analysis is a data text analysis that classifies data into positive and negative sentiments. This study aims to obtain the results of sentiment classification related to Merdeka Belajar Kampus Merdeka policy on Twitter using support vector machine algorithm with Word2Vec feature extraction. Support Vector Machine is a classification algorithm that separates data classes using the optimum hyperplane. Text data used in sentiment analysis must change its numerical form by performing feature extraction. In this study, the feature extraction used is Word2Vec which represents words in vector form. Data in this study are tweets with the keyword "Kampus Merdeka" uploaded on Twitter as many as 10000 tweets. After preprocessing text data, data used to analyze sentiment was 1579 tweets. Sentiment classification resulted in classification model accuracy 89.87%, precision 91.20%, recall 84.44% and F-Measure 87.68%. Classification sentiment using support vector machine with Word2Vec feature extraction in this study produces a good model.

Structure Graph Refined Information Propagate Network for Aspect-Based Sentiment Analysis

The main task of aspect-based sentiment analysis is to determine the sentiment polarity of a given aspect in the sentence. A major issue lies in identifying the aspect sentiment is to establish the relationship between the aspect and its opinion words. The application of syntactic dependency trees is one such resolution. However, the widely-used dependency parsers still have challenges in obtaining a solid sentiment classification result. In this work, an information propagation graph convolutional network based on syntactic structure optimization is proposed on the task of ABSA. To further complement the syntactic information, the semantic information is incorporated to learn the representations using graph information propagation mechanism. In addition, the effects of syntactic and semantic information are adapted via feature separation. Experimental results on three benchmark datasets show that the proposed model achieves satisfying performance against the state-of-the-art methods, indicating that the model can precisely build the relation between aspect and its context words.

Klasifikasi Sentimen Transformasi dan Reformasi Sepak Bola Indonesia Pada Twitter Menggunakan Algoritma Bernoulli Naïve Bayes

Federation Internationale de Football Association (FIFA) carried out Transformations and Reformations to Indonesian Football with one of them Indonesia was chosen as the Host of the U-20 World Cup in 2023. The transformations and reformations carried out cause people to often provide opinions through social media Twitter. Opinions given by the public can be positive or negative. The research uses Text Mining to classify sentiment in 2 categories with the Bernoulli Naïve Bayes algorithm. This research aims to classify positive and negative sentiments and determine the level of accuracy value of the sentiment classification results of Indonesian Football Transformation and Reformation. The research stages carried out are data collection, text preprocessing, data labeling, TF-IDF weighting, Bernoulli Naïve Bayes classification, and evaluation. Based on the research results from 4907 data there is duplicate data and only uses 2125 data which is divided into 90% training data and 10% testing data, so as to get accuracy with a high category value of 88%. The classification results show that many tweets are positive sentiments.

Effect of daily new cases of COVID-19 on public sentiment and concern: Deep learning-based sentiment classification and semantic network analysis.

This study explored the influence of daily new case videos posted by public health agencies (PHAs) on TikTok in the context of COVID-19 normalization, as well as public sentiment and concerns. Five different stages were used, based on the Crisis and Emergency Risk Communication model, amidst the 2022 Shanghai lockdown. After dividing the duration of the 2022 Shanghai lockdown into stages, we crawled all the user comments of videos posted by Healthy China on TikTok with the theme of daily new cases based on these five stages. Third, we constructed the pre-training model, ERNIE, to classify the sentiment of user comments. Finally, we performed semantic network analyses based on the sentiment classification results. First, the high cost of fighting the epidemic during the 2022 Shanghai lockdown was why ordinary people were reluctant to cooperate with the anti-epidemic policy in the pre-crisis stage. Second, Shanghai unilaterally revised the definition of asymptomatic patients led to an escalation of risk levels and control conditions in other regions, ultimately affecting the lives and work of ordinary people in the area during the initial event stage. Third, the public reported specific details that affected their lives due to the long-term resistance to the epidemic in the maintenance stage. Fourth, the public became bored with videos regarding daily new cases in the resolution stage. Finally, the main reason for the negative public sentiment was that the local government did not follow the central government's anti-epidemic policy. Our results suggest that the methodology used in this study is feasible. Furthermore, our findings will help the Chinese government or PHAs improve the possible behaviors that displease the public in the anti-epidemic process.

SENTIMEN PERCAKAPAN PENGGUNA TWITTER PADA HASHTAG #NONHALAL DALAM TIPOLOGI EKSKLUSIVISME, INKLUSIVISME, PLURALISME DAN TOLERANSI BERAGAMA

This study was conducted to analyze the negative sentiment of the impact of news coverage at a Padang restaurant in Jakarta with rendang innovation made from pork. This is unprincipled from the origins of the cultural philosophy, namely Adat Basandi Syara', Syara' Basandi Kitabullah associated with the principle of halal products as one of the characteristics of Padang food. However, it turns out that there are many public responses that view this as not reflecting tolerance in culture and religion, one of the social media used in analyzing these opinions and responses is the social media Twitter. This study uses a mix method approach with the data collection technique used is conversations (tweets) from the Twitter social media application with a special topic approach using the 'hashtag' feature. Data collection was carried out using the Twitter API (Application Programming Interface) tool. The text and sentiment analysis is carried out using several tools and stages of data analysis, including data mining assistance, namely using python with stages covering the process of crawling data, text preprocessing, language adjustment, and the results of sentiment classification (labelling) technically using Vader-TDIF. The results of these sentiments by researchers are then transformed into Alan Race's theory. This aims to apply, translate, and perceive in a simpler way social-psychological perceptions. Alan Race's typology which includes Exclusivism, Inclusivism, Pluralism is the basis for categorizing the sentiments of Twitter users in interpreting a popular issue. The sentiment results formed from the prediction model with the Naive-Bayes algorithm show the result that the percentage of dominant sentiment is neutral or represents pluralism with the highest yield, namely 47.9%. , negative sentiment represents exclusivism, namely 21.23%, and positive sentiment or inclusivism, namely 30.87% with an accuracy rate of 66.43%. By knowing the conditions of community sentiment and views, this research is expected to become a reference in decision making and increase the intensity of relevant campaigns on social media Twitter specifically in the context of religious and cultural differences in order to further enhance the understanding of tolerant religious moderation in a plural-multicultural society.

They May Not Work! An evaluation of eleven sentiment analysis tools on seven social media datasets

ObjectiveSentiment analysis is an important method for understanding emotions and opinions expressed through social media exchanges. Little work has been done to evaluate the performance of existing sentiment analysis tools on social media datasets, particularly those related to health, healthcare, or public health. This study aims to address the gap. Material and methodsWe evaluated 11 commonly used sentiment analysis tools on five health-related social media datasets curated in previously published studies. These datasets include Human Papillomavirus Vaccine, Health Care Reform, COVID-19 Masking, Vitals.com Physician Reviews, and the Breast Cancer Forum from MedHelp.org. For comparison, we also analyzed two non-health datasets based on movie reviews and generic tweets. We conducted a qualitative error analysis on the social media posts that were incorrectly classified by all tools. ResultsThe existing sentiment analysis tools performed poorly with an average weighted F1 score below 0.6. The inter-tool agreement was also low; the average Fleiss Kappa score is 0.066. The qualitative error analysis identified two major causes for misclassification: (1) correct sentiment but on wrong subject(s) and (2) failure to properly interpret inexplicit/indirect sentiment expressions. Discussion and conclusionThe performance of the existing sentiment analysis tools is insufficient to generate accurate sentiment classification results. The low inter-tool agreement suggests that the conclusion of a study could be entirely driven by the idiosyncrasies of the tool selected, rather than by the data. This is very concerning especially if the results may be used to inform important policy decisions such as mask or vaccination mandates.