Visual Sentiment Research Articles

Sentiment Analysis of User Reviews on BSI Mobile and Action Mobile Applications on the Google Play Store Using Multinomial Naive Bayes Algorithm

Mobile banking services are designed to facilitate customer transactions. Bank Syariah Indonesia (BSI) and Bank Aceh also provide these online services through their respective applications, BSI Mobile and Action Mobile. The mobile banking apps aim to simplify customer transactions, which can be conducted remotely via several features, from transfers, payments, and purchases to zakat payments, by simply connecting to the internet. Therefore, this research aims to classify the sentiment of user reviews for BSI Mobile and Action Mobile applications on Google Play Store to understand the users' experiences. The Multinomial Naïve Bayes algorithm is used in this study, where the algorithm analyzes and classifies the user reviews into positive and negative sentiment categories. The study involves several stages, such as text preprocessing, sentiment visualization, splitting the data into an 80:20 ratio for training and testing datasets, and training the model using the Multinomial Naïve Bayes algorithm. The results of this study show that the Multinomial Naïve Bayes algorithm performs well in analyzing user sentiment for BSI Mobile and Action Mobile, achieving an accuracy of 78.7%, precision of 76.5%, recall of 86.2%, and an F1-score of 80.6% for BSI Mobile, and an accuracy of 85.6%, precision of 75%, recall of 75%, and an F1-score of 75% for Action Mobile. Additionally, the sentiment classification results reveal that 52.8% of BSI Mobile user reviews are positive and 47.2% are negative, while for Action Mobile, 35.1% are positive and 64.9% are negative. For BSI Mobile, 21,497 reviews express a positive sentiment with dominant keywords such as "updated," "good," "balance," "transaction," and "thank." Meanwhile, for Action Mobile, 274 reviews express a negative sentiment with dominant keywords such as "transaction," "application," "network," "register," "please," and "update."

An analysis of public topics and sentiments based on social media during the COVID-19 Omicron Variant outbreak in Shanghai 2022

The outbreak of the COVID-19 Omicron variant in Shanghai in 2022 elicited complex emotions among Shanghainese during the two-month quarantine period. This paper aims to identify prevailing public themes and sentiments by analyzing social media posts from Weibo. Initially, we conducted research based on a dataset of 90,000 Weibo posts during the 2022 COVID-19 outbreak in Shanghai. By examining social media data that mirrors residents' emotional shifts and areas of focus during unforeseen circumstances, we have developed an analytical framework combining hotspot analysis and public sentiment assessment. Subsequently, we employed the Latent Dirichlet Allocation (LDA) method to conduct topic modeling on the Weibo text data. The SnowNLP sentiment classification method was then utilized to quantify sentiment values. Ultimately, we performed spatial visualization of sentiment and concern data, categorizing them into distinct time periods based on Shanghai's infection curve. This approach allowed us to investigate concern focal points, sentiment trends, and their spatiotemporal evolution characteristics. Our findings indicate that variations in public sentiment primarily hinge on the severity of the epidemic's spread, emerging events, the availability of essential resources, and the government's ability to respond promptly and accurately. It is evident that, while residents' concerns shift over time, their primary objective on social media remains expressing demands and releasing emotions. This research offers an avenue for leveraging public opinion analysis to enhance governance capacity during crises, fortify urban resilience, and promote public involvement in governmental decision-making processes.

Towards visual sentiment summary to understand customers’ satisfaction

Due to the COVID-19 pandemic, the shopping behavior of customers has been significantly affected and is being shifted towards online shopping. Understanding the customers’ opinions, attitudes, and emotions in feedback and comments plays an essential role in making decisions for organizations and individuals (e.g., companies and customers). In this study, we propose sentiment summaries from the customer knowledgebase (SSoCK) framework that analyses customer feedback and improve a mechanism for sentiment summarization by using text analysis including sentiment analysis. In the experiments, various domains from customer reviews (e.g., computer and Canon) are used to conduct. The results show that the proposed SSoCK framework has the high performance of sentiment classification in terms of its accuracy when compared to the other approaches. Moreover, the proposed framework generates various kinds of sentiment summaries that can support managers/potential customers understand trending/interesting aspects of the product with customer satisfaction and can be easily updated with new reviews within the same domain without storing any original data.

GPT-4V with emotion: A zero-shot benchmark for Generalized Emotion Recognition

Recently, GPT-4 with Vision (GPT-4V) has demonstrated remarkable visual capabilities across various tasks, but its performance in emotion recognition has not been fully evaluated. To bridge this gap, we present the quantitative evaluation results of GPT-4V on 21 benchmark datasets covering 6 tasks: visual sentiment analysis, tweet sentiment analysis, micro-expression recognition, facial emotion recognition, dynamic facial emotion recognition, and multimodal emotion recognition. This paper collectively refers to these tasks as “Generalized Emotion Recognition (GER)”. Through experimental analysis, we observe that GPT-4V exhibits strong visual understanding capabilities in GER tasks. Meanwhile, GPT-4V shows the ability to integrate multimodal clues and exploit temporal information, which is also critical for emotion recognition. However, it is worth noting that GPT-4V is primarily designed for general domains and cannot recognize micro-expressions that require specialized knowledge. To the best of our knowledge, this paper provides the first quantitative assessment of GPT-4V for GER tasks. We have open-sourced the code and encourage subsequent researchers to broaden the evaluation scope by including more tasks and datasets. Our code and evaluation results are available at: https://github.com/zeroQiaoba/gpt4v-emotion.

An exploration system to effectively analyze the visual metaphor used in sentiment visualization

In the field of data visualization, there has been a recent trend of using a complex type of visualization with a multidimensional structure or using several visualizations in parallel when summarizing the results of sentiment analysis. Although this trend may be useful for sophisticated sentiment analysis, such analysis is difficult for the general public and novice researchers. To address this issue, there has recently been a trend of visualizing sentiments using visual metaphors. To facilitate the understanding of related cases, it is necessary to have a systematic means of grasping the sentiment target, the purpose and motivation of research, and the representations used as substitutes for visual metaphors. Therefore, the objective of the present study was to develop an exploration system that can analyze the visual metaphors used in the case of sentiment visualization. For this study, (1) sentiment visualization cases in which visual metaphors are used were collected. (2) After a taxonomy composed of the categories of “target, intermediation, representation, visual variable, and visualization technique” was constructed, it was used to analyze sentences of visual metaphors appearing in sentiment visualization cases. (3) An exploration system capable of grasping the semantic relationships of sub-elements within the five categories of the taxonomy and intuitively interpreting visual metaphors was developed so that appropriate cases can be recommended to sentiment visualization researchers. (4) The approach and usefulness of the exploration system were explained using user scenarios. (5) A case study was conducted to show that the provided system can be analyzed from various perspectives. (6) The usability of the exploration system was demonstrated through a verification targeting experts. The proposed system allows researchers and analysts to intuitively grasp “what types of visual metaphor method and idea should be equipped to visualize sentiment data in an easier way to understand.”

The Effect of Ad Image's Sentiment Scores and Mobile Device Attributes on Mobile Ad Response Behavior

Using a state-of-the-art image processing algorithm applied to mobile ad creative and a unique dataset on technology features for mobile platforms, we examine how visual sentiment of mobile ad images, mobile device attributes that aid visual communication, and their interactions influence consumers’ mobile ad responses. Our results show that an increase in mobile ads’ valence and arousal levels by one-unit enhances the instantaneous probability of consumers’ mobile ad response by 2.4% and 1.6%, respectively. An increase in mobile devices’ display size and pixel density by 1 in and 1%, respectively, decreases the probability of mobile ad response by 42.6% and 0.7%. Interestingly, display size positively moderates the effect of arousal, while pixel density positively moderates the effect of valence on consumers’ mobile ad responses. This study contributes to the IS and marketing literature by examining the role of ad images’ visual sentiment scores such as valence and arousal, device-specific attributes on mobile display, such as screen display size and pixel density, and their interaction effects on consumers’ mobile ad response behaviors. Importantly, the application of computationally efficient image sentiment processing sheds new light on the extant mobile advertising and marketing literature both from a methodological and findings perspective.

Sentiment Analysis: A Hybrid Approach on Twitter Data

In the present scenario, Internet communities, forum and blogging sites play a very crucial role to present opinions, views and the comments on various events. As the reachability of sites are beyond the control of national boundaries, sometimes this leads to the conviction and persuasion of thoughts without considering any legal subsequences, and also influence the belief of others as well, thus finding or identifying the sentiments of public from the social media content is one of the major research issue. Analysis of the sentiments of social media data is very difficult to understand as this typically does not exhibit a suspicious pattern in the flow of information like individual's actual opinion on a specific occasion etc. With the advancement of methodology and easy to use, the users of social media sites are now expressing their opinions, sharing their views and experiences through images, text, animation, audio, video etc. Because of the complexity the conventional text based sentiment analysis procedure, a lot of methods have been evolved, however studies suggest mostly more complicated procedure and approaches. Twitter® (or X), is one of the most common platform used widely by individual to express their opinions and sentiments on various events. Twitter sentiment analysis basically deals with the analysis of twitter quotes to find the hidden pattern in the sentiments expressed by the users in past. This paper aims to takes the challenges regarding social media sentiments analysis and developed a hybrid approach (Text and visual sentiment) on twitter data for sentiment analysis by using NLP-based opinion clustering, textual mining, emotion API and some machine learning techniques for visual ontology. Simulation result shows the significance of work.

Utilizing AI for extracting insights on post WHO's COVID-19 vaccination declaration from X (Twitter) social network

<abstract> <p>This study explores the use of artificial intelligence (AI) to analyze information from X (previously Twitter) feeds related to COVID-19, specifically focusing on the time following the World Health Organization's (WHO) vaccination announcement. This aspect of the pandemic has not been studied by other researchers focusing on vaccination news. By utilizing advanced AI algorithms, the research aims to examine a wealth of data, sentiments, and trends to enhance crisis management strategies effectively. Our methods involved collecting a dataset of tweets from December 2020 to July 2021. By using specific keywords strategically, we gathered a substantial 15.5 million tweets, focusing on important hashtags like #vaccine and #coronavirus while filtering out irrelevant replies and retweets. The assessment of three different machine learning models–BiLSTM, FFNN, and CNN – highlights the exceptional performance of BiLSTM, achieving an impressive F1-score of 0.84 on the test set, with Precision and Recall metrics at 0.85 and 0.83, respectively. The study provides a detailed visualization of global sentiments on COVID-19 topics, with a main goal of extracting insights to manage public health crises effectively. Sentiment labels were predicted using various classification models and categorized as positive, negative, and neutral for each country after adjusting for population differences. An important finding from the analysis is the variation in sentiments across regions, for instance, with Eastern European countries showing positive views on post-vaccination economic recovery, while China and the United States express negative opinions on the same topic.</p> </abstract>

Sentiment visualization of correlation of loneliness mapped through social intelligence analysis

BackgroundLoneliness is a global public health issue affecting a considerable number of people as well as burdening the public health system and increasing the risk of other life-threatening and life-damaging conditions. In USA an estimated 17% adults aged 18–70 report loneliness. The monetary loss as result of loneliness is estimated to be between USD 8074.80 and USD 12,0777.70 per person per year in the United Kingdom. But the dynamics of loneliness are not understood. Social media platforms have become a valuable source of data to study this phenomenon. ObjectivesThis paper aims to visualize the frequency of loneliness-related themes and topics in Twitter data. By using natural language (NLP) processing, sentiment analysis, and topic modeling, we seek to understand prevalent sentiments and concerns. Through interactive tree maps and radar plots, we present an engaging view of loneliness dimensions, allowing users to explore and gain insights into this issue on social media. We focus on comparative analysis of USA and India through analyzing tweets from both countries on loneliness. These two countries are the biggest countries population-wise where access to Twitter is legally allowed. MethodsThis study consists of two parts. In the first part, we employ NLP techniques and machine learning algorithms to extract and analyze tweets containing keywords related to loneliness. Through sentiment analysis and topic modeling, we discern linguistic patterns and contextual information to categorize the recurring themes and topics. Advanced text analytics is used to gain nuanced insights into the experiences, emotions, and challenges connected with loneliness. In the second part, interactive visualizations are developed to present the findings in an engaging and intuitive manner. Techniques such as tree maps and radar plots are utilized to transform the analyzed data into visually appealing representations. ResultsThe analysis of Twitter data yields valuable knowledge about the prevalence and nature of themes and topics associated with loneliness. The interactive visualizations present a comprehensive view of the sentiments and concerns expressed by Twitter users. These interactive plots provide a holistic view of the distribution of themes and topics associated with loneliness, allowing experts to explore and interact with the data, gaining deeper insights into the complexities surrounding this issue. ConclusionThis paper successfully explores themes and topics related to loneliness on Twitter by employing NLP, sentiment analysis, and topic modeling. The interactive visualizations enhance the accessibility and usability of the findings, providing valuable insights for various stakeholders. The study contributes to a deeper comprehension of loneliness in the context of social media.

Visual sentiment analysis with semantic correlation enhancement

Visual sentiment analysis is in great demand as it provides a computational method to recognize sentiment information in abundant visual contents from social media sites. Most of existing methods use CNNs to extract varying visual attributes for image sentiment prediction, but they failed to comprehensively consider the correlation among visual components, and are limited by the receptive field of convolutional layers as a result. In this work, we propose a visual semantic correlation network VSCNet, a Transformer-based visual sentiment prediction model. Precisely, global visual features are captured through an extended attention network stacked by a well-designed extended attention mechanism like Transformer. An off-the-shelf object query tool is used to determine the local candidates of potential affective regions, by which redundant and noisy visual proposals are filtered out. All candidates considered affective are embedded into a computable semantic space. Finally, a fusion strategy integrates semantic representations and visual features for sentiment analysis. Extensive experiments reveal that our method outperforms previous studies on 5 annotated public image sentiment datasets without any training tricks. More specifically, it achieves 1.8% higher accuracy on FI benchmark compared with other state-of-the-art methods.

EERCA-ViT: Enhanced Effective Region and Context-Aware Vision Transformers for image sentiment analysis

Different parts of an image have a strong or weak guiding effect on emotions. The key to emotion recognition of images is to fully exploit the regions associated with emotions. Therefore, this paper proposes a visual sentiment classification model with two branches based on visual transformer, termed as Enhanced Effective Region and Context-Aware Vision Transformers (EERCA-ViT). This model includes a primary branch and an auxiliary branch. The primary branch simulates interdependencies between patches by squeezing and stimulating patches (P-SE), thereby highlighting effective region features in the global feature. The auxiliary branch removes feature patches that have been tagged by the primary branch through the context-aware module (CAM), forcing the network to discover different sentiment discriminative regions. At the same time, a two-part loss function is constructed to improve the robustness of the model. Finally, extensive experiments on six benchmark datasets show that the proposed method outperforms the state-of-the-art image sentiment analysis methods. Furthermore, the effectiveness of the different modules of the framework (P-SE and CAM) has been well demonstrated through extensive comparative experiments.

BayesSentiRS: Bayesian sentiment analysis for addressing cold start and sparsity in ranking-based recommender systems

Recommendation systems are widely used to filter massive information. However, they often face the challenges of cold start and sparsity problems, limiting their effectiveness. Bayesian Personalized Ranking (BPR), which focuses on predicting the relative order of user items, has been conventionally proposed to address these challenges for ranking-based recommendation systems. However, conventional BPR approaches rely solely on implicit feedback and lack semantic and visual sentiment information. In this study, we propose a novel multi-modal BPR method that incorporates both semantic and visual sentiment to capture more nuanced user preferences and provide more accurate recommendations. Experimental evaluations show that the performance of our proposed method across seven metrics outperforms conventional BPR in both the review-rich and image-rich scenarios, indicating the potential and significance of considering sentiment for improving the performance of BPR recommendation systems.

Mixed Methods Framework for Understanding Visual Frames in Social Movements

ABSTRACTAttempting to understand visual frame perspectives in social movement posts online is important to develop an account of how social movements communicate and for what purpose. This paper builds a Mixed‐Methods Matrix framework that combines computational applications with visual methodologies to discover frames of meaning making in a large image collection. Frame analysis and Critical Visual Methodology are reviewed and used in the framework to work in tangent with quantitative research methods. The quantitative methods consist of network analysis applications and network structure analysis. Visual sentiment analysis is explored using methods of computer vision. The methods framework is presented in the form of a matrix that enables researchers to identify applications for looking at social movements online through theoretical and computational approaches. The broader implication for the framework is to help researchers understand how online image collections can show meaning through perspective.

Multimodal dual emotion with fusion of visual sentiment for rumor detection

Multimodal dual emotion with fusion of visual sentiment for rumor detection

Representing uncertainty through sentiment and stance visualizations: A survey

Visual analytics combines automated analysis techniques with interactive visualizations for effective understanding, reasoning, and decision-making on complex data. However, accurately classifying sentiments and stances in sentiment analysis remains challenging due to ambiguity and individual differences. This survey examines 35 papers published between 2016 and 2022, identifying unaddressed sources of friction that contribute to a gap between individual sentiment, processed data, and visual representation. We explore the impact of visualizations on data perception, analyze existing techniques, and investigate the many facets of uncertainty in sentiment and stance visualizations. We also discuss the evaluation methods used and present opportunities for future research. Our work addresses a gap in previous surveys by focusing on uncertainty and the visualization of sentiment and stance, providing valuable insights for researchers in graphical models, computational methods, and information visualization.

Feature analysis of fake news: improving fake news detection in social media

ABSTRACT Fake news is a threat to society, and its spread can have real-world consequences in many situations. For example, attackers have weaponised fake news to influence user opinion by causing users to emotionally react to fake news. On the other hand, fake news can also be a threat to national democracy. Therefore, we investigate textual sentiment, visual sentiment, behavioural and metadata features that entice users, using a dataset of posts from Reddit and another from Twitter, that were already categorised into the labels of fake news and real news. First, we extract features, such as visual sentiment, textual sentiment, behavioural reactions, and metadata, and then analyse various features for fake news prediction. We then run a machine learning experiment to classify posts that help improve fake news detection in social media.

PerceptSent - Exploring Subjectivity in a Novel Dataset for Visual Sentiment Analysis

Visual sentiment analysis is a challenging problem. Many datasets and approaches have been designed to foster breakthroughs in this trending research topic. However, most works scrutinize only subsymbolic models through visual attributes of the evaluated images, paying less attention to the subjectivity of viewers' perceptions as a basis for neuro-symbolic systems. Aiming to fill this gap, we present PerceptSent, a novel dataset for visual sentiment analysis that spans 5,000 images shared by users on social networks. Besides the sentiment opinion (positive, slightly positive, neutral, slightly negative, negative) expressed by every evaluator about each image analyzed, the dataset contains evaluator's metadata (age, gender, socioeconomic status, education, and psychological hints) as well as perceptions observed by the evaluator about the image — such as the presence of nature, violence, lack of maintenance, etc. Deep architectures and different problem formulations are explored using our dataset to combine visual and extra attributes (external knowledge) for automatic sentiment analysis. We show evidence that evaluator's perceptionss, when correctly employed, are crucial in visual sentiment analysis, improving the F-score performance from 61% to an impressive rate above 97%. Although, at this point, we do not have automatic approaches to capture these perceptions, our results open up new investigation avenues.

Multimodal sentiment analysis for social media contents during public emergencies

Abstract Purpose Nowadays, public opinions during public emergencies involve not only textual contents but also contain images. However, the existing works mainly focus on textual contents and they do not provide a satisfactory accuracy of sentiment analysis, lacking the combination of multimodal contents. In this paper, we propose to combine texts and images generated in the social media to perform sentiment analysis. Design/methodology/approach We propose a Deep Multimodal Fusion Model (DMFM), which combines textual and visual sentiment analysis. We first train word2vec model on a large-scale public emergency corpus to obtain semantic-rich word vectors as the input of textual sentiment analysis. BiLSTM is employed to generate encoded textual embeddings. To fully excavate visual information from images, a modified pretrained VGG16-based sentiment analysis network is used with the best-performed fine-tuning strategy. A multimodal fusion method is implemented to fuse textual and visual embeddings completely, producing predicted labels. Findings We performed extensive experiments on Weibo and Twitter public emergency datasets, to evaluate the performance of our proposed model. Experimental results demonstrate that the DMFM provides higher accuracy compared with baseline models. The introduction of images can boost the performance of sentiment analysis during public emergencies. Research limitations In the future, we will test our model in a wider dataset. We will also consider a better way to learn the multimodal fusion information. Practical implications We build an efficient multimodal sentiment analysis model for the social media contents during public emergencies. Originality/value We consider the images posted by online users during public emergencies on social platforms. The proposed method can present a novel scope for sentiment analysis during public emergencies and provide the decision support for the government when formulating policies in public emergencies.

Signatures of the uncanny valley effect in an artificial neural network

Robots and computer graphics characters that resemble humans but are not perfectly human-like tend to evoke negative feelings in human observers, which is known as the “uncanny valley effect.” In this study, we used a recent artificial neural network called Contrastive Language-Image Pre-training (CLIP) that learns visual concepts from natural language supervision as a visual sentiment model for humans to examine the semantic match between images with graded manipulation of human-likeness and words used in previous studies to describe the uncanny valley effect. Our results showed that CLIP estimated the matching of words of negative valence to be maximal at the midpoint of the transition from a human face to other objects, thereby indicating the signature of the uncanny valley effect. Our findings suggest that visual features characteristic to the conflicts of visual cues, particularly cues related to human faces, are associated with negative verbal expressions in our everyday experiences, and CLIP learned such an association from the training datasets. Our study is a step toward exploring how visual cues are related to human observers’ sentiment using a novel psychological platform, that is, an artificial neural network.

A Method of Sentiment Analysis and Visualized Interaction Based on Ernie-Tiny and BiGRU

Sentiment analysis of subjective texts in social media is beneficial to help people adjust and intervene in a negative mental state in time, which is significant to mental health care. At present, limited by the accuracy of word segmentation, sentiment analysis of subjective text has difficulties in dealing with context, sentence patterns, and word co-occurrence. This paper aims to propose an efficient method of semantic feature representation and sentiment analysis, thereby providing a basis for sentiment visualization and interactive applications. Based on Ernie-Tiny and BiGRU, this paper proposes a sentiment analysis model ET_s_BG+p to solve problems in analyzing Chinese subjective texts’ complex semantics, diverse sentence patterns, and shortness. The model inputs the semantic features obtained via Ernie-Tiny into BiGRU and then splices the output with the sentence vectors of Ernie-Tiny to form final text features and perform sentiment classification. Experiments are performed on a dataset integrating text comments from Weibo, takeaway, and e-commerce platforms. The results show that the model proposed in this paper performs best in most of the evaluation indicators compared with baseline models such as CNN, BiLSTM, and GRU. The experiments show that the accuracy of the model on the dataset built in this research is 84.30%, the precision is 83.95%, the recall rate is 88.35%, and the F1 value is 85.98%. At the same time, based on ET_s_BG+p, this paper develops a prototype visual display platform that integrates functions such as text input, sentiment analysis, and agent interaction, which can provide support for daily emotion monitoring and adjustment.