Sentiment Analysis Datasets Research Articles

MACSA: A multimodal aspect-category sentiment analysis dataset with multimodal fine-grained aligned annotations

MACSA: A multimodal aspect-category sentiment analysis dataset with multimodal fine-grained aligned annotations

Optimizer algorithms and convolutional neural networks for text classification

<span lang="EN-US">Lately, deep learning has improved the algorithms and the architectures of several natural language processing (NLP) tasks. In spite of that, the performance of any deep learning model is widely impacted by the used optimizer algorithm; which allows updating the model parameters, finding the optimal weights, and minimizing the value of the loss function. Thus, this paper proposes a new convolutional neural network (CNN) architecture for text classification (TC) and sentiment analysis and uses it with various optimizer algorithms in the literature. Actually, in NLP, and particularly for sentiment classification concerns, the need for more empirical experiments increases the probability of selecting the pertinent optimizer. Hence, we have evaluated various optimizers on three types of text review datasets: small, medium, and large. Thereby, we examined the optimizers regarding the data amount and we have implemented our CNN model on three different sentiment analysis datasets so as to binary label text reviews. The experimental results illustrate that the adaptive optimization algorithms Adam and root mean square propagation (RMSprop) have surpassed the other optimizers. Moreover, our best CNN model which employed the RMSprop optimizer has achieved 90.48% accuracy and surpassed the state-of-the-art CNN models for binary sentiment classification problems.</span>

Punctuation and lexicon aid representation: A hybrid model for short text sentiment analysis on social media platform

Sentiment analysis measures user experience on social media. With the emergence of pre-trained models, text classification tasks have become homogeneous, without a significant improvement in accuracy. Therefore, we present a hybrid model called PLASA to classify the sentiment polarity of short comments, particularly barrages. PLASA introduces a collaborative attention module that integrates information about relative position and knowledge from summarized lexicons to better adjust the relationship between word representations. Our model is evaluated using three new curated sentiment analysis datasets: SentiTikTok-2023 (4613 items), SentiBilibili-2023 (7755 items), and SentiWeibo-2023 (5614 items). Although the comment length varies across datasets, all maintain a consistent punctuation percentage at approximately 13%. Consequently, PLASA with the optimal combination demonstrates notable performance improvements compared to both the baseline and commonly used models. It achieves micro-F1 scores of 93.94%, 90.34%, and 88.79% on the respective datasets. We also observed that the representation capacity of the pre-trained model decreases as the text length increases. Moreover, the proposed collaborative attention module effectively addresses this limitation, as confirmed by our ablation study.

Sentiment Analysis on a Large Indonesian Product Review Dataset

Background: The publicly available large dataset plays an important role in the development of the natural language processing/computational linguistic research field. However, up to now, there are only a few large Indonesian language datasets accessible for research purposes, including sentiment analysis datasets, where sentiment analysis is considered the most popular task. Objective: The objective of this work is to present sentiment analysis on a large Indonesian product review dataset, employing various features and methods. Two tasks have been implemented: classifying reviews into three classes (positive, negative, neutral), and predicting ratings. Methods: Sentiment analysis was conducted on the FDReview dataset, comprising over 700,000 reviews. The analysis treated sentiment as a classification problem, employing the following methods: Multinomial Naïve Bayes (MNB), Support Vector Machine (SVM), LSTM, and BiLSTM. Result: The experimental results indicate that in the comparison of performance using conventional methods, MNB outperformed SVM in rating prediction, whereas SVM exhibited better performance in the review classification task. Additionally, the results demonstrate that the BiLSTM method outperformed all other methods in both tasks. Furthermore, this study includes experiments conducted on balanced and unbalanced small-sized sample datasets. Conclusion: Analysis of the experimental results revealed that the deep learning-based method performed better only in the large dataset setting. Results from the small balanced dataset indicate that conventional machine learning methods exhibit competitive performance compared to deep learning approaches. Keywords: Indonesian review dataset, Large dataset, Rating prediction, Sentiment analysis

Combination of Chinese sentiment analysis datasets based on BiLSTM+Attention model

By training the Chinese sentiment analysis model, it is found that the prediction accuracy of the model trained by one dataset is obviously low on other datasets. Considering that the existing sentiment analysis work mainly uses a single domain corpus dataset and referring to the existing data processing methods on natural language processing, this paper designs an experiment to combine Chinese datasets from different fields into a large field-imbalanced dataset, and the number of samples from different fields in this dataset is obviously different. The new dataset is used to train a comprehensive Chinese sentiment analysis model and achieves satisfactory training results. According to the results of the experiments, the model trained by the field-imbalanced dataset has high prediction accuracy for samples from various fields, and the prediction accuracy increases with the increase of the proportion of corpus in this field in the training dataset. Through the experiment in this paper, some ideas are provided for the construction of large-scale cross-domain Chinese sentiment analysis datasets in the future.

Deep learning for sentiment analysis on IMDB movie reviews using N-gram features

In the rapidly evolving digital landscape, the synergy of deep learning techniques and abundant datasets has opened new frontiers in various domains. This research delves into the film industry, specifically harnessing the potential of the International Movie DataBase (IMDB) dataset for sentiment analysis. Through a deep learning paradigm, we embark on sentiment classification of movie reviews, discerning between positive and negative sentiments. By navigating data preprocessing and N-gram feature extraction, we engineer a deep learning model comprising embedding, global average pooling, and multi-layer dense architectures. The experimental results underscore the model's prowess in sentiment analysis, emphasizing its capacity to empower informed decision-making within the film industry.

Development and Evaluation of Gold Standard Dataset for Sentiment Analysis of Tweets

Pre-labeled data is typically required for supervised machine learning. A limited number of object classes in the majority of open access and pre-annotated datasets make them unsuitable for certain tasks, even though they are readily available for training machine learning algorithms. For custom models, previously available pre-annotated data is typically insufficient, so gathering and preparing training data is necessary for the majority of real-world applications. The quantity and quality of annotations clearly trade-off with one another. Either more annotated data can be produced or better data quality can be guaranteed by allocating time and resources. Development of the gold standard by annotating textual information is an essential part of the “Text Analytics” domain in the field of “Natural Language Processing-NLP”. In “Text Analytics”, annotation can be done by adopting a manual, semi-automatic or automatic approach. In the case of the manual approach, annotators often work with partial parts of the corpus, and the results are generalized by automated text classification which may affect the final classification results. Annotations reliability and suitability of assigned labels are particularly important in the NLP applications related to opinion mining or sentiment analysis. In this research study, we have evaluated the significance of the annotation process on a novel dataset that contained multiple languages (English, Roman Urdu), a free text dataset that was extracted from Twitter. This unique dataset contained multiple languages which makes this annotation process essential for researching this data. Using this multi-language dataset, we examine the inter-annotator agreement in multiclass and multi-label sentiment annotation. To scrutinize the reliability of this research work, several annotation agreement metrics, statistical analysis, and Machine Learning methods have been considered to evaluate the accuracy of resulting annotations. It was observed that the annotation process is significant and a complex step that is essential for the proper implementation of Natural Language processing tasks for text analytics in machine learning. During this research, different gaps were identified and resolved which can impact the overall reliability of the annotation process which are reported in this paper. We conclude that while inaccurate annotations worsen the results, the impact is minimal, at least when using text data. The advantages of the larger annotated data set (obtained by employing subpar auto-annotation techniques) surpass the degradation resulting from the use of annotated data.

Sentiment Analysis Using Deep Learning Approaches on Multi-Domain Dataset in Telugu Language

Recent advancements in Natural Language Processing (NLP) have made sentiment analysis an essential component of a variety of NLP jobs, including recommendation systems, question answering, and business intelligence products. While sentiment analysis research has, to put it mildly, been widely pursued in English, Telugu has barely ever attempted the task. The majority of research works concentrate on analysing the sentiments of Tweets, news, or reviews containing Hindi and English words. There is a growing interest among academics in studying how people express their thoughts and views in Indian languages like Bengali, Telugu, Malayalam, Tamil and so on. Due to a paucity of labelled datasets, microscopic investigation on Indian languages has been published to our knowledge. This work suggested a sentence-level sentiment analysis on multi-domain datasets that has been collected in Telugu. Deep learning models have been used in this work because it demonstrates the significant expertise in sentiment analysis and is widely regarded as the cutting-edge model in Telugu Sentiment Analysis. Our proposed work investigates a productive Bidirectional Long Short-Term Memory (BiLSTM) Network and Bidirectional GRU Network (BiGRU) for improving Telugu Sentiment Analysis by encapsulating contextual information from Telugu feature sequences using Forward-Backward encapsulation. Further, the model has been deployed by merging the domains so as to predict the accuracy and other performance metrics. The experimental test findings show that the deep learning models outperform when compared with the baseline traditional ML methods in four benchmark sentiment analysis datasets. There is evidence that the proposed sentiment analysis method has improved precision, recall, F1-score and accuracy in certain cases. The proposed model has achieved the F1-score of 86% for song datasets when compared with the other existing models.

FMSA-SC: A Fine-Grained Multimodal Sentiment Analysis Dataset Based on Stock Comment Videos

FMSA-SC: A Fine-Grained Multimodal Sentiment Analysis Dataset Based on Stock Comment Videos

An Efficient Sentiment Classification Method with the Help of Neighbors and a Hybrid of RNN Models

Some texts that are challenging to recognize on their own may become more understandable in a neighborhood of related texts with similar contexts. Motivated by this intuition, a novel deep text sentiment classification (DTSC) model is proposed to improve the model’s performance by incorporating the neighborhood of related texts. Our framework uses a nonparametric approach to construct neighborhoods of related texts based on Jaccard similarities. Then, a new deep recurrent neural network architecture is proposed, comprising two distinct modules: bidirectional long short-term memory (Bi-LSTM) and gated recurrent unit (GRU). The proposed model aims to effectively capture informative features from the input text and its neighbors. The result of each module is processed through the maximum operation, which selects the most pertinent data. Finally, the extracted features are concatenated and subjected to classification to achieve accurate sentiment prediction. Previous studies have commonly employed a parametric approach to represent textual metadata. However, our approach utilizes a nonparametric approach, enabling our model to perform strongly even when the text vocabulary varies between training and testing. The proposed DTSC model has been evaluated on five real-world sentiment datasets, achieving 99.60% accuracy on the Binary_Getty (BG) dataset, 98.32% accuracy on the Binary_iStock (BIS) dataset, 96.13% accuracy on Twitter, 82.19% accuracy on the multi-view sentiment analysis (MVSA) dataset, and 87.60% accuracy on the IMDB dataset. These findings demonstrate that the proposed model outperforms established baseline techniques in terms of model evaluation criteria for text sentiment classification.

Mining User Opinions: A Balanced Bangla Sentiment Analysis Dataset for E-Commerce

The utilization of sentiment analysis has gained significant importance as a valuable method for obtaining meaningful insights from textual data. The research progress in languages such as English and Chinese has been notable. However, there is a noticeable dearth of attention towards creating tools for sentiment analysis in the Bangla language. Currently, datasets are limited for Bangla sentiment analysis, especially balanced datasets capturing both binary and multiclass sentiment for e-commerce applications. This paper introduces a new sentiment analysis dataset from the popular Bangladeshi e-commerce site “Daraz”. The dataset contains 1000 reviews across 5 product categories, with both binary (positive/negative) and multiclass (very positive, positive, negative, very negative) sentiment labels manually annotated by native Bangla speakers. Reviews were collected using an organized process, and labels were assigned based on standardized criteria to ensure accuracy. In addition, a benchmark evaluation of the performance achieved by Machine Learning and Deep Learning algorithms on this dataset is also provided. The new dataset can aid research on multiclass and binary Bangla sentiment analysis utilizing both machine learning, deep learning, and Large Language Models. It can aid e-commerce platforms in analysing nuanced user opinions and emotions from online reviews. The utilization of categorized product reviews also facilitates research in the field of text categorization.

Advancement in Bangla Sentiment Analysis: A Comparative Study of Transformer-Based and Transfer Learning Models for E-commerce Sentiment Classification

Background: As a direct result of the Internet's expansion, the quantity of information shared by Internet users across its numerous platforms has increased. Sentiment analysis functions at a higher level when there are more available perspectives and opinions. However, the lack of labeled data significantly complicates sentiment analysis utilizing Bangla natural language processing (NLP). In recent years, nevertheless, due to the development of more effective deep learning models, Bangla sentiment analysis has improved significantly. Objective: This article presents a curated dataset for Bangla e-commerce sentiment analysis obtained solely from the "Daraz" platform. We aim to conduct sentiment analysis in Bangla for binary and understudied multiclass classification tasks. Methods: Transfer learning (LSTM, GRU) and Transformers (Bangla-BERT) approaches are compared for their effectiveness on our dataset. To enhance the overall performance of the models, we fine-tuned them. Results: The accuracy of Bangla-BERT was highest for both binary and multiclass sentiment classification tasks, with 94.5% accuracy for binary classification and 88.78% accuracy for multiclass sentiment classification. Conclusion: Our proposed method performs noticeably better classifying multiclass sentiments in Bangla than previous deep learning techniques. Keywords: Bangla-BERT, Deep Learning, E-commerce, NLP, Sentiment Analysis

A hybrid multi-source data fusion for word, sentence, aspect, and document-level sentiment analysis on real-time databases

Sentiment analysis (SA)at the sentence, aspect, and document levels determines the sentiment of particular aspect phrases in a given sentence. Due to their capacity to extract sentiment information from text in aspect-level sentiment classification, neural networks (NNs) have achieved significant success. Generally speaking, sufficiently sizable training corpora are necessary for NNs to be effective. The performance of NN-based systems is reduced by the small size of the aspect-level corpora currently available. In this research, we suggest a gated bilateral recurrent neural network (G-Bi-RNN) as a foundation for multi-source data fusion, their system offers sentiment information that several sources. We develop a uniform architecture specifically to include information from sentimental lexicons, including aspect- and sentence-level corpora. To further provide aspect-specific phrase representations for SA, we use G-Bi-RNN, a deep bilateral Transformer-based pre-trained language model. We assess our methods using SemEval 2014 datasets for laptops and restaurants. According to experimental findings, our method consistently outperforms cutting-edge techniques on all datasets. We use a number of well-known aspect-level SA datasets to assess the efficacy of our model. Experiments show that when compared to baseline models, the suggested model can produce state-of-the-art results.

Multi-Modal Representation via Contrastive Learning with Attention Bottleneck Fusion and Attentive Statistics Features.

The integration of information from multiple modalities is a highly active area of research. Previous techniques have predominantly focused on fusing shallow features or high-level representations generated by deep unimodal networks, which only capture a subset of the hierarchical relationships across modalities. However, previous methods are often limited to exploiting the fine-grained statistical features inherent in multimodal data. This paper proposes an approach that densely integrates representations by computing image features' means and standard deviations. The global statistics of features afford a holistic perspective, capturing the overarching distribution and trends inherent in the data, thereby facilitating enhanced comprehension and characterization of multimodal data. We also leverage a Transformer-based fusion encoder to effectively capture global variations in multimodal features. To further enhance the learning process, we incorporate a contrastive loss function that encourages the discovery of shared information across different modalities. To validate the effectiveness of our approach, we conduct experiments on three widely used multimodal sentiment analysis datasets. The results demonstrate the efficacy of our proposed method, achieving significant performance improvements compared to existing approaches.

FABSA: An aspect-based sentiment analysis dataset of user reviews

Aspect-based sentiment analysis (ABSA) aims at automatically extracting aspects of entities and classifying the polarity of each extracted aspect. The majority of available ABSA systems heavily rely on manually annotated datasets to train supervised machine learning models. However, the development of such manually curated datasets is a labour-intensive process and therefore existing ABSA datasets cover only a few domains and they are limited in size. In response, we present FABSA (Feedback ABSA), a new large-scale and multi-domain ABSA dataset of feedback reviews. FABSA consists of approximately 10,500 reviews which span across 10 domains. We conduct a number of experiments to evaluate the performance of state-of-the-art deep learning models when applied to the FABSA dataset. Our results demonstrate that ABSA models can generalise across different domains when trained on our FABSA dataset while the performance of the models is enhanced when using a larger training dataset. Our FABSA dataset is publicly available.11https://github.com/kontonag86/fabsa-dataset.

An aspect-level sentiment analysis dataset for therapies on Twitter.

The dataset described is an aspect-level sentiment analysis dataset for therapies, including medication, behavioral and other therapies, created by leveraging user-generated text from Twitter. The dataset was constructed by collecting Twitter posts using keywords associated with the therapies (often referred to as treatments). Subsequently, subsets of the collected posts were manually reviewed, and annotation guidelines were developed to categorize the posts as positive, negative, or neutral. The dataset contains a total of 5364 posts mentioning 32 therapies. These posts are further categorized manually into 998 (18.6%) positive, 619 (11.5%) negatives, and 3747 (69.9%) neutral sentiments. The inter-annotation agreement for the dataset was evaluated using Cohen's Kappa score, achieving an 0.82 score. The potential use of this dataset lies in the development of automatic systems that can detect users' sentiments toward therapies based on their posts. While there are other sentiment analysis datasets available, this is the first that encodes sentiments associated with specific therapies. Researchers and developers can utilize this dataset to train sentiment analysis models, natural language processing algorithms, or machine learning systems to accurately identify and analyze the sentiments expressed by consumers on social media platforms like Twitter.

A text guided multi-task learning network for multimodal sentiment analysis

Multimodal Sentiment Analysis (MSA) is an active area of research that leverages multimodal signals for affective understanding of user-generated videos. Existing research tends to develop sophisticated fusion techniques to fuse unimodal representations into multimodal representation and treat MSA as a single prediction task. However, we find that the text modality with the pre-trained model (BERT) learn more semantic information and dominates the training in multimodal models, inhibiting the learning of other modalities. Besides, the classification ability of each modality is also suppressed by single-task learning. In this paper, We propose a text guided multi-task learning network to enhance the semantic information of non-text modalities and improve the learning ability of unimodal networks. We conducted experiments on multimodal sentiment analysis datasets, CMU-MOSI, CMU-MOSEI, and CH-SIMS. The results show that our method outperforms the current SOTA method.

SentiUrdu-1M: A large-scale tweet dataset for Urdu text sentiment analysis using weakly supervised learning.

Low-resource languages are gaining much-needed attention with the advent of deep learning models and pre-trained word embedding. Though spoken by more than 230 million people worldwide, Urdu is one such low-resource language that has recently gained popularity online and is attracting a lot of attention and support from the research community. One challenge faced by such resource-constrained languages is the scarcity of publicly available large-scale datasets for conducting any meaningful study. In this paper, we address this challenge by collecting the first-ever large-scale Urdu Tweet Dataset for sentiment analysis and emotion recognition. The dataset consists of a staggering number of 1, 140, 821 tweets in the Urdu language. Obviously, manual labeling of such a large number of tweets would have been tedious, error-prone, and humanly impossible; therefore, the paper also proposes a weakly supervised approach to label tweets automatically. Emoticons used within the tweets, in addition to SentiWordNet, are utilized to propose a weakly supervised labeling approach to categorize extracted tweets into positive, negative, and neutral categories. Baseline deep learning models are implemented to compute the accuracy of three labeling approaches, i.e., VADER, TextBlob, and our proposed weakly supervised approach. Unlike the weakly supervised labeling approach, the VADER and TextBlob put most tweets as neutral and show a high correlation between the two. This is largely attributed to the fact that these models do not consider emoticons for assigning polarity.

Sentiment analysis of imbalanced datasets using BERT and ensemble stacking for deep learning

The Internet is a crucial way to share information in both personal and professional areas. Sentiment analysis attracts great interest in marketing, research, and business today. The instability faced by imbalanced datasets on sentiment analysis is examined in this research. Balancing the datasets using techniques based on under-sampling and over-sampling is examined to achieve more efficient classification results as the effects of using BERT as word embedding and ensemble learning methods for classification. The effects of the resampling training set algorithms on different deep learning classifiers were investigated using BERT as a word embedding model and Cohen's kappa, accuracy, ROC-AUC curve, and MCC as evaluation metrics with k-fold validation on three sentiment analysis datasets containing English, Arabic, and Moroccan Arabic Dialect texts. Also, we did those performance metrics for all models when scaling the dataset for training and testing, and we calculated the memory and the execution time for each model. Finally, we analyzed the National Office of Railways of Morocco (ONCF) customers' Facebook comments in Modern Standard Arabic (MSA) and MD to determine customer satisfaction as positive, negative, and neutral comments.

A comprehensive dataset for aspect-based sentiment analysis in evaluating teacher performance

Teacher performance evaluation is an essential task in the field of education. In recent years, aspect-based sentiment analysis (ABSA) has emerged as a promising technique for evaluating teaching performance by providing a more nuanced analysis of student evaluations. This article presents a novel approach for creating a large-scale dataset for ABSA of teacher performance evaluation. The dataset was constructed by collecting student feedback from American International University-Bangladesh and then labeled by undergraduate-level students into three sentiment classes: positive, negative, and neutral. The dataset was carefully cleaned and preprocessed to ensure data quality and consistency. The final dataset contains over 2,000,000 student feedback instances related to teacher performance, making it one of the largest datasets for ABSA of teacher performance evaluation. This dataset can be used to develop and evaluate ABSA models for teacher performance evaluation, ultimately leading to better feedback and improvement for educators. The results of this study demonstrate the usefulness and effectiveness of ABSA in evaluating teacher performance and highlight the importance of creating high-quality datasets for this task.