Bidirectional Encoder Representations From Transformers Research Articles

Cascaded Adaptive Dilated Temporal Convolution Network-Based Efficient Sentiment Analysis Model from Social Media Posts

Sentimental analysis is one of the most complicated tasks in text mining. It is the process of converting unstructured text to structural text to identify the meaningful pattern of data. Different kinds of sentiments in social media posts are classified into positive and negative. Natural Language Processing (NLP) plays an imperative task in examining sentiments on social media. Sentimental analysis techniques frequently use text data to help to monitor customer feedback about products and understand the needs of customers in the field of business. The text data are gathered from social media comments that lead to several drawbacks such as spelling errors, noise, and unstructured data, and some of the data are in abbreviation form in which abbreviations are hard to understand from the text data. Therefore, the innovative sentiment analysis method is implemented to identify the kinds of sentiments from the collected social media posts. At first, the text data are assembled from the social media posts. The collected text data are subjected to text preprocessing techniques to improve the data quality. NLP techniques such as a bag of n-grams, glove embedding, and Bidirectional Encoder Representations from Transformers (BERT) are applied to separate the features from the text data. The removed features from these techniques are given to the fusion of the optimal feature stage, where the weight optimization takes place via the Mutated Random Parameter-based Serval Optimization Algorithm (MRP-SOA). The attained optimal weighted fused features contain the most relevant information, and it is fed to the classifier for analyzing the sentiments. For analyzing the sentiments, the Cascaded Adaptive Dilated Temporal Convolutional Network (CADTCN) is utilized, and the parameters from ADTCN are optimized with the same to get higher classification outcomes. The analysis outcomes are compared with the traditional sentiment analysis models in terms of various metrics to show the success of the advanced design.

Exploring Players’ Perspectives: A Comprehensive Topic Modeling Case Study on Elden Ring

Game reviews heavily influence public perception. User feedback is crucial for developers, offering valuable insights to enhance game quality. In this research, Metacritic game reviews for Elden Ring were analyzed for topic modeling using Latent Dirichlet Allocation (LDA), Bidirectional Encoder Representations from Transformers (BERT), and a hybrid model combining both to identify effective methods for extracting underlying themes in player feedback. We analyzed and interpreted these models’ outputs to learn the game reviews. We aimed to identify the differences, similarities, and variations between the three to determine which provided more valuable and instructive information. Our findings indicate that each method successfully identified keywords with some similarities in identified words. The LDA model had the highest silhouette score, indicating the most distinct clustering. The LDA-BERT model had a 1% higher coherence score than LDA, indicating more meaningful topics.

Do firms listen to the ESG voices of minority investors? Evidence from China

This study examines whether and how the expression of ESG(Environmental, Social, and Governance) concerns by minority investors in their communication with firms affects firms' ESG performance. We employ the bidirectional encoder representation from transformer (BERT) model to identify ESG statements in questions asked by minority investors and construct sentence-level ESG concern indicators. The findings indicate that the more ESG-related questions investors ask in their interactions with firms, the better the firms' future ESG performance. This effect is more pronounced when investors' ESG-related questions are more negative, when managers show greater willingness to learn from investor interactions, when listed firms emphasize investor relations management, and when listed firms are held by ESG funds. Additional analyses reveal that pressure from the market and regulation may explain why listed companies listen to the ESG voice of minority investors. In addition, we find that minority investor voices do not lead to opportunistic greenwashing when driving improvements in firms' ESG performance. Overall, our findings suggest that minority investors promote corporate sustainability through active voice.

An advanced learning approach for detecting sarcasm in social media posts: Theory and solutions

AbstractObjectiveUsers of social media platforms such as Facebook, Instagram, and Twitter can view and share their daily life events through text, photographs, or videos. These platforms receive many sarcastic posts daily because there were fewer limits on what could be posted. The presence of multiple languages and media types in a single post makes it harder to identify sarcastic messages on the current platform than on posts written solely in English.MethodsThis study provides both the theory and solutions about sarcastic post detection on social platforms. Hindi–English code‐mixed data were used to train and test the automated models for sarcasm detection. The models in this study were constructed using traditional machine learning, deep neural networks, LSTM (long short‐term memory), CNN (convolutional neural network), and the combinations of BERT (Bidirectional Encoder Representations from Transformers) with LSTM.ResultsThe experimental results confirm that in the Hindi–English code‐mixed data set, the CNN, LSTM, and BERT‐LSTM ensemble perform best for sarcasm detection. The proposed model achieved an accuracy of 96.29 percent and outperformed by 2.29 percent compared to the existing models.ConclusionThe performance of the proposed system strengthens the code‐mixed sarcastic post detection on social platforms. The model will help filter not only English but also Hindi‐English code‐mixed sarcastic posts on social platforms.

On the development and validation of large language model-based classifiers for identifying social determinants of health

The assessment of social determinants of health (SDoH) within healthcare systems is crucial for comprehensive patient care and addressing health disparities. Current challenges arise from the limited inclusion of structured SDoH information within electronic health record (EHR) systems, often due to the lack of standardized diagnosis codes. This study delves into the transformative potential of large language models (LLM) to overcome these challenges. LLM-based classifiers-using Bidirectional Encoder Representations from Transformers (BERT) and A Robustly Optimized BERT Pretraining Approach (RoBERTa)-were developed for SDoH concepts, including homelessness, food insecurity, and domestic violence, using synthetic training datasets generated by generative pre-trained transformers combined with authentic clinical notes. Models were then validated on separate datasets: Medical Information Mart for Intensive Care-III and our institutional EHR data. When training the model with a combination of synthetic and authentic notes, validation on our institutional dataset yielded an area under the receiver operating characteristics curve of 0.78 for detecting homelessness, 0.72 for detecting food insecurity, and 0.83 for detecting domestic violence. This study underscores the potential of LLMs in extracting SDoH information from clinical text. Automated detection of SDoH may be instrumental for healthcare providers in identifying at-risk patients, guiding targeted interventions, and contributing to population health initiatives aimed at mitigating disparities.

SigBERT: vibration-based steel frame structural damage detection through fine-tuning BERT

PurposeThis article introduces SigBERT, a novel approach that fine-tunes bidirectional encoder representations from transformers (BERT) for the purpose of distinguishing between intact and impaired structures by analyzing vibration signals. Structural health monitoring (SHM) systems are crucial for identifying and locating damage in civil engineering structures. The proposed method aims to improve upon existing methods in terms of cost-effectiveness, accuracy and operational reliability.Design/methodology/approachSigBERT employs a fine-tuning process on the BERT model, leveraging its capabilities to effectively analyze time-series data from vibration signals to detect structural damage. This study compares SigBERT's performance with baseline models to demonstrate its superior accuracy and efficiency.FindingsThe experimental results, obtained through the Qatar University grandstand simulator, show that SigBERT outperforms existing models in terms of damage detection accuracy. The method is capable of handling environmental fluctuations and offers high reliability for non-destructive monitoring of structural health. The study mentions the quantifiable results of the study, such as achieving a 99% accuracy rate and an F-1 score of 0.99, to underline the effectiveness of the proposed model.Originality/valueSigBERT presents a significant advancement in SHM by integrating deep learning with a robust transformer model. The method offers improved performance in both computational efficiency and diagnostic accuracy, making it suitable for real-world operational environments.

Sentiment Analysis of Indonesian Citizens on Electric Vehicle Using FastText and BERT Method

Electric vehicles have become one of the most important innovations in the automotive industry in recent years. This is not only related to technological developments, but also to its significant impact on the environment and lifestyle of global society. Lot of people do not know about the benefit of using electric vehicles for our environment. The transition from conventional vehicles to electric vehicles can really make our environment healthier and also reducing the pollution. At the same time, debates and feelings about electric vehicles continue to grow around the world. This study aims to understand the dynamics of people's feelings and opinions about electric vehicles through sentiment analysis using the FastText and IndoBERT methods. FastText is an efficient text classification and representation learning method developed by Facebook's AI Research (FAIR) lab. IndoBERT is a pre-trained language model specifically designed for the Indonesian language, leveraging the Bidirectional Encoder Representations from Transformers (BERT) architecture. By analyzing a total of 119,310 data from January 2020 to June 2023, the tweets data were categorized into negative, neutral, and positive classes. Model yielded the highest accuracy of 82.5% using IndoBERT method. The results outcomes positive perceptions of electric vehicles among Indonesian citizen with a percentage of 58%. By carrying out this research, it is hoped that it can produce quality information for producers, the community and the government in developing and advancing public interest in purchasing electric vehicles considering the very positive impact they have on the surrounding environment.

A Domain-Specific Lexicon for Improving Emergency Management in Gas Pipeline Networks through Knowledge Fusing

Emergencies in gas pipeline networks can lead to significant loss of life and property, necessitating extensive professional knowledge for effective response and management. Effective emergency response depends on specialized knowledge, which can be captured efficiently through domain-specific lexicons. The goal of this research is to develop a specialized lexicon that integrates domain-specific knowledge to improve emergency management in gas pipeline networks. The process starts with an enhanced version of Term Frequency–Inverse Document Frequency (TF-IDF), a statistical method used in information retrieval, combined with filtering logic to extract candidate words from investigation reports. Simultaneously, we fine tune the Chinese Bidirectional Encoder Representations from Transformers (BERT) model, a state-of-the-art language model, with domain-specific data to enhance semantic capture and integrate domain knowledge. Next, words with similar meanings are identified through word similarity analysis based on standard terminology and risk inventories, facilitating lexicon expansion. Finally, the domain-specific lexicon is formed by amalgamating these words. Validation shows that this method, which integrates domain knowledge, outperforms models that lack such integration. The resulting lexicon not only assigns domain-specific weights to terms but also deeply embeds domain knowledge, offering robust support for cause analysis and emergency management in gas pipeline networks.

Unveiling the hidden patterns: A novel semantic deep learning approach to fake news detection on social media

The rise of social media as a source of news consumption has led to the spread of fake news, posing serious consequences for both individuals and society. The detection and prevention of fake news are essential, and previous research has shown that incorporating news content along with its associated headlines and user comments can improve detection performance. However, the semantic relationships between these elements have not been fully explored. This paper proposes a novel approach that models the relationships between news bodies and associated headlines/user comments using deep learning techniques, such as fine-tuned Bidirectional Encoder Representations from Transformers (BERT) and cross-level cross-modality attention sub-networks. In our proposed model, we utilize two different configurations of BERT: pool-based representation, which provides a representation of the entire document, and sequence representation, which represents each token within the document (i.e., at the word and text levels). The approach also encodes user-posting behavioural features and fuses the output of these components to detect fake news using a classification layer. Our experiments on benchmark datasets demonstrate the superiority of the proposed method over existing state-of-the-art (SOTA) approaches, highlighting the importance of utilizing semantic relationships for improved fake news detection (FND). These findings have significant implications for combating the spread of fake news and protecting society from its negative effects.

CACER: Clinical concept Annotations for Cancer Events and Relations.

Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. We present Clinical concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48000 medical problems and drug events and 10000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction models such as Bidirectional Encoder Representations from Transformers (BERT), Fine-tuned Language Net Text-To-Text Transfer Transformer (Flan-T5), Large Language Model Meta AI (Llama3), and Generative Pre-trained Transformers-4 (GPT-4) using fine-tuning and in-context learning (ICL). In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, large language models offer no performance advantage over the smaller BERT models. We introduce CACER, a novel corpus with fine-grained annotations for medical problems, drugs, and their relationships in clinical narratives of oncology notes. State-of-the-art transformer models achieved performance comparable to IAA for several extraction tasks.

Unveiling the drivers of satisfaction in mobile trading: Contextual mining of retail investor experience through BERTopic and generative AI

The proliferation of mobile stock trading has introduced various apps with distinct features, emphasizing the need to understand users’ evaluations after adopting the service. This study explores the determinants of retail investors’ satisfaction with mobile stock trading services by employing an advanced textual analysis of customer reviews for four leading trading applications. We utilized Bidirectional Encoder Representations from Transformers (BERT) based Topic modeling (BERTopic modeling) to identify key topics within customer reviews and used the results as input for generative AI to discern the theme and sentiment of each topic. Based on Service Quality (SERVQUAL) theory, topics are categorized into key quality dimensions: functionality, usability, information quality, customer service, and system quality. Regression models were employed to assess the impact of the quality dimensions on investor satisfaction, revealing positive feedback on usability, information quality, and service quality as primary enhancers of satisfaction. In contrast, negative feedback on service quality, system quality, and functionality was identified as the primary inhibitor of satisfaction. This study explores how the influence of each quality dimension varies among different types of brokers (full-service vs. online-only brokerages). Finally, we propose a visualization tool called Topic Rating Impact and Frequency Analysis (TRIFA), which is designed to categorize topics based on their frequency of occurrence and impact on satisfaction. This tool aids in identifying the strengths and areas for improvement in services by effectively visualizing the results of text review analysis. This research not only deepens our understanding of the quality dimensions of mobile financial services but also offers valuable insights for service providers by suggesting predictive models that could help increase customer retention.

Fine-grained vulnerability detection for medical sensor systems

The Internet of Things (IoT) has revolutionized the healthcare system by connecting medical sensors to the internet, while also posing challenges to the security of medical sensor networks (MSN). Given the extreme sensitivity of medical data, any vulnerability may result in data breaches and misuse, impacting patient safety and privacy. Therefore, safeguarding MSN security is critical. As medical sensor devices rely on smart healthcare software systems for data management and communication, precisely detecting system code vulnerabilities is essential to ensuring network security. Effective software vulnerability detection targets two key objectives: (i) achieving high accuracy and (ii) directly identifying vulnerable code lines for developers to fix. To address these challenges, we introduce Vulcoder, a novel vulnerability-oriented, encoder-driven model based on the Bidirectional Encoder Representations from Transformers (BERT) architecture. We propose a one-to-one mapping function to capture code semantics through abstract syntax trees (AST). Combined with multi-head attention, Vulcoder achieves precise function- and line-level detection of software vulnerabilities in MSN. This accelerates the vulnerability remediation process, thereby strengthening network security. Experimental results on various datasets demonstrate that Vulcoder outperforms previous models in identifying vulnerabilities within MSN. Specifically, it achieves a 1%–419% improvement in function-level prediction F1 scores and a 12.5%–380% increase in line-level localization precision. Therefore, Vulcoder helps enhance security defenses and safeguard patient privacy in MSN, facilitating the development of smart healthcare.

A large language model-based clinical decision support system for syncope recognition in the emergency department: A framework for clinical workflow integration

Differentiation of syncope from transient loss of consciousness can be challenging in the emergency department (ED). Natural Language Processing (NLP) enables the analysis of free text in the electronic medical records (EMR).The present paper aimed to develop a large language models (LLM) for syncope recognition in the ED and proposed a framework for model integration within the clinical workflow.Two models, based on both the Italian and Multilingual Bidirectional Encoder Representations from Transformers (BERT) language model, were developed using consecutive EMRs. The “triage” model was only based on notes contained in the “triage” section of the EMR. The “anamnesis” model added data contained in the “medical history” section. Interpretation and calibration plots were generated.The Italian and Multi BERT models were developed and tested on both 15,098 and 15,222 EMRs, respectively. The triage model had an AUC of 0·95 for the Italian BERT and 0·94 for the Multi BERT. The anamnesis model had an AUC of 0·98 for the Italian BERT and 0·97 for Multi BERT. The LLM identified syncope when not explicitly mentioned in the EMR and also recognized common prodromal symptoms preceding syncope.Both models identified syncope patients in the ED with a high discriminative capability from nurses and doctors’ notes, thus potentially acting as a tool helping physicians to differentiate syncope from others transient loss of consciousness.

Exploring RoBERTa model for cross-domain suggestion detection in online reviews

Detecting suggestions in online review requires contextual understanding of review text, which is an important real-world application of natural language processing. Given the disparate text domains found in product reviews, a common strategy involves fine-tuning bidirectional encoder representations from transformers (BERT) models using reviews from various domains. However, there hasn't been an empirical examination of how BERT models behave across different domains in tasks related to detecting suggestion sentences from online reviews. In this study, we explore BERT models for suggestion classification that have been fine-tuned using single-domain and cross-domain Amazon review datasets. Our results indicate that while single-domain models achieved slightly better performance within their respective domains compared to cross-domain models, the latter outperformed single-domain models when evaluated on cross-domain data. This was also observed for single-domain data not used for fine-tuning the single-domain model and on average across all tests. Although fine-tuning single-domain models can lead to minor accuracy improvements, employing multi-domain models that perform well across domains can help in cold start problems and reduce annotation costs.

Virality classification from Twitter data using pre-trained language model and multi-layer perceptron

Twitter is one of the well-known text-based social media that is often used to disseminate content. According to Katadata, Indonesia ranked fifth in the world in 2023. So many people or organizations want to make tweets go viral. Therefore, this research aims to develop a model that uses tweet data from the Indonesian language Twitter social media to categorize the level of virality. There are several tasks in classifying the level of virality, such as upsampling data, predicting sentiment and emotion, and text embedding. Upsampling data was carried out because the dataset used was an imbalanced dataset. Data upsampling, emotions, and text embedding is carried out using the bidirectional encoder representation from transformers (BERT) model. Meanwhile, sentiment prediction uses the Ro-bustly optimized BERT pretraining approach (RoBERTa). The results of text embedding, sentiment, emotion, will be combined with Twitter metadata then all features will be fed into the multi-layer perceptron (MLP) model to classifying the level of virality which is divided into 3 classes based on the number of retweets, namely low, medium and high. The proposed method produces an F1-score of 49% and an accuracy of 95% and performs better than the baseline model.

Hidden emotional trends on social media regarding the Thailand–China high-speed railway project: a deep learning approach with ChatGPT integration

Leveraging sentiment analysis on social media reveals hidden emotional trends by providing deep insights into the public’s collective opinions and feelings because social media can reflect most opinions based on true feelings. However, there is no time-series analysis of Thai opinions on social media regarding government projects in Thailand. Therefore, the focus of this research is to survey and dynamically analyze sentiment on social media platforms, particularly YouTube, regarding the Thailand–China High-Speed Rail Project (Thai–China HSR Project). This is a major project involving an investment of over 500 billion baht, a collaborative effort between the Thai and Chinese governments aimed at developing transportation infrastructure. It leads the public concerns of Thai people about the potential economic and social impacts. The deep learning (DL) technique was used for this analysis by the Bidirectional Encoder Representations from Transformers (BERT) model, together with WangchanBERTa, a Thai language model specifically designed to analyze opinions from Thai comments received from YouTube from July 2015 to January 2024. The analysis reveals that citizens’ sentiments are primarily neutral, with more negative than positive sentiments with the passing years. Results also indicate that the data volume has increased since 2019 and peaked in 2022. Negative comments peaked in January 2022, and based on our analysis, such negative comments followed news covering the opening of the China–Laos Railway on December 1, 2021. Video contents were related to the Thai–China HSR Project, which provides comparative reviews and analysis of government operations. Additionally, we explore the possibility of integrating AI technology based on the ChatGPT 4 model to support data interpretation. Although the AI model can interpret data quickly and efficiently serve as a convenient method for users, the results lack the in-depth analysis that should be done by experts in the field. The proposed framework could be useful to government agencies and can be used to quickly and efficiently survey and track public opinion. The results can be used for policy planning, strategic decision-making, and improving communications to better inform citizens about projects and operations, leading to the country’s sustainable economic and social development.

Virtual Assistant for Local Government Units Using BERT-Based Natural Language Processing

Local Government Units (LGUs) play a critical role in public service delivery, yet they often encounter challenges such as limited resources, bureaucratic inefficiencies, and high volumes of citizen inquiries. To address these issues, this paper proposes the development of a virtual assistant powered by Natural Language Processing (NLP), specifically utilizing the Bidirectional Encoder Representations from Transformers (BERT) model. This assistant is designed to automate the handling of common citizen inquiries, thereby enhancing the efficiency of service delivery. The virtual assistant was implemented as a web application using the Django framework, with an evaluation based on ISO 9126 standards, including aspects of functionality, reliability, usability, and efficiency. The findings demonstrate that the BERT-based NLP system significantly improves the responsiveness and accuracy of LGU services, leading to enhanced citizen satisfaction and more streamlined public service operations. The model achieved an Exact Match score of 48.0 and an F1 score of 68.01, indicating its effectiveness in accurately understanding and responding to citizen queries. Additionally, the system scored highly in usability, efficiency, maintainability, and portability, confirming its ease of use and operational effectiveness. These results give rather strong evidence that the use of the NLP-driven virtual assistant can be beneficial for increasing the citizen satisfaction and improving the effectiveness of the public service delivery.

Multi-Feature Fusion in Graph Convolutional Networks for Data Network Propagation Path Tracing

With the rapid development of information technology, the complexity of data networks is increasing, especially in electric power systems, where data security and privacy protection are of great importance. Throughout the entire distribution process of the supply chain, it is crucial to closely monitor the propagation paths and dynamics of electrical data to ensure security and quickly initiate comprehensive traceability investigations if any data tampering is detected. This research addresses the challenges of data network complexity and its impact on the security of power systems by proposing an innovative data network propagation path tracing model, which is constructed based on graph convolutional networks (GCNs) and the BERT model. Firstly, propagation trees are constructed based on the propagation structure, and the key attributes of data nodes are extracted and screened. Then, GCNs are utilized to learn the representation of node features with different attribute feature combinations in the propagation path graph, while the Bidirectional Encoder Representations from Transformers (BERT) model is employed to capture the deep semantic features of the original text content. The core of this research is to effectively integrate these two feature representations, namely the structural features obtained by GCNs and the semantic features obtained by the BERT model, in order to enhance the ability of the model to recognize the data propagation path. The experimental results demonstrate that this model performs well in power data propagation and tracing tasks, and the data recognition accuracy reaches 92.5%, which is significantly better than the existing schemes. This achievement not only improves the power system’s ability to cope with data security threats but also provides strong support for protecting data transmission security and privacy.

Movie Review Based Sentiment Analysis

Sentiment analysis, an automated method for detecting emotions and opinions in text, has become a versatile tool applied across various domains, from assessing online customer feedback to monitoring sentiment trends on social media. While Bag of Words (BoW) has been the traditional method, more advanced techniques like Bidirectional Encoder Representations from Transformers (BERT) and Long Short-Term Memory (LSTM) have revolutionized sentiment analysis by elevating accuracy and context comprehension. This study conducts an extensive comparison between BERT, a hybrid model that combines CNN and LSTM in the realm of sentiment analysis within movie reviews. Through meticulous experimentation, the study strongly proves that both the hybrid model of CNN and LSTM models significantly outshine the techniques like BERT and BoW with noticeably higher accuracy in precisely categorizing sentiments expressed in movie reviews. These results highlight how deep learning methodologies remarkably refine the precision and effectiveness of sentiment analysis, allowing for more nuanced and context-aware sentiment classification. The study's significance goes beyond performance contrasts; it demonstrates the exceptional capacity of BERT along with the hybrid model of CNN and LSTM to comprehend the complexities of language and contextual nuances within movie reviews. This heightened contextual understanding is particularly crucial in sentiment analysis, enabling the models to discern subtle shifts in sentiment that simpler methods like BoW might overlook. By showcasing the supremacy of hybrid model that combines CNN and LSTM in movie review sentiment analysis, this research opens new avenues for the field's advancement. It emphasizes the potential of deep learning techniques in revolutionizing sentiment analysis across diverse domains, offering more accurate and contextually attuned sentiment classification for deeper insights and well-informed decision-making. This study stands as a valuable asset for researchers and practitioners aiming to leverage deep learning for sentiment analysis applications

Comparison of Service and Ease of e-Commerce User Applications Using BERT

The development of e-commerce has transformed shopping patterns by harnessing the internet, enabling consumers to shop online. In Indonesia, e-commerce has experienced rapid growth, with numerous options such as Tokopedia, Shopee, and Lazada, leading to intense competition. Sentiment analysis using machine learning techniques has become crucial for understanding consumer views on these e-commerce services. This study analyzes user comments on Tokopedia, Shopee, and Lazada e-commerce platforms from Instagram social media, totaling 3900 data points, using the Bidirectional Encoder Representations from Transformers (BERT) model with 5 epochs and a batch size of 32. Sentiment analysis utilizes 3 types of labels: positive, neutral, and negative. The final results of the study include the performance analysis of the BERT model, as well as comparisons for each predefined category, namely Promotions & Offers, and Services. The final results of the model indicate good performance, with accuracy rates of 95%, 97%, and 99%, respectively.