Multinomial Naive Bayes Research Articles

Machine Learning Approach for Classification of Cyber Threats Actors in Web Region

In the interconnected scape of today's internet, the dark web emerges as a concealed point, covering a myriad of illicit activities that pose substantial cybersecurity risks. This study investigates the attribution of threats within the dark web environment, leveraging on a machine learning approach to bridge the gap between technical indicators and linguistic and behavioral insights. Through a comprehensive methodology involving web crawling and data gathering, a dataset encompassing key variables such as attack motivation, method, web part, and threat actor was gathered. Principal Component Analysis was employed for feature selection, followed by the application of Multinomial Naive Bayes (MNB), Support Vector Machine (SVM), Random Forest (RF), and CatBoost algorithms for classification. Performance evaluation metrics including precision, recall, and F1-score were utilized to assess the efficacy of each algorithm. Results indicate a notable prevalence of cybercrimes within the dark web, underscoring the necessity for enhanced cybersecurity strategies tailored to address its unique challenges. Furthermore, the comparative analysis demonstrates varying performance levels among the machine learning algorithms, with Multinomial Naive Bayes exhibiting the highest accuracy. This research contributes to advancing threat attribution techniques in the dark web, ultimately aiming to bolster cybersecurity defenses and mitigate future cyber threats.

Turkish Text Classification Based On Wrapper Feature Selection Using Particle Swarm Optimization

The vast majority of the digital era data is stored as text. Text mining is an integral part of data mining. Text classification (TC) is a natural language processing (NLP) operation often needed in text mining. This operation is needed in numerous kinds of research such as information retrieval, document classification, language detection, sentiment analysis, etc. According to the literature, the filter feature selection methods have often been applied to reduce the dimensionality of data in Turkish TC. However, the wrapper-based feature selection methods can provide better classification accuracies than the filter methods. Motivated by this idea, a Turkish TC method based on wrapper feature selection using particle swarm optimization algorithm (PSO) and multinomial naive bayes (MNB) classifier is proposed in this study. TTC-3600 Turkish news texts are used for TC in the experiments. The proposed method achieves a classification accuracy of 94.55% on TTC-3600 Turkish news text dataset by using stemming Tf-Idf features. Hence, it produces competitive accuracies to the cutting-edge Turkish TC methods.

Human lung cancer classification and comprehensive analysis using different machine learning techniques.

Lung cancer is the most common causes of death among all cancer-related diseases. A lung scan examination of the patient is the primary diagnostic technique. This scan analysis pertains to an MRI, CT, or X-ray. The automated classification of lung cancer is difficult due to the involvement of multiple steps in imaging patients' lungs. In this manuscript, human lung cancer classification and comprehensive analysis using different machine learning techniques is proposed. Initially, the input images are gathered using lung cancer dataset. The proposed method processes these images using image-processing techniques, and further machine learning techniques are utilized for categorization. Seven different classifiers including the k-nearest neighbors (KNN), support vector machine (SVM), decision tree (DT), multinomial naive Bayes (MNB), stochastic gradient descent (SGD), random forest (RF), and multi-layer perceptron (MLP) classifier are used, which classifies the lung cancer as malignant and benign. The performance of the proposed approach is examined using performances metrics, like positive predictive value, accuracy, sensitivity, and f-score are evaluated. Among them, the performance of the MLP classifier provides 25.34%, 45.39%, 15.39%, 41.28%, 22.17%, and 12.12% higher accuracy than other KNN, SVM, DT, MNB, SGD, and RF respectively. RESEARCH HIGHLIGHTS: Lung cancer is a leading cause of cancer-related death. Imaging (MRI, CT, and X-ray) aids diagnosis. Automated classification of lung cancer faces challenges due to complex imaging steps. This study proposes human lung cancer classification using diverse machine learning techniques. Input images from lung cancer dataset undergo image processing and machine learning. Classifiers like k-nearest neighbors, support vector machine, decision tree, multinomial naive Bayes, stochastic gradient descent, random forest, and multi-layer perceptron (MLP) classify cancer types; MLP excels in accuracy.

A Decision Support System for Crop Recommendation Using Machine Learning Classification Algorithms

Today, crop suggestions and necessary guidance have become a regular need for a farmer. Farmers generally depend on their local agriculture officers regarding this, and it may be difficult to obtain the right guidance at the right time. Nowadays, crop datasets are available on different websites in the agriculture sector, and they play a crucial role in suggesting suitable crops. So, a decision support system that analyzes the crop dataset using machine learning techniques can assist farmers in making better choices regarding crop selections. The main objective of this research is to provide quick guidance to farmers with more accurate and effective crop recommendations by utilizing machine learning methods, global positioning system coordinates, and crop cloud data. Here, the recommendation can be more personalized, which enables the farmers to predict crops in their specific geographical context, taking into account factors like climate, soil composition, water availability, and local conditions. In this regard, an existing historical crop dataset that contains the state, district, year, area-wise production rate, crop name, and season was collected for 246,091 sample records from the Dataworld website, which holds data on 37 different crops from different areas of India. Also, for better analysis, a dataset was collected from the agriculture offices of the Rayagada, Koraput, and Gajapati districts in Odisha state, India. Both of these datasets were combined and stored using a Firebase cloud service. Thirteen different machine learning algorithms have been applied to the dataset to identify dependencies within the data. To facilitate this process, an Android application was developed using Android Studio (Electric Eel | 2023.1.1) Emulator (Version 32.1.14), Software Development Kit (SDK, Android SDK 33), and Tools. A model has been proposed that implements the SMOTE (Synthetic Minority Oversampling Technique) to balance the dataset, and then it allows for the implementation of 13 different classifiers, such as logistic regression, decision tree (DT), K-Nearest Neighbor (KNN), SVC (Support Vector Classifier), random forest (RF), Gradient Boost (GB), Bagged Tree, extreme gradient boosting (XGB classifier), Ada Boost Classifier, Cat Boost, HGB (Histogram-based Gradient Boosting), SGDC (Stochastic Gradient Descent), and MNB (Multinomial Naive Bayes) on the cloud dataset. It is observed that the performance of the SGDC method is 1.00 in accuracy, precision, recall, F1-score, and ROC AUC (Receiver Operating Characteristics–Area Under the Curve) and is 0.91 in sensitivity and 0.54 in specificity after applying the SMOTE. Overall, SGDC has a better performance compared to all other classifiers implemented in the predictions.

AN AUTOMATED NEW APPROACH IN FAST TEXT CLASSIFICATION: A CASE STUDY FOR KURDISH TEXT

With the rapid development of internet technology, text classification has become a vital part of obtaining quick and accurate data. Traditional machine learning methods often suffer from poor performance and high-dimensional feature spaces, which reduce their accuracy. In this paper, the FastText model is proposed as the first-ever classifier on Kurdish text and the results are compared with traditional machine learning methods to show the effects on Kurdish Text. For evaluating the model four datasets Kurdish News Dataset Headlines (KNDH), Medical Kurdish Dataset (MKD), Kurdish-Emotional-Dataset (KMD-77000), and KurdiSent are utilized and compared the results with the traditional machine learning algorithms such as: Random Forest (RF), k-nearest Neighbor (k-NN), Logistic Regression (LR), Multinomial Naïve Bayes (MNB), Support Vector Machine (SVM), Decision Tree (DT), Stochastic Gradient Descent (SGD), as well as the deep learning model Bidirectional Encoder Representations from Transformers (BERT). The outcomes indicate that the FastText model achieved the highest performance with 89% for each precision, recall, F1-score, and 89.10% accuracy for the KNDH dataset. Moreover, when the KMD dataset is utilized the FatText model obtained outperforms all others by approximately 2%. In addition, the comparative analysis showed that FastText is superior when Kurdisent is considered with precision, recall, F1-score, and accuracy by 81.32, 81.83, 81.57, and 81.4 respectively. In addition, when MKD is implemented, the FastText model obtained the highest performance with a precision of 93.32%, recall of 93.36, F1-score of 93.34, and accuracy of 93.1%.

A Comparative Analysis of Machine Learning Models in News Categorization

The constant stream of news nowadays highlights the necessity for meticulous assessment to ensure that the information accurately reaches its intended audience with the least amount of delay least delay. Despite the flexibility and efficiency of Deep Learning (DL) models, their intricate training and substantial resource demands pose significant challenges for their deployment in real-time applications. In this regard, this study evaluates the performance of resource-efficient Machine Learning (ML) techniques – Multinomial Naive Bayes (MNB), Random Forest (RF), Support Vector Machine (SVM), and Logistic Regression (LR) – in categorizing news. Based on the results, all the evaluated models attain a commendable level of accuracy in news categorization. Notably, the SVM excels, achieving an accuracy rate of 98% and a mean squared error of 0.28. This performance exemplifies the robust effectiveness of classical ML models in the categorization of news, particularly when enhanced by a suitably tailored preprocessing pipeline.

HAPI: An efficient Hybrid Feature Engineering-based Approach for Propaganda Identification in social media.

Social media platforms serve as communication tools where users freely share information regardless of its accuracy. Propaganda on these platforms refers to the dissemination of biased or deceptive information aimed at influencing public opinion, encompassing various forms such as political campaigns, fake news, and conspiracy theories. This study introduces a Hybrid Feature Engineering Approach for Propaganda Identification (HAPI), designed to detect propaganda in text-based content like news articles and social media posts. HAPI combines conventional feature engineering methods with machine learning techniques to achieve high accuracy in propaganda detection. This study is conducted on data collected from Twitter via its API, and an annotation scheme is proposed to categorize tweets into binary classes (propaganda and non-propaganda). Hybrid feature engineering entails the amalgamation of various features, including Term Frequency-Inverse Document Frequency (TF-IDF), Bag of Words (BoW), Sentimental features, and tweet length, among others. Multiple Machine Learning classifiers undergo training and evaluation utilizing the proposed methodology, leveraging a selection of 40 pertinent features identified through the hybrid feature selection technique. All the selected algorithms including Multinomial Naive Bayes (MNB), Support Vector Machine (SVM), Decision Tree (DT), and Logistic Regression (LR) achieved promising results. The SVM-based HaPi (SVM-HaPi) exhibits superior performance among traditional algorithms, achieving precision, recall, F-Measure, and overall accuracy of 0.69, 0.69, 0.69, and 69.2%, respectively. Furthermore, the proposed approach is compared to well-known existing approaches where it overperformed most of the studies on several evaluation metrics. This research contributes to the development of a comprehensive system tailored for propaganda identification in textual content. Nonetheless, the purview of propaganda detection transcends textual data alone. Deep learning algorithms like Artificial Neural Networks (ANN) offer the capability to manage multimodal data, incorporating text, images, audio, and video, thereby considering not only the content itself but also its presentation and contextual nuances during dissemination.

Mining software insights: uncovering the frequently occurring issues in low-rating software applications.

In today's digital world, app stores have become an essential part of software distribution, providing customers with a wide range of applications and opportunities for software developers to showcase their work. This study elaborates on the importance of end-user feedback for software evolution. However, in the literature, more emphasis has been given to high-rating & popular software apps while ignoring comparatively low-rating apps. Therefore, the proposed approach focuses on end-user reviews collected from 64 low-rated apps representing 14 categories in the Amazon App Store. We critically analyze feedback from low-rating apps and developed a grounded theory to identify various concepts important for software evolution and improving its quality including user interface (UI) and user experience (UX), functionality and features, compatibility and device-specific, performance and stability, customer support and responsiveness and security and privacy issues. Then, using a grounded theory and content analysis approach, a novel research dataset is curated to evaluate the performance of baseline machine learning (ML), and state-of-the-art deep learning (DL) algorithms in automatically classifying end-user feedback into frequently occurring issues. Various natural language processing and feature engineering techniques are utilized for improving and optimizing the performance of ML and DL classifiers. Also, an experimental study comparing various ML and DL algorithms, including multinomial naive Bayes (MNB), logistic regression (LR), random forest (RF), multi-layer perception (MLP), k-nearest neighbors (KNN), AdaBoost, Voting, convolutional neural network (CNN), long short-term memory (LSTM), bidirectional long short term memory (BiLSTM), gated recurrent unit (GRU), bidirectional gated recurrent unit (BiGRU), and recurrent neural network (RNN) classifiers, achieved satisfactory results in classifying end-user feedback to commonly occurring issues. Whereas, MLP, RF, BiGRU, GRU, CNN, LSTM, and Classifiers achieved average accuracies of 94%, 94%, 92%, 91%, 90%, 89%, and 89%, respectively. We employed the SHAP approach to identify the critical features associated with each issue type to enhance the explainability of the classifiers. This research sheds light on areas needing improvement in low-rated apps and opens up new avenues for developers to improve software quality based on user feedback.

Language Identifier

This project presents a language identifier system focused on achieving high accuracy in identifying a specific set of 22 languages: Estonian, Swedish, English, Russian, Romanian, Persian, Pashto, Spanish, Hindi, Korean, Chinese, French, Portuguese, Indonesian, Urdu, Latin, Turkish, Japanese, Dutch, Tamil, Thai, and Arabic. Existing LI systems might struggle with the nuances of these languages, often prioritizing identification of more common languages. Our targeted approach allows for tailored optimization to achieve superior accuracy. We employ the Multinomial Naive Bayes (MNB) algorithm due to its effectiveness in text classification tasks and its ability to handle the high-dimensional, sparse features characteristic of LI based on character and word frequencies. The system achieves a promising accuracy of 95% using an 80/20 split for training and testing data. Keywords: Natural Language Processing (NLP), Language Identification (LI), Multinomial Naive Bayes (MNB), Text Classification, Language Barriers, Accuracy, Text Analysis.

A hybrid NLP and domain validation technique for disposable email detection

Disposable email address services have gained popularity recently, offering users a way to register for online services without revealing their primary email. However, this poses challenges for organizations aiming to engage with genuine users. This paper introduces a novel approach combining natural language processing (NLP) and domain validation to identify disposable email addresses. The technique employs various machine learning methods, including Support Vector Classifier (SVC), Multinomial Naive Bayes (MNB), Gaussian Naive Bayes (GNB), Logistic Regression, XGBoost Classifier, K Neighbors Classifier, Random Forest Classifier, Linear, and Discriminant Analysis. Unlike traditional methods relying on blacklists, this technique achieves a 97% accuracy rate by effectively detecting and classifying new disposable emails.

Analysis of the emotional coloring of text using machine and deep learning methods

The presented scientific article is a comprehensive study of machine learning and deep learning methods in the context of emotion recognition in text data. The main goal of the study is to conduct a comprehensive analysis and comparison of various machine learning and deep learning methods to classify emotions in text. During the work, special attention was paid to the analysis of traditional machine learning algorithms, such as multinomial naive Bayes (MNB), multilayer perceptron (MLP), and support vector machine (SVM), as well as the use of deep learning methods based on long short-term memory (LSTM). The experimental part of the study involves the analysis of different data sets covering a variety of text styles and contexts. The experimental results are analyzed in detail, identifying the advantages and limitations of each method. The article provides practical recommendations for choosing the optimal method depending on the specific tasks and context of the application. The data obtained is important for the development of intelligent systems that can effectively adapt to the emotional aspects of interaction with users. Overall, this work makes a significant contribution to the field of emotion recognition in text and provides a basis for further research in this area.

Automated Matching Skills to Improve the Accuracy of Job Applicant Selection Using Indonesian National Work Competency Standards

The high number of cyberattack anomalies and data leaks in Indonesia increases the need for cybersecurity in various companies. Cybersecurity capabilities and skills in Indonesia are divided into three categories based on the Indonesian National Work Competency Standards (SKKNI), namely Security Operation Center (SOC), Cybersecurity test/Penetration testing (Pentest), and Information Security Audit. Although various approaches have been applied in different companies to select job applicants, a new method with automated matching is explored in this study. This method matches the skills possessed by prospective job applicants with the profile of their job task requirements based on the SKKNI Decree of the Minister of Manpower of the Republic of Indonesia using Machine Learning (ML) models. The empirical comparison of results comes from automated matchmaking processed by Multinomial Naive Bayes (MNB) and Decision Tree algorithm models. Before modeling, the data is trained and evaluated for testing. Then to assess the most optimal algorithm between MNB and Decision Tree, a confusion matrix is proposed and used to find the best model. From the evaluation results, both models performed well and were highly accurate during training and test evaluation. The Decision Tree model performs slightly better than the MNB model, but both still provide satisfactory results in classifying data based on the Indonesian National Work Competency Standards (SKKNI) categories. This study offers a solution to minimize the number of potential applicants who are not competent in the three SKKNI cybersecurity job categories due to the mismatch of their abilities and skills.

#1756 Development and internal validation of machine learning algorithms for mortality prediction model of people with DM and CKD

Abstract Background and Aims Diabetes Mellitus (DM) and Chronic Kidney Disease (CKD) are leading causes of morbidity and mortality, presenting challenges in patient management. The aim of this study is to develop a machine learning-based predictive model for mortality in DM and CKD patients, improving early intervention and treatment personalization. Method 3,637 participants with DM and CKD from the National Health and Nutrition Examination Survey (NHANES) from 1999 to 2018 were included. All-cause mortality was ascertained by linkage to National Death Index records through 31 December 2019 in NHANES. The dataset included clinical profiles, demographic details, and laboratory results of patients with DM and CKD. We performed 100 random groupings, dividing the data into a 75% training set and a 25% validation set each time. We then visualized the results of 100 AUC measurements using a boxplot. The objective was to forecast patient survival over 1, 3, 5, and 10-year horizons using a variety of machine learning algorithms. The algorithms tested included Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), K-Nearest Neighbors (KNN), Gaussian Naive Bayes (GNB), Multinomial Naive Bayes (MNB), Bayesian Network Classifier (BNC), AdaBoost (Ada), Gradient Boosting (Gradient), and Extreme Gradient Boosting (XG). The performance of each algorithm was evaluated based on the Area Under the Receiver Operating Characteristic (ROC) Curve (AUC), with the results visualized in boxplot format. Results Upon result shown as the boxplots, it becomes evident that the Random Forest (RF) algorithm consistently exhibits one of the highest AUCs on average across all prediction intervals, suggesting a strong and stable predictive capacity. In the 1-year survival prediction, the RF algorithm demonstrates a highest AUC, implying consistent performance. The 3 and 5-year predictions reveal a slight dip in AUC for most models, including RF, but it still remains one of the top-performing algorithms with fewer outliers than XG and Gradient Boosting. The mean AUC of RF is particularly noteworthy at the 10-year mark, where most algorithms struggle with prediction accuracy, as indicated by lower median AUCs and greater variability. Notably, the RF algorithm identified six major factors for 1-year survival prediction: systolic blood pressure, uric acid, blood urea nitrogen, lactate dehydrogenase, lymphocyte count and total cholesterol, as well as three major factors for 10-year survival prediction: age, lymphocyte and gamma-glutamyl transferase. Conclusion Overall, the study indicates that while machine learning can be a powerful tool for survival prediction in patients with DM and CKD, the choice of algorithm is crucial, with RF standing out for its consistent and reliable performance.

Twitter sentiment analysis for Hausa abbreviations and acronyms

The use of natural language processing, to identify, extract and organize sentiment from user generated texts in social networks, blogs or product review of text is known as sentiment analysis or opinion mining. Hausa language belongs to one of the major well-spoken languages in Africa and one of the three major Nigerian languages. Now investigating into such a language will have significant influence on social, economic business political and even educational services and settings. Some of these Hausa texts are abbreviated and some in acronym format which is a challenge to researchers as such comments are in an unstructured format and needs normalization to get further understanding of that text and also there is scarcity of sentiment analysis on Hausa abbreviation and acronym. Abbreviation is a shorten form of a word while acronym is an abbreviation formed from the initial letters of other words and pronounced as a word. This research aims to develop an improved Hausa Sentiment Dataset for the enhancement of sentiment analysis with abbreviation and acronyms. This is achieved by adapting to the approach for Hausa Sentiment Analysis based on Multinomial Naïve Bayes (MNB) and Logistic Regression algorithms using the count vectorizer, along with python libraries for NLP. This research affirmed that the improved dataset with abbreviation and acronym outperforms the plain Hausa dataset by 4% in accuracy using Multinomial Naïve Bayes. The result shows that in addition to normal preprocessing techniques of the social media stream, understanding, interpreting and resolving ambiguity in the usage of abbreviations and acronyms lead to improved accuracy of algorithms with evidence in the experimental result.

Fast Naïve Bayes classifiers for COVID-19 news in social networks

The growth of fake news has emerged as a substantial societal concern, particularly in the context of the COVID-19 pandemic. Fake news can lead to unwarranted panic, misinformed decisions, and a general state of confusion among the public. Existing methods to detect and filter out fake news have accuracy, speed, and data distribution limitations. This study explores a fast and reliable approach based on Naïve Bayes algorithms for fake news detection on COVID-19 news in social networks. The study used a dataset of 10,700 tweets and applied text pre-processing, term-weighting, document frequency thresholding (DFT), and synthetic minority oversampling techniques (SMOTE) to prepare the data for classification. The study assessed the performance and runtime of four models: gradient boosting (GDBT), decision tree (DT), multinomial Naïve Bayes (MNB), and complement Naïve Bayes (CNB). The testing results showed that the CNB model reaches the highest accuracy, precision, recall, and F1-score of approximately 92% each, with the shortest runtime of 0.55 seconds. This study highlights the potential of the CNB model as an effective tool for detecting online fake news about COVID-19, given its superior performance and rapid processing time.

Bridging the Gap

Effective organization and retrieval of news content are heavily reliant on accurate news classification. While the mountainous research has been conducted in resourceful languages like English and Chinese, the researches on under-resourced languages like the Kurdish language are severely lacking. To address this challenge, we introduce a hybrid approach called RFO-CNN in this paper. The proposed method combines an improved version of red fox optimization algorithm (RFO) and convolutional neural network (CNN) for finetuning CNN’s parameters. Our model’s efficacy was tested on two widely used Kurdish news datasets, KNDH and KDC-4007, both of which contain news articles classified into various categories. We compared the performance of RFO-CNN to other cutting-edge deep learning models such as bidirectional long short-term memory networks and bidirectional encoder representations from transformers (BERT) transformers, as well as classical machine learning approaches such as multinomial naive bayes, support vector machine, and K-nearest neighbors. We trained and tested our datasets using four different scenarios: 60:40, 70:30, 80:20, and 90:10. Our experimental results demonstrate the superiority of the RFO-CNN model across all scenarios, outperforming the benchmark BERT model and other machine learning models in terms of accuracy and F1-score.

A Comprehensive Study on Email Spam Detection Techniques

The analysis of email spam detection methods is pivotal in the realm of cybersecurity. This studymeticulously explores a variety of techniques utilized for discerning spam emails, encompassing traditionalmethodologies alongside sophisticated approaches such as machine learning algorithms. The examined techniquesinclude Support Vector Machines, Naive Bayes Multinomial Classifier and Random Forest Classifier with a primaryobjective of identifying the most effective approach among them. By scrutinizing each method's strengths andweaknesses, encompassing factors like detection accuracy and computational efficiency, this research seeks to offer acomprehensive overview of their utility in combatting email spam. Additionally, the study investigates the impact ofhuman elements, such as user education and awareness initiatives, in bolstering overall detection performance. Byamalgamating insights from various scholarly endeavors, this inquiry not only enriches the understanding ofcontemporary spam detection practices but also delineates pathways for future research endeavors and informs theformulation of resilient cybersecurity protocols.

Detecting cyberbullying text using the approaches with machine learning models for the low-resource Bengali language

The rising usage of social media sites and the advances in communication technologies have led to a considerable increase in cyberbullying events. Here, people are intimidated, harassed, and humiliated via digital messaging. To identify cyberbullying texts, several research have been undertaken in English and other languages with abundant resources, but relatively few studies have been conducted in low-resource languages like Bengali. This research focuses on Bengali text to find cyberbullying material by experimenting with pre-processing, feature selection, and three types of machine learning (ML) models: classical ML, deep learning (DL), and transformer learning. In classical ML, four models, support vector machine (SVM), multinomial Naive Bayes (MNB), random forest (RF), and logistic regression (LR) are used. In DL, three models, long short term memory (LSTM), Bidirectional LSTM, and convolutional neural network with bidirectional LSTM (CNN-BiLSTM) are employed. As the transformerbased pre-trained model, bidirectional encoder representations from transformers (BERT) is utilized. Using our proposed pre-processing tasks, the MNB-based approach achieves the best accuracy of 78.816% among the other classical ML models, the LSTM-based approach gains the highest result of 77.804% accuracy among the DL models, and the BERT-based approach outperforms both with 80.165% accuracy.

BERT for Twitter Sentiment Analysis: Achieving High Accuracy and Balanced Performance

The Bidirectional Encoder Representations from Transformers (BERT) model is used in this work to analyse sentiment on Twitter data. A Kaggle dataset of manually annotated and anonymized COVID-19-related tweets was used to refine the model. Location, tweet date, original tweet content, and sentiment labels are all included in the dataset. When compared to the Multinomial Naive Bayes (MNB) baseline, BERT's performance was assessed, and it achieved an overall accuracy of 87% on the test set. The results indicated that for negative feelings, the accuracy was 0.93, the recall was 0.84, and the F1-score was 0.88; for neutral sentiments, the precision was 0.86, the recall was 0.78, and the F1-score was 0.82; and for positive sentiments, the precision was 0.82, the recall was 0.94, and the F1-score was 0.88. The model's proficiency with the linguistic nuances of Twitter, including slang and sarcasm, was demonstrated. This study also identifies the flaws of BERT and makes recommendations for future research paths, such as the integration of external knowledge and alternative designs.

Sentiment Analysis on a Large Indonesian Product Review Dataset

Background: The publicly available large dataset plays an important role in the development of the natural language processing/computational linguistic research field. However, up to now, there are only a few large Indonesian language datasets accessible for research purposes, including sentiment analysis datasets, where sentiment analysis is considered the most popular task. Objective: The objective of this work is to present sentiment analysis on a large Indonesian product review dataset, employing various features and methods. Two tasks have been implemented: classifying reviews into three classes (positive, negative, neutral), and predicting ratings. Methods: Sentiment analysis was conducted on the FDReview dataset, comprising over 700,000 reviews. The analysis treated sentiment as a classification problem, employing the following methods: Multinomial Naïve Bayes (MNB), Support Vector Machine (SVM), LSTM, and BiLSTM. Result: The experimental results indicate that in the comparison of performance using conventional methods, MNB outperformed SVM in rating prediction, whereas SVM exhibited better performance in the review classification task. Additionally, the results demonstrate that the BiLSTM method outperformed all other methods in both tasks. Furthermore, this study includes experiments conducted on balanced and unbalanced small-sized sample datasets. Conclusion: Analysis of the experimental results revealed that the deep learning-based method performed better only in the large dataset setting. Results from the small balanced dataset indicate that conventional machine learning methods exhibit competitive performance compared to deep learning approaches. Keywords: Indonesian review dataset, Large dataset, Rating prediction, Sentiment analysis