Comparing the accuracy of ANN with transformer models for sentiment analysis of tweets related to COVID-19 Pfizer vaccines

Abstract

The current study underscores the critical importance of understanding public sentiment towards vaccination efforts, particularly to achieve the widespread vaccine uptake necessary for public health security, especially in light of pandemics like COVID-19. Despite the clear critical role vaccinations play in mitigating pandemic spread, vaccine hesitancy persists as a formidable challenge. This research focuses on analyzing sentiments related to the Pfizer-BioNTech COVID-19 vaccine using Natural Language Processing (NLP) techniques, offering a comparative assessment of Artificial Neural Network (ANN) frameworks and transformer-based models. The study employs the Valence Aware Dictionary and sEntiment Reasoner (VADER) for sentiment quantification and TensorFlow for text vectorization. Within the ANN domain, both Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) models are investigated, alongside the ALBERT and Distilbert models as a representative of transformer-based architectures. The empirical analysis reveals a distinct advantage of transformer-based models over ANN frameworks in accuracy, with the ALBERT model exhibiting exceptional skill in classifying sentiments of tweets concerning the COVID-19 vaccine by Pfizer. The study meticulously employs Receiver Operating Characteristic (ROC) curve analysis to rank the performance of the evaluated models in sentiment classification, establishing ALBERT as the foremost model, followed in order by LSTM-CNN, Distilbert, CNN, and LSTM models. The ALBERT model demonstrates outstanding performance across critical evaluation metrics. The findings advocate for the strategic use of advanced NLP technologies in public health initiatives to better understand and respond to public attitudes towards vaccination, ultimately contributing to improved health outcomes and pandemic preparedness.