Comparative Analysis of State-of-the-Art Q A Models: BERT, RoBERTa, DistilBERT, and ALBERT on SQuAD v2 Dataset

Abstract

In the rapidly evolving landscape of natural language processing (NLP) and artificial intelligence, recent years have witnessed significant advancements, particularly in text-based question-answering (QA) systems. The Stanford Question Answering Dataset (SQuAD v2) has emerged as a prominent benchmark, offering diverse language understanding challenges. This study conducts a thorough examination of cutting-edge QA models—BERT, DistilBERT, RoBERTa, and ALBERT—each featuring distinct architectures, focusing on their training and performance on SQuAD v2. The analysis aims to uncover the unique strengths of each model, providing insights into their capabilities and exploring the impact of different training techniques on their performance. The primary objective is to enhance our understanding of text-based QA systems' evolution and their effectiveness in real-world scenarios. The results of this comparative study are poised to influence the utilization and development of these models in both industry and research. The investigation meticulously evaluates BERT, ALBERT, RoBERTa, and DistilBERT QA models using the SQuAD v2 dataset, emphasizing instances of accurate responses and identifying areas where completeness may be lacking. This nuanced exploration contributes to the ongoing discourse on the advancement of text-based question-answering systems, shedding light on the strengths and limitations of each QA model. Based on the results obtained, ALBERT achieved an exact match of 86.85% and an F1 score of 89.91% on the SQuAD v2 dataset, demonstrating superior performance in both answerable ('HasAns') and unanswerable ('NoAns') questions. BERT and RoBERTa also showed strong performance, while DistilBERT lagged slightly behind. This study provides a significant contribution to the advancement of text-based question-answering systems, offering insights that can shape the utilization of these models in both industry and research domains.