Analysis of Variants of BERT and Big Bird on Question Answering Datasets in the Context of Scientific Research Article Reviews

Abstract

AbstractBERT and Big Bird are some of the greatest successful representation models in the field of natural language processing (NLP) in the recent past. Question answering comes as a significant part when comes to automating the scientific research article reviewing process. No major research study is available how BERT, Big Bird, and its variants could be efficiently utilized for question answering (QA)-based problems in the context of automating scientific research article reviews. This article focuses on a detailed study of NLP-based state-of-the-art methods, on question answering datasets and its usage in the field. The major insights and observations identified are mentioned for helping the researchers in the similar field of study. This work is conducted as a base work to identify how well these transformer-based methods and QA datasets can be applied for building an efficient and automated research article review system in the near future.KeywordsBERTBig BirdTransformersScientific research article reviewQuestion answeringNLP