Machine Reading Comprehension on Multiclass Questions Using Bidirectional Attention Flow Models with Contextual Embeddings and Transfer Learning in Thai Corpus

Abstract

In this paper, we develop a machine reading comprehension (MRC) model for Thai corpus provided by National Electronics and Computer Technology Center (NECTEC) under Thailand's 22nd National Software Contest (NSC2019). The dataset consists of 17,000 question-answer pairs which can be classified as (1) factoid and (2) yes-no questions. In this paper, we propose a deep learning architecture that supports multiclass questions based on Bidirectional Attention Flow model (BIDAF), one of recent machine reading comprehension models. It originally supports only factoid questions. To obtain the best performance, various architectures have been compared including (1) single model with special tokens, (2) joint model, and (3) cascading model. To further improve accuracy, there are two more contributions in our framework. First, contextualized word embeddings are built from a pre-trained language model called Bidirectional Encoder Representations from Transformers (BERT) to be used with our model. Second, transfer learning from a natural language inference (NLI) dataset has been employed. The experimental results show that our model significantly outperforms a baseline on both types of questions.