Modified VS3-NET for Reading Comprehension and Question Answering with No-Answers

Abstract

Span-based machine reading comprehension of question answering (MRQA) is basically composed of an encoding module for the question and passage respectively, and a question-passage matching module to locate the answer in the passage using an attention mechanism. Recently, the newly published SQuAD 2.0 is a span-based MRQA for dealing with unanswerable questions. In this paper, we solve three issues of current practices in SQuAD 2.0: (1) modules that encode slowly, (2) type-unaware questions encoding module with unanswerable questions, and (3) a non-compositional matching module. We propose a modified VS3-NET model. This modified model is based on co-attention with contextual embedding using simple recurrent unit (SRU) instead of other recurrent neural network (RNN) to speed up training and test while retaining the recurrency of the model. Moreover, it uses variational inference to infer the question type with the unanswerable questions, uses a hierarchical matching and scoring module, and uses a gated module for features and a context vector. Experiments show that the modified VS3-NET provides outstanding improvements over the base model performance and produces performances competitive with the state-of-the-art models on SQuAD v2.0.