Multi-task joint training model for machine reading comprehension

Abstract

Machine reading comprehension is an important topic in natural language processing. However, the redundancy of text and the diversity of answer types are neglected in existing machine reading comprehension. To tackle these issues, a multi-task joint training scheme is proposed for the multi-type answer machine reading comprehension task. In this scheme, a feature-based paragraph extraction mechanism is first designed in the training stage to extract various text features from answers and sentences. For efficiency, the redundancy of the text is greatly reduced, while the effective information of the text is retained. And then, a reading comprehension module is optimized by enhancing the representation of the pre-trained language model, as well as an answer type classification module is optimized through the capsule network. Finally, a multi-task joint training model is designed to simultaneous obtaining answer text and answer type. Experiments are conducted to evaluate the proposed approach with two open datasets, CJRC and Natural Questions, and demonstrate that our model is efficient and promising, in terms of both the reading comprehension and answer classification tasks. In particular, our model won the best method that is winner of the first prize in the CAIL-2020 machine reading comprehension contest.