Abstract
Machine reading comprehension is a very challenging task, which aims to determine the answer span based on the given context and question. The newly developed pre-training language model has achieved a series of successes in various natural language understanding tasks with its powerful contextual representation ability. However, these pre-training language models generally lack the downstream processing structure for specific tasks, which limits further performance improvement. In order to solve this problem and deepen the model’s understanding of the question and context, this paper proposes S&I Reader. On the basis of the pre-training model, skimming, intensive reading, and gated mechanism modules are added to simulate the behavior of humans reading text and filtering information. Based on the idea of granular computing, a multi-granularity module for computing context granularity and sequence granularity is added to the model to simulate the behavior of human beings to understand the text from words to sentences, from parts to the whole. Compared with the previous machine reading comprehension model, our model structure is novel. The skimming module and multi-granularity module proposed in this paper are used to solve the problem that the previous model ignores the key information of the text and cannot understand the text with multi granularity. Experiments show that the model proposed in this paper is effective for both Chinese and English datasets. It can better understand the question and context and give a more accurate answer. The performance has made new progress on the basis of the baseline model.
Highlights
INTRODUCTIONImportant books must be read over and over again, and every time you read it, you will find it beneficial to open the book
Important books must be read over and over again, and every time you read it, you will find it beneficial to open the book.Jules Renard(1864-1910) Machine reading comprehension (MRC) is a basic and challenging task in natural language processing
The model focuses on developing the downstream processing structure of the pre-training model to solve the performance bottleneck caused by the development of the current pre-training model
Summary
Important books must be read over and over again, and every time you read it, you will find it beneficial to open the book. People can directly benefit from multiple powerful encoders with similar structures, leading to a bottleneck in the development of downstream processing structures tailored to specific tasks It is time-consuming and resource-consuming to encode the general knowledge contained in large-scale corpus into language models with ultra-large-scale parameters. Due to the slow development of language representation encoding technology, the performance of the pre-training language model is limited These all highlight the importance of developing downstream processing structures for specific tasks. This paper proposes S&I Reader, which aims to help the model determine the valid information in the question and context, and understand the text in terms of word granularity, context granularity, and sequence granularity. Skimming Reading Module, which is used to determine the keywords in question and context and their corresponding related parts, helping the model pay attention to the important content of the text. Ablation experiments and experimental analysis prove that this model is more effective for grasping the key information of the text and solving the over-stability problem
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
