Machine Reading Comprehension Research Articles

TSQA: Tabular Scenario Based Question Answering

Scenario-based question answering (SQA) has attracted an increasing research interest. Compared with the well-studied machine reading comprehension (MRC), SQA is a more challenging task: a scenario may contain not only a textual passage to read but also structured data like tables, i.e., tabular scenario based question answering (TSQA). AI applications of TSQA such as answering multiple-choice questions in high-school exams require synthesizing data in multiple cells and combining tables with texts and domain knowledge to infer answers. To support the study of this task, we construct GeoTSQA. This dataset contains 1k real questions contextualized by tabular scenarios in the geography domain. To solve the task, we extend state-of-the-art MRC methods with TTGen, a novel table-to-text generator. It generates sentences from variously synthesized tabular data and feeds the downstream MRC method with the most useful sentences. Its sentence ranking model fuses the information in the scenario, question, and domain knowledge. Our approach outperforms a variety of strong baseline methods on GeoTSQA.

KAAPA: Knowledge Aware Answers from PDF Analysis

We present KaaPa (Knowledge Aware Answers from Pdf Analysis), an integrated solution for machine reading comprehension over both text and tables extracted from PDFs. KaaPa enables interactive question refinement using facets generated from an automatically induced Knowledge Graph. In addition it provides a concise summary of the supporting evidence for the provided answers by aggregating information across multiple sources. KaaPa can be applied consistently to any collection of documents in English with zero domain adaptation effort. We showcase the use of KaaPa for QA on scientific literature using the COVID-19 Open Research Dataset.

Read, Retrospect, Select: An MRC Framework to Short Text Entity Linking

Entity linking (EL) for the rapidly growing short text (e.g. search queries and news titles) is critical to industrial applications. Most existing approaches relying on adequate context for long text EL are not effective for the concise and sparse short text. In this paper, we propose a novel framework called Multi-turn Multiple-choice Machine reading comprehension (M3) to solve the short text EL from a new perspective: a query is generated for each ambiguous mention exploiting its surrounding context, and an option selection module is employed to identify the golden entity from candidates using the query. In this way, M3 framework sufficiently interacts limited context with candidate entities during the encoding process, as well as implicitly considers the dissimilarities inside the candidate bunch in the selection stage. In addition, we design a two-stage verifier incorporated into M3 to address the commonly existed unlinkable problem in short text. To further consider the topical coherence and interdependence among referred entities, M3 leverages a multi-turn fashion to deal with mentions in a sequence manner by retrospecting historical cues. Evaluation shows that our M3 framework achieves the state-of-the-art performance on five Chinese and English datasets for the real-world short text EL.

A Multi-Perspective Co-Matching Model for Machine Reading Comprehension

A Multi-Perspective Co-Matching Model for Machine Reading Comprehension

A neuro-symbolic method for understanding free-text medical evidence.

ObjectiveWe introduce Medical evidence Dependency (MD)–informed attention, a novel neuro-symbolic model for understanding free-text clinical trial publications with generalizability and interpretability.Materials and MethodsWe trained one head in the multi-head self-attention model to attend to the Medical evidence Ddependency (MD) and to pass linguistic and domain knowledge on to later layers (MD informed). This MD-informed attention model was integrated into BioBERT and tested on 2 public machine reading comprehension benchmarks for clinical trial publications: Evidence Inference 2.0 and PubMedQA. We also curated a small set of recently published articles reporting randomized controlled trials on COVID-19 (coronavirus disease 2019) following the Evidence Inference 2.0 guidelines to evaluate the model’s robustness to unseen data. ResultsThe integration of MD-informed attention head improves BioBERT substantially in both benchmark tasks—as large as an increase of +30% in the F1 score—and achieves the new state-of-the-art performance on the Evidence Inference 2.0. It achieves 84% and 82% in overall accuracy and F1 score, respectively, on the unseen COVID-19 data.Conclusions MD-informed attention empowers neural reading comprehension models with interpretability and generalizability via reusable domain knowledge. Its compositionality can benefit any transformer-based architecture for machine reading comprehension of free-text medical evidence.

Biomedical named entity recognition using BERT in the machine reading comprehension framework

Recognition of biomedical entities from literature is a challenging research focus, which is the foundation for extracting a large amount of biomedical knowledge existing in unstructured texts into structured formats. Using the sequence labeling framework to implement biomedical named entity recognition (BioNER) is currently a conventional method. This method, however, often cannot take full advantage of the semantic information in the dataset, and the performance is not always satisfactory. In this work, instead of treating the BioNER task as a sequence labeling problem, we formulate it as a machine reading comprehension (MRC) problem. This formulation can introduce more prior knowledge utilizing well-designed queries, and no longer need decoding processes such as conditional random fields (CRF). We conduct experiments on six BioNER datasets, and the experimental results demonstrate the effectiveness of our method. Our method achieves state-of-the-art (SOTA) performance on the BC4CHEMD, BC5CDR-Chem, BC5CDR-Disease, NCBI-Disease, BC2GM and JNLPBA datasets, achieving F1-scores of 92.92%, 94.19%, 87.83%, 90.04%, 85.48% and 78.93%, respectively.

EVALUATION OF SINGLE-SPAN MODELS ON EXTRACTIVE MULTI-SPAN QUESTION-ANSWERING

Machine Reading Comprehension (MRC), particularly extractive close-domain question-answering, is a prominent field in Natural Language Processing (NLP). Given a question and a passage or set of passages, a machine must be able to extract the appropriate answer from the passage(s). However, the majority of these existing questions have only one answer, and more substantial testing on questions with multiple answers, or multi-span questions, has not yet been applied.

Integrate Candidate Answer Extraction with Re-Ranking for Chinese Machine Reading Comprehension.

Machine Reading Comprehension (MRC) research concerns how to endow machines with the ability to understand given passages and answer questions, which is a challenging problem in the field of natural language processing. To solve the Chinese MRC task efficiently, this paper proposes an Improved Extraction-based Reading Comprehension method with Answer Re-ranking (IERC-AR), consisting of a candidate answer extraction module and a re-ranking module. The candidate answer extraction module uses an improved pre-training language model, RoBERTa-WWM, to generate precise word representations, which can solve the problem of polysemy and is good for capturing Chinese word-level features. The re-ranking module re-evaluates candidate answers based on a self-attention mechanism, which can improve the accuracy of predicting answers. Traditional machine-reading methods generally integrate different modules into a pipeline system, which leads to re-encoding problems and inconsistent data distribution between the training and testing phases; therefore, this paper proposes an end-to-end model architecture for IERC-AR to reasonably integrate the candidate answer extraction and re-ranking modules. The experimental results on the Les MMRC dataset show that IERC-AR outperforms state-of-the-art MRC approaches.

Enhancing Transformer-based language models with commonsense representations for knowledge-driven machine comprehension

Compared to the traditional machine reading comprehension (MRC) with limitation to the information in a passage, knowledge-driven MRC tasks aim to enable models to answer the question according to text and related commonsense knowledge. Although pre-trained Transformer-based language models (TrLMs) such as BERT and Roberta, have shown powerful performance in MRC, external knowledge such as unspoken commonsense and world knowledge still cannot be used and explained explicitly. In this work, we present three simple yet effective injection methods integrated into the structure of TrLMs to fine-tune downstream knowledge-driven MRC tasks with off-the-shelf commonsense representations. Moreover, we introduce a mask mechanism for a token-level multi-hop relationship searching to filter external knowledge. We have conducted extensive experiments on DREAM and CosmosQA, two prevalent knowledge-driven datasets. Experimental results indicate that the incremental TrLMs have outperformed the baseline systems by 1%-4.1% with a fewer computational cost. Further analysis shows the effectiveness of the proposed methods and the robustness of the incremental model in the case of an incomplete training set.

Frame-based Neural Network for Machine Reading Comprehension

Machine Reading Comprehension (MRC) is one of the most challenging tasks in Natural Language Understanding (NLU). In particular, MRC systems typically answer a question by only utilizing the information contained in a given piece of text passage itself, while human beings can easily understand the meanings of the passage based on their background knowledge. To bridge the gap, we propose a novel Frame-based Neural Network for Machine Reading Comprehension(FNN-MRC) method, which employs Frame semantic knowledge to facilitate question answering. Specifically, different from existing Frame based methods that only model lexical units (LUs), our FNN-MRC has a Frame representation model, which utilizes both LUs in Frame and Frame-to-Frame (F-to-F) relations, designed to model Frames and sentences (in passage) together with attention schema. In addition, FNN-MRC has a Frame-based Sentence Representation (FSR) model, which is able to integrate multiple-Frame semantic information to obtain much better sentence representation. As such, FNN-MRC explicitly leverages the above Frame knowledge to assist its semantic understanding and representation. Extensive experiments demonstrate that our FNN-MRC method is able to achieve better results than existing state-of-the-art techniques across multiple datasets.

Evaluation of Single-Span Models on Extractive Multi-Span Question-Answering

Machine Reading Comprehension (MRC), particularly extractive close-domain question-answering, is a prominent field in Natural Language Processing (NLP). Given a question and a passage or set of passages, a machine must be able to extract the appropriate answer from the passage(s). However, the majority of these existing questions have only one answer, and more substantial testing on questions with multiple answers, or multi-span questions, has not yet been applied. Thus, we introduce a newly compiled dataset consisting of questions with multiple answers that originate from previously existing datasets. In addition, we run BERT-based models pre-trained for question-answering on our constructed dataset to evaluate their reading comprehension abilities. Runtime of base models on the entire dataset is approximately one day while the runtime for all models on a third of the dataset is a little over two days. Among the three of BERT-based models we ran, RoBERTa exhibits the highest consistent performance, regardless of size. We find that all our models perform similarly on this new, multi-span dataset compared to the single-span source datasets. While the models tested on the source datasets were slightly fine-tuned in order to return multiple answers, performance is similar enough to judge that task formulation does not drastically affect question-answering abilities. Our evaluations indicate that these models are indeed capable of adjusting to answer questions that require multiple answers. We hope that our findings will assist future development in question-answering and improve existing question-answering products and methods.

PH-model: enhancing multi-passage machine reading comprehension with passage reranking and hierarchical information

Machine reading comprehension(MRC), which employs computers to answer questions from given passages, is a popular research field. In natural language, a natural hierarchical representation can be seen: characters, words, phrases, sentences, paragraphs, and documents. Current studies have demonstrated that hierarchical information can help machines understand natural language. However, prior works focused on the overall performance of MRC tasks without considering hierarchical information. In addition, the noise problem still has not been adequately addressed, even though many researchers have adopted the technique of passage reranking. Thus, in this paper, focusing on noise information processing and the extraction of hierarchical information, we propose a model (PH-Model) with a passage reranking framework (P) and hierarchical neural network (H) for a Chinese multi-passage MRC task. PH-Model produces more precise answers by reducing noise information and extracting hierarchical information. Experimental results on the DuReader 2.0 dataset (a large scale real-world Chinese MRC dataset) show that PH-Model outperforms the ROUGE-L and BLEU-4 baseline by 18.24% and 24.17%, respectively.

Retracted] Analyzing the Effect of Masking Length Distribution of MLM: An Evaluation Framework and Case Study on Chinese MRC Datasets

Machine reading comprehension (MRC) is a challenging natural language processing (NLP) task. It has a wide application potential in the fields of question answering robots, human‐computer interactions in mobile virtual reality systems, etc. Recently, the emergence of pretrained models (PTMs) has brought this research field into a new era, in which the training objective plays a key role. The masked language model (MLM) is a self‐supervised training objective widely used in various PTMs. With the development of training objectives, many variants of MLM have been proposed, such as whole word masking, entity masking, phrase masking, and span masking. In different MLMs, the length of the masked tokens is different. Similarly, in different machine reading comprehension tasks, the length of the answer is also different, and the answer is often a word, phrase, or sentence. Thus, in MRC tasks with different answer lengths, whether the length of MLM is related to performance is a question worth studying. If this hypothesis is true, it can guide us on how to pretrain the MLM with a relatively suitable mask length distribution for MRC tasks. In this paper, we try to uncover how much of MLM’s success in the machine reading comprehension tasks comes from the correlation between masking length distribution and answer length in the MRC dataset. In order to address this issue, herein, (1) we propose four MRC tasks with different answer length distributions, namely, the short span extraction task, long span extraction task, short multiple‐choice cloze task, and long multiple‐choice cloze task; (2) four Chinese MRC datasets are created for these tasks; (3) we also have pretrained four masked language models according to the answer length distributions of these datasets; and (4) ablation experiments are conducted on the datasets to verify our hypothesis. The experimental results demonstrate that our hypothesis is true. On four different machine reading comprehension datasets, the performance of the model with correlation length distribution surpasses the model without correlation.

Machine Reading Comprehension Framework Based on Self-Training for Domain Adaptation

Machine reading comprehension (MRC) is a type of question answering mechanism in which a computer reads documents and answers related questions. The accuracies of recent MRC systems surpass those of humans. However, most MRC systems exhibit significant performance deteriorations when domains are changed. Hence, we propose a self-training framework for MRC. The proposed framework is composed of a pseudo-answer extractor, a pseudo-question generator, and an MRC system. In the source domain, components are pretrained using an MRC training dataset. In the target domain, the performances of the pseudo-question generator and MRC system is improved through a mutual self-training scheme. During the mutual self-training, the pseudo-question generator provides new training data to the MRC system and obtains rewards from the MRC system for reinforcement learning. In experiments with a Wikipedia domain (source domain) and civil affair domain (target domain), an MRC system based on the proposed self-training scheme demonstrates better performances than that based on automatic data augmentation.

Teaching Machines to Read and Comprehend Tibetan Text

Teaching machine to comprehend a passage and answer corresponding questions, the machine reading comprehension (MRC) has attracted much attention in current years. However, most models are designed to finish English or Chinese MRC task, Considering lack of MRC dataset, the low-resource languages MRC tasks, such as Tibetan, it is hard to get high performance. To solve this problem, this paper constructs a span-style Tibetan MRC dataset named TibetanQA and proposes a hierarchical attention network model for Tibetan MRC task which includes word-level attention and re-read attention. And the experiments prove the effectiveness of our model.

S&I Reader: Multi-Granularity Gated Multi-Hop Skimming and Intensive Reading Model for Machine Reading Comprehension

Machine reading comprehension is a very challenging task, which aims to determine the answer span based on the given context and question. The newly developed pre-training language model has achieved a series of successes in various natural language understanding tasks with its powerful contextual representation ability. However, these pre-training language models generally lack the downstream processing structure for specific tasks, which limits further performance improvement. In order to solve this problem and deepen the model’s understanding of the question and context, this paper proposes S&I Reader. On the basis of the pre-training model, skimming, intensive reading, and gated mechanism modules are added to simulate the behavior of humans reading text and filtering information. Based on the idea of granular computing, a multi-granularity module for computing context granularity and sequence granularity is added to the model to simulate the behavior of human beings to understand the text from words to sentences, from parts to the whole. Compared with the previous machine reading comprehension model, our model structure is novel. The skimming module and multi-granularity module proposed in this paper are used to solve the problem that the previous model ignores the key information of the text and cannot understand the text with multi granularity. Experiments show that the model proposed in this paper is effective for both Chinese and English datasets. It can better understand the question and context and give a more accurate answer. The performance has made new progress on the basis of the baseline model.

Analysis of English Multitext Reading Comprehension Model Based on Deep Belief Neural Network.

In order to solve the problems of low accuracy and low efficiency of answer prediction in machine reading comprehension, a multitext English reading comprehension model based on the deep belief neural network is proposed. Firstly, the paragraph selector in the multitext reading comprehension model is constructed. Secondly, the text reader is designed, and the deep belief neural network is introduced to predict the question answering probability. Finally, the popular English dataset of SQuAD is used for test analysis. The final results show that, after the comparative analysis of different learning methods, it is found that the English multitext reading comprehension model has a strong reading comprehension ability. In addition, two evaluation methods are used to score the overall performance of the model, which shows that the overall score of the English multitext reading comprehension model based on the deep confidence neural network is more than 90, and the efficiency will not be reduced because of the change of the number of documents in the dataset. The above results show that the use of the deep belief neural network to improve the probability generation performance of the model can well solve the task of English multitext reading comprehension, effectively reduce the difficulty of machine reading comprehension in multitask reading, and has a good guiding significance for promoting human convenient Internet knowledge acquisition.

Enhancement of Target‐Oriented Opinion Words Extraction with Multiview‐Trained Machine Reading Comprehension Model

Target-oriented opinion words extraction (TOWE) seeks to identify opinion expressions oriented to a specific target, and it is a crucial step toward fine-grained opinion mining. Recent neural networks have achieved significant success in this task by building target-aware representations. However, there are still two limitations of these methods that hinder the progress of TOWE. Mainstream approaches typically utilize position indicators to mark the given target, which is a naive strategy and lacks task-specific semantic meaning. Meanwhile, the annotated target-opinion pairs contain rich latent structural knowledge from multiple perspectives, but existing methods only exploit the TOWE view. To tackle these issues, we formulate the TOWE task as a question answering (QA) problem and leverage a machine reading comprehension (MRC) model trained with a multiview paradigm to extract targeted opinions. Specifically, we introduce a template-based pseudo-question generation method and utilize deep attention interaction to build target-aware context representations and extract related opinion words. To take advantage of latent structural correlations, we further cast the opinion-target structure into three distinct yet correlated views and leverage meta-learning to aggregate common knowledge among them to enhance the TOWE task. We evaluate the proposed model on four benchmark datasets, and our method achieves new state-of-the-art results. Extensional experiments have shown that the pipeline method with our approach could surpass existing opinion pair extraction models, including joint methods that are usually believed to work better.

Exploring Machine Reading Comprehension for Continuous Questions via Subsequent Question Completion

In recent years, the Sq-MRC (machine reading comprehension for separate questions) task, where the questioner poses a separate question each time, has experienced rapid development. However, we often encounter circumstances where the questioner poses a series of nonstop questions. The corresponding task, called Cq-MRC (machine reading comprehension for continuous questions), has been rarely investigated and is nontrivial due to (1) the incomplete expressions of subsequent questions, such as anaphora or subject ellipsis, and (2) the shortage of annotated samples for model training and performance evaluation. We explore this challenging task, in this article, and propose a completion-then-prediction approach. To tackle the first issue, we identify the key-entity in the first question and employ it to complete the subsequent ones, and hence decompose the task into a set of Sq-MRC tasks which are accomplished using a BERT-based model. In addition, to mitigate the shortage of annotated samples, we train a meta-learner on the Sq-MRC task which is related to Cq-MRC and has abundant labeled data, and then perform few-shot learning by fine-tuning the model on our target task, viz. Cq-MRC, using the limited number of samples available. The results of extensive experiments conducted on a multi-document Chinese machine reading comprehension dataset show the effectiveness and superiority of our method, achieving 2.3% to 3.7% absolute improvements in terms of major metrics.

QANet-based candidate answer rethink model for machine reading comprehension

The current model applied to the span extraction reading comprehension task fuses the information of context and question, and outputs the index with the highest probability calculated in the context as the prediction span. In this process, the model discards all the remaining candidate answers, which results in a waste of the available information in the candidate answers. Our model is designed to simulate the behaviour of human beings choosing multiple candidate answers and comprehensively judging the final answer in reading comprehension tasks. We propose the QANet-based candidate answer rethink model. The model interacts and fuses multiple candidate answers with context and question, prompting the model to obtain a more accurate answer by synthesising these three aspects of information. Experiments show that our model has made new progress in performance.