Machine Reading Comprehension Models Research Articles

JEEMRC: Joint Event Detection and Extraction via an End-to-End Machine Reading Comprehension Model

Event extraction (EE) generally contains two subtasks: viz., event detection and argument extraction. Owing to the success of machine reading comprehension (MRC), some researchers formulate EE into MRC frameworks. However, existing MRC-based EE techniques are pipeline methods that suffer from error propagation. Moreover, the correlation between event types and argument roles is pre-defined by experts, which is time-consuming and inflexible. To avoid these issues, event detection and argument extraction are formalized as joint MRC. Different from previous methods, which just generate questions for argument roles for identified event types, questions are generated for all arguments that appear in the given sentence in our approach. Moreover, an end-to-end MRC model, JEEMRC, is proposed, which consists of an event classifier and a machine reader with a coarse-to-fine attention mechanism. Our proposed model can train two subtasks jointly to alleviate error propagation and utilizes interaction information between event types and argument roles to improve the performance of both tasks. Experiments on ACE 2005 verify that our JEEMRC achieves competitive results compared with previous work. In addition, it performs well when detecting events and extracting arguments in data-scarce scenarios.

Combining permuted language model and adversarial training for Chinese machine reading comprehension

Due to the polysemy and complexity of the Chinese language, Chinese machine reading comprehension has always been a challenging task. To improve the semantic understanding and robustness of Chinese machine reading comprehension models, we propose a model that utilizes adversarial training algorithms and Permuted Language Model (PERT). Firstly, we employ the PERT pre-training model to embed paragraphs and questions into vector space to obtain corresponding sequential representations. Secondly, we use a multi-head self-attention mechanism to extract key textual information from the sequence and employ a Bi-GRU network to semantically fuse the output feature vectors, aiming to learn deep semantic representations in the text. Finally, we introduce perturbations into the model training process. We achieve this by utilizing adversarial training algorithms such as Fast Gradient Method (FGM) and Projected Gradient Descent (PGD). These algorithms generate adversarial samples to enhance the model’s robustness and stability when facing diverse inputs. We conducted comparative experiments on the publicly available Chinese reading comprehension datasets CMRC2018 and DRCD. The experimental results show that our proposed model has achieved significant improvements in both EM and F1-Score compared to the baseline model. To validate the model’s generalization and robustness, we utilized ChatGPT to construct a scientific dataset that includes a large number of domain-specific terms, sentences with mixed Chinese and English, and complex comprehension tasks. Our model also performed remarkably well on the self-built dataset. In conclusion, the proposed model not only effectively enhances the understanding of semantic information in Chinese text but also demonstrates a certain level of generalization capability.

面向机器阅读理解的医学域数据集MedicalQA

Machine reading comprehension aims to make the computer understand the paragraph semantics and answer the questions raised by users using algorithms. The quality of the dataset used in this task can directly affect the experimental results of the model. In order to enrich the medical domain dataset of machine reading comprehension, this paper constructs MedicalQA, a medical domain dataset for machine reading comprehension, employing a combination of web crawlers and manual annotation techniques. The dataset takes two medical platforms (i.e. Xunyiwenyao Network and 39 Health Network) as main data sources, and includes 19,502 paragraphs and Q & A pairs, covering 9 medical departments, such as internal medicine, surgery, obstetrics and gynecology. The dataset is formatted as an Excel file, organized with 5e columns. The first column denotes the paragraph ID; the second column indicates the department to which the paragraph belongs; the third column contains the paragraph content; the fourth column lists the questions, and the fifth column provides corresponding answers to the questions. The construction of this dataset is conducive to the establishment of machine reading comprehension models in the medical domain, and can also promote the sharing of medical datasets in the field of machine reading comprehension.

Efficient Machine Reading Comprehension for Health Care Applications: Algorithm Development and Validation of a Context Extraction Approach.

Extractive methods for machine reading comprehension (MRC) tasks have achieved comparable or better accuracy than human performance on benchmark data sets. However, such models are not as successful when adapted to complex domains such as health care. One of the main reasons is that the context that the MRC model needs to process when operating in a complex domain can be much larger compared with an average open-domain context. This causes the MRC model to make less accurate and slower predictions. A potential solution to this problem is to reduce the input context of the MRC model by extracting only the necessary parts from the original context. This study aims to develop a method for extracting useful contexts from long articles as an additional component to the question answering task, enabling the MRC model to work more efficiently and accurately. Existing approaches to context extraction in MRC are based on sentence selection strategies, in which the models are trained to find the sentences containing the answer. We found that using only the sentences containing the answer was insufficient for the MRC model to predict correctly. We conducted a series of empirical studies and observed a strong relationship between the usefulness of the context and the confidence score output of the MRC model. Our investigation showed that a precise input context can boost the prediction correctness of the MRC and greatly reduce inference time. We proposed a method to estimate the utility of each sentence in a context in answering the question and then extract a new, shorter context according to these estimations. We generated a data set to train 2 models for estimating sentence utility, based on which we selected more precise contexts that improved the MRC model's performance. We demonstrated our approach on the Question Answering Data Set for COVID-19 and Biomedical Semantic Indexing and Question Answering data sets and showed that the approach benefits the downstream MRC model. First, the method substantially reduced the inference time of the entire question answering system by 6 to 7 times. Second, our approach helped the MRC model predict the answer more correctly compared with using the original context (F1-score increased from 0.724 to 0.744 for the Question Answering Data Set for COVID-19 and from 0.651 to 0.704 for the Biomedical Semantic Indexing and Question Answering). We also found a potential problem where extractive transformer MRC models predict poorly despite being given a more precise context in some cases. The proposed context extraction method allows the MRC model to achieve improved prediction correctness and a significantly reduced MRC inference time. This approach works technically with any MRC model and has potential in tasks involving processing long texts.

A machine reading comprehension model with counterfactual contrastive learning for emotion-cause pair extraction

A machine reading comprehension model with counterfactual contrastive learning for emotion-cause pair extraction

Chinese EMR Named Entity Recognition Using Fused Label Relations Based on Machine Reading Comprehension Framework.

Chinese electronic medical records (EMR) presents significant challenges for named entity recognition (NER) due to their specialized nature, unique language features, and diverse expressions. Traditionally, NER is treated as a sequence labeling task, where each token is assigned a label. Recent research has reframed NER within the machine reading comprehension (MRC) framework, extracting entities in a question-answer format, achieving state-of-the-art performance. However, these MRC-based methods have a significant limitation: they extract entities of various types independently, ignoring their interrelations. To address this, we introduce the Fusion Label Relations with MRC (FLR-MRC) model, which enhances the MRC model by implicitly capturing dependencies among entity types. FLR-MRC models interrelations between labels using graph attention networks, integrating these with textual data to identify entities. On the benchmark CMeEE and CCKS2017-CNER datasets, FLR-MRC achieves F1-scores of 0.6652 and 0.9101, respectively, outperforming existing clinical NER methods.

Mixed inference machine reading comprehension method based on symbolic logic

With the rapid development of machine learning, challenging question and answer datasets have also emerged, and the machine reading comprehension technology has emerged. Traditional machine reading comprehension methods mostly focus on the understanding word level semantics, with the weak ability to extract logical relationships from text, resulting in the lower ability of logical reasoning. In order to strengthen the ability of machine reading comprehension method to extract the logical relationship of text and the ability of logical reasoning, a neural symbol model based on logical reasoning was proposed, and the logical expressions captured by the neural symbol model were converted into text input and trained in a mixed reasoning reading comprehension model based on symbolic logic. The mixed reasoning reading comprehension model based on symbolic logic is different from the traditional machine reading comprehension model. It uses symbolic definition and logical capture to extract logical symbols and generate logical expressions. The research results show that the accuracy and F-measure values of the neural symbol model based on the logical reasoning are 70.08% and 70.05%, respectively, when the training set sample size is 4000. The accuracy of the mixed reasoning reading comprehension model based on symbolic logic in the logical reasoning data set of the standard postgraduate entrance examination is 88.31%, which is higher than the 58.74% of the language perception map network model. The accuracy rate in the four-choice and one-choice question-and-answer data set is 40.92%, which is 1.58% higher than that of the language awareness graph network model. In summary, the neural symbol model and hybrid inference reading comprehension model proposed in the study have superior performance, which can capture the logical relationship of text in data sets well, improve the model feature abstraction and reasoning ability, effectively shorten the training time and improve the model efficiency.

A T5-based interpretable reading comprehension model with more accurate evidence training

Pre-trained language models (PLMs) have achieved outstanding performance on Machine Reading Comprehension (MRC) tasks, but these models’ interpretability remains uncertain. In this paper, we exploit the strengths of the pre-trained T5 (Text-to-Text Transfer Transformer) model on evidence inference to improve the interpretability of the MRC model and propose an interpretable reading comprehension model based on T5, which is trained on a more accurate evidence corpus and can infer precise interpretations for the generated answers. First, we propose a novel T5-based Semantic Textual Similarity (STS) model to label training evidences more precisely, including label reconstruction and data augmentation. Then, we propose a T5-based interpretable reading comprehension model with more accurate evidence training, including a threshold-based method to filter out erroneous evidence during model training. Experiments show that our model significantly outperforms the baseline BERT (Pseudo-data Training) model by 8.7 and 8.0 points for the evidence F1-score of SQuAD1.1 respectively in the base and large levels. Our code is available at https://github.com/MN-Guan/T5-InterMRC.

Robustness-Eva-MRC: Assessing and analyzing the robustness of neural models in extractive machine reading comprehension

Deep neural networks, despite their remarkable success in various language understanding tasks, have been found vulnerable to adversarial attacks and subtle input perturbations, revealing a robustness shortfall. To explore this, this paper presents Robustness-Eva-MRC, an interactive platform designed to assess and analyze the robustness of pre-trained and large-scale language models in extractive machine reading comprehension (MRC) tasks. The platform integrates eight adversarial attack methods across character-, word-, and sentence-levels, and applies them to five MRC datasets, thereby fabricating challenging adversarial testing sets. Then it evaluates the MRC models on both original and adversarial sets, yielding insights into their robustness through performance gaps. Moreover, Robustness-Eva-MRC provides comprehensive visualizations and detailed case studies, enhancing the understanding of model robustness. A screencast video and additional material are available at https://github.com/distantJing/Robustness-Eva-MRC.

The DistilBERT Model: A Promising Approach to Improve Machine Reading Comprehension Models

Machine Reading Comprehension (MRC) is a challenging task in the field of Natural Language Processing (NLP), where a machine is required to read a given text passage and answer a set of questions based on it. This paper provides an overview of recent advances in MRC and highlights some of the key challenges and future directions of this research area. It also evaluates the performance of several baseline models on the dataset, evaluates the challenges that the dataset poses for existing MRC models, and introduces the DistilBERT model to improve the accuracy of the answer extraction process. The supervised paradigm for training machine reading and comprehension models represents a practical path forward for creating comprehensive natural language understanding systems. To enhance the DistilBERT basic model's functionality, we have experimented with a variety of question heads that differ in the number of layers, activation function, and general structure. DistilBERT is a model for question-resolution tasks that is successful and delivers state-of-the-art performance while requiring less computational resources than large models like BERT, according to the presented technique. We could enhance the model's functionality and obtain a better understanding of how the model functions by investigating other question head architectures. These findings could serve as a foundation for future study on how to make question-and-answer systems and other tasks connected to the processing of natural languages.

Knowledge Graph based Mutual Attention for Machine Reading Comprehension over Anti-Terrorism Corpus

ABSTRACT Machine reading comprehension has been a research focus in natural language processing and intelligence engineering. However, there is a lack of models and datasets for the MRC tasks in the anti-terrorism domain. Moreover, current research lacks the ability to embed accurate background knowledge and provide precise answers. To address these two problems, this paper first builds a text corpus and testbed that focuses on the anti-terrorism domain in a semi-automatic manner. Then, it proposes a knowledge-based machine reading comprehension model that fuses domain-related triples from a large-scale encyclopedic knowledge base to enhance the semantics of the text. To eliminate knowledge noise that could lead to semantic deviation, this paper uses a mixed mutual attention mechanism among questions, passages, and knowledge triples to select the most relevant triples before embedding their semantics into the sentences. Experiment results indicate that the proposed approach can achieve a 70.70% EM value and an 87.91% F1 score, with a 4.23% and 3.35% improvement over existing methods, respectively.

Robust Domain Adaptation for Machine Reading Comprehension

Most domain adaptation methods for machine reading comprehension (MRC) use a pre-trained question-answer (QA) construction model to generate pseudo QA pairs for MRC transfer. Such a process will inevitably introduce mismatched pairs (i.e., Noisy Correspondence) due to i) the unavailable QA pairs in target documents, and ii) the domain shift during applying the QA construction model to the target domain. Undoubtedly, the noisy correspondence will degenerate the performance of MRC, which however is neglected by existing works. To solve such an untouched problem, we propose to construct QA pairs by additionally using the dialogue related to the documents, as well as a new domain adaptation method for MRC. Specifically, we propose Robust Domain Adaptation for Machine Reading Comprehension (RMRC) method which consists of an answer extractor (AE), a question selector (QS), and an MRC model. Specifically, RMRC filters out the irrelevant answers by estimating the correlation to the document via the AE, and extracts the questions by fusing the candidate questions in multiple rounds of dialogue chats via the QS. With the extracted QA pairs, MRC is fine-tuned and provides the feedback to optimize the QS through a novel reinforced self-training method. Thanks to the optimization of the QS, our method will greatly alleviate the noisy correspondence problem caused by the domain shift. To the best of our knowledge, this could be the first study to reveal the influence of noisy correspondence in domain adaptation MRC models and show a feasible solution to achieve the robustness against the mismatched pairs. Extensive experiments on three datasets demonstrate the effectiveness of our method.

Machine reading comprehension model based on query reconstruction technology and deep learning

Machine reading comprehension model based on query reconstruction technology and deep learning

Clinical concept and relation extraction using prompt-based machine reading comprehension.

To develop a natural language processing system that solves both clinical concept extraction and relation extraction in a unified prompt-based machine reading comprehension (MRC) architecture with good generalizability for cross-institution applications. We formulate both clinical concept extraction and relation extraction using a unified prompt-based MRC architecture and explore state-of-the-art transformer models. We compare our MRC models with existing deep learning models for concept extraction and end-to-end relation extraction using 2 benchmark datasets developed by the 2018 National NLP Clinical Challenges (n2c2) challenge (medications and adverse drug events) and the 2022 n2c2 challenge (relations of social determinants of health [SDoH]). We also evaluate the transfer learning ability of the proposed MRC models in a cross-institution setting. We perform error analyses and examine how different prompting strategies affect the performance of MRC models. The proposed MRC models achieve state-of-the-art performance for clinical concept and relation extraction on the 2 benchmark datasets, outperforming previous non-MRC transformer models. GatorTron-MRC achieves the best strict and lenient F1-scores for concept extraction, outperforming previous deep learning models on the 2 datasets by 1%-3% and 0.7%-1.3%, respectively. For end-to-end relation extraction, GatorTron-MRC and BERT-MIMIC-MRC achieve the best F1-scores, outperforming previous deep learning models by 0.9%-2.4% and 10%-11%, respectively. For cross-institution evaluation, GatorTron-MRC outperforms traditional GatorTron by 6.4% and 16% for the 2 datasets, respectively. The proposed method is better at handling nested/overlapped concepts, extracting relations, and has good portability for cross-institute applications. Our clinical MRC package is publicly available at https://github.com/uf-hobi-informatics-lab/ClinicalTransformerMRC.

GFMRC: A machine reading comprehension model for named entity recognition

Recent advances in natural language representation have enabled the internal state of an upstream trained model to migrate to downstream tasks such as named entity recognition (NER). To better utilize pretrained models to perform NER tasks, the latest approach implements NER using the machine reading comprehension (MRC) framework. However, existing MRC approaches do not consider the limited performance of reading comprehension models due to the absence of contextual information in a single sample. Moreover, only word-level features are employed in the feature extraction phase in existing approaches. In this paper, a novel MRC model named GFMRC is proposed to realize NER. GRMRC enhances the MRC model with contextual information and hybrid features. In the preprocessing stage, the samples of the initial MRC dataset are spliced with N-gram information. In the feature extraction stage, global features are extracted for each token using a CNN, and local features are extracted using LSTM. Experiments are carried out on both Chinese datasets and English datasets, and the results demonstrated the effectiveness of the proposed model. The improvements on the English CoNLL 2003, English OntoNotes 5.0, Chinese MSRA, and Chinese OntoNotes 4.0 datasets are 0.07%, 0.23%, 0.04%, and 0.26%, respectively, compared to BERT-MRC+DSC.

An Event Extraction Approach Based on a Multi-Round Q&A Framework

Event extraction aims to present unstructured text containing event information in a structured form to help people quickly mine the target information. Most of the traditional event extraction methods focus on the design of complex neural network models, which rely on a large amount of annotated data to train the models. In recent years, some researchers have proposed the use of machine reading comprehension models for event extraction; however, the existing methods are limited to the single-round question-and-answer model, ignoring the dependency relation between the elements of event arguments. In addition, the existing methods do not fully utilize knowledge such as a priori information. To address these shortcomings, a multi-round Q&A framework is proposed for event extraction, which extends the existing methods in two aspects: first, by constructing a multi-round extraction problem framework, the model can effectively exploit the hierarchical dependencies among the argument elements; second, the question-and-answer framework is populated with historical answer information encoding slots, which are integrated into the multi-round Q&A process to assist in inference. Finally, experimental results on a publicly available dataset show that the proposed model achieves superior results compared to existing methods.

Information Extraction Network Based on Multi-Granularity Attention and Multi-Scale Self-Learning.

Transforming the task of information extraction into a machine reading comprehension (MRC) framework has shown promising results. The MRC model takes the context and query as the inputs to the encoder, and the decoder extracts one or more text spans as answers (entities and relationships) from the text. Existing approaches typically use multi-layer encoders, such as Transformers, to generate hidden features of the source sequence. However, increasing the number of encoder layers can lead to the granularity of the representation becoming coarser and the hidden features of different words becoming more similar, potentially leading to the model's misjudgment. To address this issue, a new method called the multi-granularity attention multi-scale self-learning network (MAML-NET) is proposed, which enhances the model's understanding ability by utilizing different granularity representations of the source sequence. Additionally, MAML-NET can independently learn task-related information from both global and local dimensions based on the learned multi-granularity features through the proposed multi-scale self-learning attention mechanism. The experimental results on two information extraction tasks, named entity recognition and entity relationship extraction, demonstrated that the method was superior to the method based on machine reading comprehension and achieved the best performance on the five benchmark tests.

A Consistent Dual-MRC Framework for Emotion-cause Pair Extraction

Emotion-cause pair extraction (ECPE) is a recently proposed task that aims to extract the potential clause pairs of emotions and its corresponding causes in a document. In this article, we propose a new paradigm for the ECPE task. We cast the task as a two-turn machine reading comprehension (MRC) task, i.e., the extraction of emotions and causes is transformed to the task of identifying answer clauses from the input document specific to a query. This two-turn MRC formalization brings several key advantages: First, the QA manner provides an explicit pairing way to identify causes specific to the target emotion; second, it provides a natural way of jointly modeling the emotion extraction, the cause extraction, and the pairing of emotion and cause; and third, it allows us to exploit the well-developed MRC models. Based on the two-turn MRC formalization, we propose a dual-MRC framework to extract emotion-cause pairs in a dual-direction way, which enables a more comprehensive coverage of all pairing cases. Furthermore, we propose a consistent training strategy for the second-turn query, so the model is able to filter the errors produced by the first turn at inference. Experiments on two benchmark datasets demonstrate that our method outperforms previous methods and achieves state-of-the-art performance. All the code and data of this work can be obtained at https://github.com/zifengcheng/CD-MRC .

NEREL-BIO: a dataset of biomedical abstracts annotated with nested named entities.

This article describes NEREL-BIO-an annotation scheme and corpus of PubMed abstracts in Russian and smaller number of abstracts in English. NEREL-BIO extends the general domain dataset NEREL by introducing domain-specific entity types. NEREL-BIO annotation scheme covers both general and biomedical domains making it suitable for domain transfer experiments. NEREL-BIO provides annotation for nested named entities as an extension of the scheme employed for NEREL. Nested named entities may cross entity boundaries to connect to shorter entities nested within longer entities, making them harder to detect. NEREL-BIO contains annotations for 700+ Russian and 100+ English abstracts. All English PubMed annotations have corresponding Russian counterparts. Thus, NEREL-BIO comprises the following specific features: annotation of nested named entities, it can be used as a benchmark for cross-domain (NEREL → NEREL-BIO) and cross-language (English → Russian) transfer. We experiment with both transformer-based sequence models and machine reading comprehension models and report their results. The dataset and annotation guidelines are freely available at https://github.com/nerel-ds/NEREL-BIO.

Building a Closed-Domain Question Answering System for a Low-Resource Language

In recent years, the Question Answering System (QAS) has been widely used to develop many systems, such as conversation systems, chatbots, and intelligent search. Depending on the amount of information or knowledge that the system processes, the system can be applied in answering the questions in an open domain or closed domain. There are many approaches to solving the QA problem, but the neural network models have yielded impressive and promising results, especially the Machine Reading Comprehension approach. In this article, we build a closed-domain QAS for a low-resource language, Vietnamese—specifically, “The Postgraduate Admission of Ho Chi Minh City University of Food Industry, Vietnam.” In addition, we have created two datasets to serve our QAS: vi-SQuAD v1.1, which is automatically translated and edited from SQuAD (Stanford University Question Answering Dataset), and HUFI-PostGrad, which is manually collected. We use two main models for the system, including the Intent Classification model and the Machine Reading Comprehension model. Experimental results initially show that our QAS gives encouraging results.