Machine Reading Comprehension Task Research Articles

Medical knowledge graph question answering for drug‐drug interaction prediction based on multi‐hop machine reading comprehension

AbstractDrug‐drug interaction (DDI) prediction is a crucial issue in molecular biology. Traditional methods of observing drug‐drug interactions through medical experiments require significant resources and labour. The authors present a Medical Knowledge Graph Question Answering (MedKGQA) model, dubbed MedKGQA, that predicts DDI by employing machine reading comprehension (MRC) from closed‐domain literature and constructing a knowledge graph of “drug‐protein” triplets from open‐domain documents. The model vectorises the drug‐protein target attributes in the graph using entity embeddings and establishes directed connections between drug and protein entities based on the metabolic interaction pathways of protein targets in the human body. This aligns multiple external knowledge and applies it to learn the graph neural network. Without bells and whistles, the proposed model achieved a 4.5% improvement in terms of DDI prediction accuracy compared to previous state‐of‐the‐art models on the QAngaroo MedHop dataset. Experimental results demonstrate the efficiency and effectiveness of the model and verify the feasibility of integrating external knowledge in MRC tasks.

MRC-Sum: An MRC framework for extractive summarization of academic articles in natural sciences and medicine

Extractive summarization for academic articles in natural sciences and medicine has attracted attention for a long time. However, most existing extractive summarization models often process academic articles with sentence classification models, which are hard to produce comprehensive summaries. To address this issue, we explore a new view to solve the extractive summarization of academic articles in natural sciences and medicine by taking it as a question-answering process. We propose a novel framework, MRC-Sum, where the extractive summarization for academic articles in natural sciences and medicine is cast as an MRC (Machine Reading Comprehension) task. To instantiate MRC-Sum, article-summary pairs in the summarization datasets are firstly reconstructed into (Question, Answer, Context) triples in the MRC task. Several questions are designed to cover the main aspects (e.g. Background, Method, Result, Conclusion) of the articles in natural sciences and medicine. A novel strategy is proposed to solve the problem of the non-existence of the ground truth answer spans. Then MRC-Sum is trained on the reconstructed datasets and large-scale pre-trained models. During the inference stage, four answer spans of the predefined questions are given by MRC-Sum and concatenated to form the final summary for each article. Experiments on three publicly available benchmarks, i.e., the Covid, PubMed, and arXiv datasets, demonstrate the effectiveness of MRC-Sum. Specifically, MRC-Sum outperforms advanced extractive summarization baselines on the Covid dataset and achieves competitive results on the PubMed and arXiv datasets. We also propose a novel metric, COMPREHS, to automatically evaluate the comprehensiveness of the system summaries for academic articles in natural sciences and medicine. Abundant experiments are conducted and verified the reliability of the proposed metric. And the results of the COMPREHS metric show that MRC-Sum is able to generate more comprehensive summaries than the baseline models.

CBKI: A confidence-based knowledge integration framework for multi-choice machine reading comprehension

External knowledge has been introduced into pretrained models to enhance answer reasoning in machine reading comprehension tasks. However, most methods directly take the retrieved knowledge as input without considering the reliability and applicability, which may induce information redundancy and generate noise. In this paper, we propose a confidence-based knowledge integration framework (CBKI) to enhance the pretrained model with the ability of active knowledge acquisition and identification. In particular, we introduce a confidence mechanism, which is composed of three modules, namely, confidence estimation, confidence perception and knowledge integration. Specifically, for the candidate knowledge retrieved from the external knowledge base, the confidence estimation module first calculates the probability score of the candidate knowledge effectiveness in answer prediction. Then, the confidence perception module determines whether the candidate knowledge should be adopted for semantic inference at the current step. Finally, the knowledge integration module evaluates how much information should be absorbed from the candidate knowledge and merges the knowledge into the model. Experimental results on standard datasets show that the proposed CBKI is superior to BERT with an increase of 4.08% in accuracy. Furthermore, the CBKI also shows great advantages in robustness and interpretability.

Knowledge Adaptive Multi-Way Matching Network for Biomedical Named Entity Recognition via Machine Reading Comprehension.

Rapid and effective utilization of biomedical literature is paramount to combat diseases like COVID19. Biomedical named entity recognition (BioNER) is a fundamental task in text mining that can help physicians accelerate knowledge discovery to curb the spread of the COVID-19 epidemic. Recent approaches have shown that casting entity extraction as the machine reading comprehension task can significantly improve model performance. However, two major drawbacks impede higher success in identifying entities (1) ignoring the use of domain knowledge to capture the context beyond sentences and (2) lacking the ability to deeper understand the intent of questions. In this paper, to remedy this, we introduce and explore external domain knowledge which cannot be implicitly learned in text sequence. Previous works have focused more on text sequence and explored little of the domain knowledge. To better incorporate domain knowledge, a multi-way matching reader mechanism is devised to model representations of interaction between sequence, question and knowledge retrieved from Unified Medical Language System (UMLS). Benefiting from these, our model can better understand the intent of questions in complex contexts. Experimental results indicate that incorporating domain knowledge can help to obtain competitive results across 10 BioNER datasets, achieving absolute improvement of up to 2.02% in the f1 score.

Named Entity Recognition Networks Based on Syntactically Constrained Attention

The task of named entity recognition can be transformed into a machine reading comprehension task by associating the query and its context, which contains entity information, with the encoding layer. In this process, the model learns a priori knowledge about the entity, from the query, to achieve good results. However, as the length of the context and query increases, the model struggles with an increasing number of less relevant words, which can distract it from the task. Although attention mechanisms can help the model understand contextual semantic relations, without explicit constraint information, attention may be allocated to less task-relevant words, leading to a bias in the model’s understanding of the context. To address this problem, we propose a new model, the syntactic constraint-based dual-context aggregation network, which uses syntactic information to guide query and context modeling. By incorporating syntactic constraint information into the attention mechanism, the model can better determine the relevance of each word in the context of the task, and selectively focus on the relevant parts of the context. This enhances the model’s ability to read and understand the context, ultimately improving its performance in named entity recognition tasks. Extensive experiments on three datasets, ACE2004, ACE2005, and GENIA, show that this method achieves superior performance when compared to previous methods.

CpgQA: A Benchmark Dataset for Machine Reading Comprehension Tasks on Clinical Practice Guidelines and a Case Study Using Transfer Learning

Biomedical machine reading comprehension (bio-MRC), a crucial task in natural language processing, is a vital application of a computer-assisted clinical decision support system. It can help clinicians extract critical information effortlessly for clinical decision-making by comprehending and answering questions from biomedical text data. While recent advances in bio-MRC consider text data from resources such as clinical notes and scholarly articles, the clinical practice guidelines (CPGs) are still unexplored in this regard. CPGs are a pivotal component of clinical decision-making at the point of care as they provide recommendations for patient care based on the most up-to-date information available. Although CPGs are inherently terse compared to a multitude of articles, often, clinicians find them lengthy and complicated to use. In this paper, we define a new problem domain – bio-MRC on CPGs – where the ultimate goal is to assist clinicians in efficiently interpreting the clinical practice guidelines using MRC systems. To that end, we develop a manually annotated and subject-matter expert-validated benchmark dataset for the bio-MRC task on CPGs – cpgQA. This dataset aims to evaluate intelligent systems performing MRC tasks on CPGs. Hence, we employ the state-of-the-art MRC models to present a case study illustrating an extensive evaluation of the proposed dataset. We address the problem of lack of training data in this newly defined domain by applying transfer learning. The results show that while the current state-of-the-art models perform well with 78% exact match scores on the dataset, there is still room for improvement, warranting further research on this problem domain. We release the dataset at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mmahbub/cpgQA</uri> .

Multi-Turn and Multi-Granularity Reader for Document-Level Event Extraction

Most existing event extraction works mainly focus on extracting events from one sentence. However, in real-world applications, arguments of one event may scatter across sentences and multiple events may co-occur in one document. Thus, these scenarios require document-level event extraction (DEE), which aims to extract events and their arguments across sentences from a document. Previous works cast DEE as a two-step paradigm: sentence-level event extraction (SEE) to document-level event fusion. However, this paradigm lacks integrating document-level information for SEE and suffers from the inherent limitations of error propagation. In this article, we propose a multi-turn and multi-granularity reader for DEE that can extract events from the document directly without the stage of preliminary SEE. Specifically, we propose a new paradigm of DEE by formulating it as a machine reading comprehension task (i.e., the extraction of event arguments is transformed to identify the answer span from the document). Beyond the framework of machine reading comprehension, we introduce a multi-turn and multi-granularity reader to capture the dependencies between arguments explicitly and model long texts effectively. The empirical results demonstrate that our method achieves superior performance on the MUC-4 and the ChFinAnn datasets.

ViMRC - VLSP 2021: Context-Aware Answer Extraction in Vietnamese Question Answering

MRC is challenging the natural language processing fields; machines automatically have to answer questions based on specific passages for this task. In recent years, machine reading comprehension (MRC) has received much attention; many articles have been written about this task. However, most of those articles only develop models in two main languages, English and Chinese. In this article, we propose to apply a new model to the task of reading comprehension in Vietnamese. Specifically, we use BLANC (BLock AttentioN for Context prediction) on pre-trained baseline models to solve the Machine reading comprehension (MRC) task on Vietnamese. We have achieved good results when using BLANC on the baseline model. Specifically, with the MRC task at the VLSP share-task 2021, we scored 76.877% of F1-score on the private test and ranked 2nd in the total. This shows that BLANC works very well in MRC tasks and further enhances the Vietnamese MRC development.

VLSP 2021 - VieCap4H Challenge: Automatic Image Caption Generation for Healthcare Domain in Vietnamese

Machine reading comprehension (MRC) is a challenging Natural Language Processing (NLP) research fieldand wide real-world applications. The great progress of this field in recents is mainly due to the emergence offew datasets for machine reading comprehension tasks with large sizes and deep learning. For the Vietnameselanguage, some datasets, such as UIT-ViQuAD [1] and UIT-ViNewsQA [2], most recently, UIT-ViQuAD 2.0 [3] - adataset of the competitive VLSP 2021-MRC Shared Task 1 . MRC systems must not only answer questions whennecessary but also tactfully abstain from answering when no answer is available according to the given passage.In this paper, we proposed two types of joint models for answerability prediction and pure-MRC prediction with/without a dependency mechanism to learn the correlation between a start position and end position in pure-MRCoutput prediction. Besides, we use ensemble models and a verification strategy by voting the best answer from thetop K answers of different models. Our proposed approach is evaluated on the benchmark VLSP 2021-MRC SharedTask challenge dataset UIT-ViQuAD 2.0 [3] shows that our approach is significantly better than the baseline.

VLSP 2021-ViMRC Challenge: Vietnamese Machine Reading Comprehension

One of the emerging research trends in natural language understanding is machine reading comprehension (MRC) which is the task to find answers to human questions based on textual data. While many datasets have been developed for MRC research for other languages, there is a lack of such resources for the Vietnamese language. Although many datasets and methodologies have been developed for English and Chinese, many Vietnamese machine reading comprehension limitations need to be solved further. Existing Vietnamese datasets for MRC research concentrate solely on answerable questions. However, in reality, questions can be unanswerable for which the correct answer is not stated in the given textual data. To address the weakness, we provide the research community with a benchmark dataset named UIT-ViQuAD 2.0 for evaluating the MRC task and question answering systems for the Vietnamese language. We use UIT-ViQuAD 2.0 as a benchmark dataset for the shared task on Vietnamese machine reading comprehension (VLSP2021-MRC) at the Eighth Workshop on Vietnamese Language and Speech Processing (VLSP 2021). This task attracted 77 participant teams from 34 universities and other organizations. Each participant was provided with the training data, including 28,457 annotated question-answer pairs, and returned the result on a public test set of more than 3,821 questions and a private test set of 3,712 questions. In this article, we present details of the organization of the shared task, an overview of the methods employed by shared-task participants, and the results. The highest performances in this competition are 77.24% (in EM) and 67.43% (in F1-score) on the private test set. The Vietnamese MRC systems proposed by the top 3 teams use XLM-RoBERTa, a powerful pre-trained language model based on the transformer architecture that has achieved state-of-the-art results on many natural language processing tasks. We believe that releasing the UIT-ViQuAD 2.0 dataset motivates more researchers to improve Vietnamese machine reading comprehension.

New Vietnamese Corpus for Machine Reading Comprehension of Health News Articles

Machine reading comprehension is a natural language understanding task where the computing system is required to read a text and then find the answer to a specific question posed by a human. Large-scale and high-quality corpora are necessary for evaluating machine reading comprehension models. Furthermore, machine reading comprehension (MRC) for the health sector has potential for practical applications; nevertheless, MRC research in this domain is currently scarce. This article presents UIT-ViNewsQA, a new corpus for the Vietnamese language to evaluate MRC models for the healthcare textual domain. The corpus consists of 22,057 human-generated question-answer pairs. Crowd-workers create the questions and answers on a collection of 4,416 online Vietnamese healthcare news articles, where the answers are textual spans extracted from the corresponding articles. We introduce a process for creating a high-quality corpus for the Vietnamese machine reading comprehension task. Linguistically, our corpus accommodates diversity in question and answer types. In addition, we conduct experiments and compare the effectiveness of different MRC methods based on the neural networks and transformer architectures. Experimental results on our corpus show that the MRC system based on ALBERT architecture outperforms the neural network architectures and the BERT-based approach, an exact match score of 65.26% and an F1-score of 84.89%. The best machine model achieves about 10.90% F1-score less efficiently than humans, which proves that exploring machine models on UIT-ViNewsQA to surpass humans is challenging for researchers in the future. Our corpus is publicly available on our website: http://nlp.uit.edu.vn/datasets for research purposes.

Utilizing Transformer for Sentence Similarity Modeling and Answer Generation in the Machine Reading Comprehension Task

Utilizing Transformer for Sentence Similarity Modeling and Answer Generation in the Machine Reading Comprehension Task

Improving the robustness of machine reading comprehension via contrastive learning

Pre-trained language models achieve high performance on machine reading comprehension task, but these models lack robustness and are vulnerable to adversarial samples. Most of the current methods for improving model robustness are based on data enrichment. However, these methods do not solve the problem of poor context representation of the machine reading comprehension model. We find that context representation plays a key role in the robustness of the machine reading comprehension model, dense context representation space results in poor model robustness. To deal with this, we propose a Multi-task machine Reading Comprehension learning framework via Contrastive Learning. Its main idea is to improve the context representation space encoded by the machine reading comprehension models through contrastive learning. This special contrastive learning we proposed called Contrastive Learning in Context Representation Space(CLCRS). CLCRS samples sentences containing context information from the context as positive and negative samples, expanding the distance between the answer sentence and other sentences in the context. Therefore, the context representation space of the machine reading comprehension model has been expanded. The model can better distinguish between sentence containing correct answers and misleading sentence. Thus, the robustness of the model is improved. Experiment results on adversarial datasets show that our method exceeds the comparison models and achieves state-of-the-art performance.

BioADAPT-MRC: adversarial learning-based domain adaptation improves biomedical machine reading comprehension task.

ABSTRACTMotivationBiomedical machine reading comprehension (biomedical-MRC) aims to comprehend complex biomedical narratives and assist healthcare professionals in retrieving information from them. The high performance of modern neural network-based MRC systems depends on high-quality, large-scale, human-annotated training datasets. In the biomedical domain, a crucial challenge in creating such datasets is the requirement for domain knowledge, inducing the scarcity of labeled data and the need for transfer learning from the labeled general-purpose (source) domain to the biomedical (target) domain. However, there is a discrepancy in marginal distributions between the general-purpose and biomedical domains due to the variances in topics. Therefore, direct-transferring of learned representations from a model trained on a general-purpose domain to the biomedical domain can hurt the model’s performance.ResultsWe present an adversarial learning-based domain adaptation framework for the biomedical machine reading comprehension task (BioADAPT-MRC), a neural network-based method to address the discrepancies in the marginal distributions between the general and biomedical domain datasets. BioADAPT-MRC relaxes the need for generating pseudo labels for training a well-performing biomedical-MRC model. We extensively evaluate the performance of BioADAPT-MRC by comparing it with the best existing methods on three widely used benchmark biomedical-MRC datasets—BioASQ-7b, BioASQ-8b and BioASQ-9b. Our results suggest that without using any synthetic or human-annotated data from the biomedical domain, BioADAPT-MRC can achieve state-of-the-art performance on these datasets.Availability and implementationBioADAPT-MRC is freely available as an open-source project at https://github.com/mmahbub/BioADAPT-MRC.Supplementary information Supplementary data are available at Bioinformatics online.

Zero-Shot Cross-Lingual Machine Reading Comprehension via Inter-sentence Dependency Graph

We target the task of cross-lingual Machine Reading Comprehension (MRC) in the direct zero-shot setting, by incorporating syntactic features from Universal Dependencies (UD), and the key features we use are the syntactic relations within each sentence. While previous work has demonstrated effective syntax-guided MRC models, we propose to adopt the inter-sentence syntactic relations, in addition to the rudimentary intra-sentence relations, to further utilize the syntactic dependencies in the multi-sentence input of the MRC task. In our approach, we build the Inter-Sentence Dependency Graph (ISDG) connecting dependency trees to form global syntactic relations across sentences. We then propose the ISDG encoder that encodes the global dependency graph, addressing the inter-sentence relations via both one-hop and multi-hop dependency paths explicitly. Experiments on three multilingual MRC datasets (XQuAD, MLQA, TyDiQA-GoldP) show that our encoder that is only trained on English is able to improve the zero-shot performance on all 14 test sets covering 8 languages, with up to 3.8 F1 / 5.2 EM improvement on-average, and 5.2 F1 / 11.2 EM on certain languages. Further analysis shows the improvement can be attributed to the attention on the cross-linguistically consistent syntactic path. Our code is available at https://github.com/lxucs/multilingual-mrc-isdg.

Machine Reading Comprehension Using Multi-Passage BERT with Dice Loss on Thai Corpus

Nowadays there is an advancement in the field of machine reading comprehension task (MRC) due to the invention of large scale pre-trained language models, such as BERT. However, the performance is still limited when the context is long and contains many passages. BERT can only embed a part of the whole passage equal to the input size; thus, sliding windows must be used which leads to discontinued information when the passage is long. In this paper, we aim to propose a BERT-based MRC framework tailored for a long passage context in the Thai corpus. Our framework employs the multi-passage BERT along with self-adjusting dice loss, which can help the model focuses more on the answer region of the context passage. We also show that there is an improvement in the performance when an auxiliary task is used. The experiment was conducted on the Thai Question Answering (QA) dataset used in Thailand National Software Competition. The results show that our method improves the model’s performance over a traditional BERT framework.

Research on Bi-LSTM machine reading comprehension algorithm based on attention mechanism

Abstract Machine reading comprehension is a very important research field in Natural Language Processing (NLP), but it is not sufficient for military research, and traditional machine reading comprehension algorithms do not fully explore every word in different contexts. In response to the above problems, this paper proposes a machine reading comprehension model that incorporates a multi-attention mechanism. It is brave to solve machine reading comprehension tasks in the military field. The model in this paper performs word embedding, part-of-speech embedding, and knowledge base embedding for each word in the article. Among them, knowledge embedding uses military knowledge bases to make up for the lack of information on military entities and relationships in military articles, and independently carries out part-of-speech embedding to solve the problem of part of speech judgment of polysemous words in specific contexts. The experimental results show that the machine reading comprehension model with multi-attention mechanism proposed in this paper can achieve better results on machine reading comprehension tasks, and the accuracy rate is improved by 1.6% compared with ROUGE-L.

Multi-task joint training model for machine reading comprehension

Machine reading comprehension is an important topic in natural language processing. However, the redundancy of text and the diversity of answer types are neglected in existing machine reading comprehension. To tackle these issues, a multi-task joint training scheme is proposed for the multi-type answer machine reading comprehension task. In this scheme, a feature-based paragraph extraction mechanism is first designed in the training stage to extract various text features from answers and sentences. For efficiency, the redundancy of the text is greatly reduced, while the effective information of the text is retained. And then, a reading comprehension module is optimized by enhancing the representation of the pre-trained language model, as well as an answer type classification module is optimized through the capsule network. Finally, a multi-task joint training model is designed to simultaneous obtaining answer text and answer type. Experiments are conducted to evaluate the proposed approach with two open datasets, CJRC and Natural Questions, and demonstrate that our model is efficient and promising, in terms of both the reading comprehension and answer classification tasks. In particular, our model won the best method that is winner of the first prize in the CAIL-2020 machine reading comprehension contest.

Improving Machine Reading Comprehension with Multi-Task Learning and Self-Training

Machine Reading Comprehension (MRC) is an AI challenge that requires machines to determine the correct answer to a question based on a given passage, in which extractive MRC requires extracting an answer span to a question from a given passage, such as the task of span extraction. In contrast, non-extractive MRC infers answers from the content of reference passages, including Yes/No question answering to unanswerable questions. Due to the specificity of the two types of MRC tasks, researchers usually work on one type of task separately, but real-life application situations often require models that can handle many different types of tasks in parallel. Therefore, to meet the comprehensive requirements in such application situations, we construct a multi-task fusion training reading comprehension model based on the BERT pre-training model. The model uses the BERT pre-training model to obtain contextual representations, which is then shared by three downstream sub-modules for span extraction, Yes/No question answering, and unanswerable questions, next we fuse the outputs of the three sub-modules into a new span extraction output and use the fused cross-entropy loss function for global training. In the training phase, since our model requires a large amount of labeled training data, which is often expensive to obtain or unavailable in many tasks, we additionally use self-training to generate pseudo-labeled training data to train our model to improve its accuracy and generalization performance. We evaluated the SQuAD2.0 and CAIL2019 datasets. The experiments show that our model can efficiently handle different tasks. We achieved 83.2EM and 86.7F1 scores on the SQuAD2.0 dataset and 73.0EM and 85.3F1 scores on the CAIL2019 dataset.

SSAG-Net: Syntactic and Semantic Attention-Guided Machine Reading Comprehension

Machine reading comprehension (MRC) is a task in natural language comprehension. It assesses machine reading comprehension based on text reading and answering questions. Traditional attention methods typically focus on one of syntax or semantics, or integrate syntax and semantics through a manual method, leaving the model unable to fully utilize syntax and semantics for MRC tasks. In order to better understand syntactic and semantic information and improve machine reading comprehension, our study uses syntactic and semantic attention to conduct text modeling for tasks. Based on the BERT model of Transformer encoder, we separate a text into two branches: syntax part and semantics part. In syntactic component, an attention model with explicit syntactic constraints is linked with a self-attention model of context. In semantics component, after the framework semantic parsing, the lexical unit attention model is utilized to process the text in the semantic part. Finally, the vectors of the two branches converge into a new vector. And it can make answer predictions based on different types of data. Thus, a syntactic and semantic attention-guided machine reading comprehension (SSAG-Net) is formed. To test the model’s validity, we ran it through two MRC tasks on SQuAD 2.0 and MCTest, and the SSAG-Net model outperformed the baseline model in both.