Document-level Relation Extraction Research Articles

Advancing document-level relation extraction with a syntax-enhanced multi-hop reasoning network

Document-level relation extraction aims to uncover relations between entities by harnessing the intricate information spread throughout a document. Previous research involved constructing discrete syntactic matrices to capture syntactic relationships within documents. However, these methods are significantly influenced by dependency parsing errors, leaving much of the latent syntactic information untapped. Moreover, prior research has mainly focused on modeling two-hop reasoning between entity pairs, which has limited applicability in scenarios requiring multi-hop reasoning. To tackle these challenges, a syntax-enhanced multi-hop reasoning network (SEMHRN) is proposed. Specifically, the approach begins by using a dependency probability matrix that incorporates richer grammatical information instead of a sparse syntactic parsing matrix to build the syntactic graph. This effectively reduces syntactic parsing errors and enhances the model’s robustness. To fully leverage dependency information, dependency-type-aware attention is introduced to refine edge weights based on connecting edge types. Additionally, a part-of-speech prediction task is included to regularize word embeddings. Unrelated entity pairs can disrupt the model’s focus, reducing its efficiency. To concentrate the model’s attention on related entity pairs, these related pairs are extracted, and a multi-hop reasoning graph attention network is employed to capture the multi-hop dependencies among them. Experimental results on three public document-level relation extraction datasets validate that SEMHRN achieves a competitive F1 score compared to the current state-of-the-art methods.

Self-supervised commonsense knowledge learning for document-level relation extraction

Compared to sentence-level relation extraction, practical document-level relation extraction (DocRE) is a more challenging task for which multi-entity problems need to be resolved. It aims at extracting relationships between two entities over multiple sentences at once while taking into account significant cross-sentence features. Learning long-distance semantic relation representation across sentences in a document, however, is a widespread and difficult task. To address the above issues, we propose a novel Self-supervised Commonsense-enhanced DocRE approach, named as SCDRE, bypassing the need for external knowledge. The methodology begins by harnessing self-supervised learning to capture the commonsense knowledge pertaining to each entity within an entity pair, drawing insights from the commonsense entailed text. This acquired knowledge subsequently serves as the foundation for transforming cross-sentence entity pairs into alias counterparts achieved by the coreference commonsense replacement. The focus then shifts to semantic relation representation learning, applied to these alias entity pairs. Through a process of entity pair rich attention fusion, these alias pairs are seamlessly and automatically translated back into the target entity pairs. This innovation harnesses self-supervised learning and contextual commonsense to distinguish SCDRE as a unique and self-contained approach, promising an enhanced ability to extract relationships from documents. We examine our model on three publicly accessible datasets, DocRED, DialogRE and MPDD, and the results show that it performs significantly better than strong baselines by 2.03% F1, and commonsense knowledge has an important contribution to the DocRE by the ablation experimental analysis.

Document-Level Relation Extraction with Progressive Self-Distillation

Document-level relation extraction (RE) aims to simultaneously predict relations (including no-relation cases denoted as NA) between all entity pairs in a document. It is typically formulated as a relation classification task with entities pre-detected in advance and solved by a hard-label training regime, which however neglects the divergence of the NA class and the correlations among other classes. This article introduces progressive self-distillation (PSD), a new training regime that employs online, self-knowledge distillation (KD) to produce and incorporate soft labels for document-level RE. The key idea of PSD is to gradually soften hard labels using past predictions from an RE model itself, which are adjusted adaptively as training proceeds. As such, PSD has to learn only one RE model within a single training pass, requiring no extra computation or annotation to pretrain another high-capacity teacher. PSD is conceptually simple, easy to implement, and generally applicable to various RE models to further improve their performance, without introducing additional parameters or significantly increasing training overheads into the models. It is also a general framework that can be flexibly extended to distilling various types of knowledge, rather than being restricted to soft labels themselves. Extensive experiments on four benchmarking datasets verify the effectiveness and generality of the proposed approach. The code is available at https://github.com/GaoJieCN/psd.

LoGo-GR: a local to global graphical reasoning framework for extracting structured information from biomedical literature.

In the biomedical literature, entities are often distributed within multiple sentences and exhibit complex interactions. As the volume of literature has increased dramatically, it has become impractical to manually extract and maintain biomedical knowledge, which would entail enormous costs. Fortunately, document-level relation extraction can capture associations between entities from complex text, helping researchers efficiently mine structured knowledge from the vast medical literature. However, how to effectively synthesize rich global information from context and accurately capture local dependencies between entities is still a great challenge. In this paper, we propose a Local to Global Graphical Reasoning framework (LoGo-GR) based on a novel Biased Graph Attention mechanism (B-GAT). It learns global context feature and information of local relation path dependencies from mention-level interaction graph and entity-level path graph respectively, and collaborates with global and local reasoning to capture complex interactions between entities from document-level text. In particular, B-GAT integrates structural dependencies into the standard graph attention mechanism (GAT) as attention biases to adaptively guide information aggregation in graphical reasoning. We evaluate our method on three publicly biomedical document-level datasets: Drug-Mutation Interaction (DV), Chemical-induced Disease (CDR), and Gene-Disease Association (GDA). LoGo-GR has advanced and stable performance compared to other state-of-the-art methods (it achieves state-of-the-art performance with 96.14%-97.39% F1 on DV dataset, advanced performance with 68.89% F1 and 84.22% F1 on CDR and GDA datasets, respectively). In addition, LoGo-GR also shows advanced performance on general-domain document-level relation extraction dataset, DocRED, which proves that it is an effective and robust document-level relation extraction framework. Our codes are publicly available at: https://github.com/Zxy-MLlab/LoGo-GR.

Graph neural networks with selective attention and path reasoning for document-level relation extraction

Graph neural networks with selective attention and path reasoning for document-level relation extraction

Self-distillation framework for document-level relation extraction in low-resource environments.

The objective of document-level relation extraction is to retrieve the relations existing between entities within a document. Currently, deep learning methods have demonstrated superior performance in document-level relation extraction tasks. However, to enhance the model's performance, various methods directly introduce additional modules into the backbone model, which often increases the number of parameters in the overall model. Consequently, deploying these deep models in resource-limited environments presents a challenge. In this article, we introduce a self-distillation framework for document-level relational extraction. We partition the document-level relation extraction model into two distinct modules, namely, the entity embedding representation module and the entity pair embedding representation module. Subsequently, we apply separate distillation techniques to each module to reduce the model's size. In order to evaluate the proposed framework's performance, two benchmark datasets for document-level relation extraction, namely GDA and DocRED are used in this study. The results demonstrate that our model effectively enhances performance and significantly reduces the model's size.

A Positive-Unlabeled Metric Learning Framework for Document-Level Relation Extraction with Incomplete Labeling

The goal of document-level relation extraction (RE) is to identify relations between entities that span multiple sentences. Recently, incomplete labeling in document-level RE has received increasing attention, and some studies have used methods such as positive-unlabeled learning to tackle this issue, but there is still a lot of room for improvement. Motivated by this, we propose a positive-augmentation and positive-mixup positive-unlabeled metric learning framework (P3M). Specifically, we formulate document-level RE as a metric learning problem. We aim to pull the distance closer between entity pair embedding and their corresponding relation embedding, while pushing it farther away from the none-class relation embedding. Additionally, we adapt the positive-unlabeled learning to this loss objective. In order to improve the generalizability of the model, we use dropout to augment positive samples and propose a positive-none-class mixup method. Extensive experiments show that P3M improves the F1 score by approximately 4-10 points in document-level RE with incomplete labeling, and achieves state-of-the-art results in fully labeled scenarios. Furthermore, P3M has also demonstrated robustness to prior estimation bias in incomplete labeled scenarios.

A Hierarchical Network for Multimodal Document-Level Relation Extraction

Document-level relation extraction aims to extract entity relations that span across multiple sentences. This task faces two critical issues: long dependency and mention selection. Prior works address the above problems from the textual perspective, however, it is hard to handle these problems solely based on text information. In this paper, we leverage video information to provide additional evidence for understanding long dependencies and offer a wider perspective for identifying relevant mentions, thus giving rise to a new task named Multimodal Document-level Relation Extraction (MDocRE). To tackle this new task, we construct a human-annotated dataset including documents and relevant videos, which, to the best of our knowledge, is the first document-level relation extraction dataset equipped with video clips. We also propose a hierarchical framework to learn interactions between different dependency levels and a textual-guided transformer architecture that incorporates both textual and video modalities. In addition, we utilize a mention gate module to address the mention-selection problem in both modalities. Experiments on our proposed dataset show that 1) incorporating video information greatly improves model performance; 2) our hierarchical framework has state-of-the-art results compared with both unimodal and multimodal baselines; 3) through collaborating with video information, our model better solves the long-dependency and mention-selection problems.

Revisiting Document-Level Relation Extraction with Context-Guided Link Prediction

Document-level relation extraction (DocRE) poses the challenge of identifying relationships between entities within a document. Existing approaches rely on logical reasoning or contextual cues from entities. This paper reframes document-level RE as link prediction over a Knowledge Graph (KG) with distinct benefits: 1) Our approach amalgamates entity context and document-derived logical reasoning, enhancing link prediction quality. 2) Predicted links between entities offer interpretability, elucidating employed reasoning. We evaluate our approach on benchmark datasets - DocRED, ReDocRED, and DWIE. The results indicate that our proposed method outperforms the state-of-the-art models and suggests that incorporating context-based Knowledge Graph link prediction techniques can enhance the performance of document-level relation extraction models.

Document-Level Relation Extraction with Uncertainty Pseudo-Label Selection and Hard-Sample Focal Loss

Relation extraction is a fundamental task in natural language processing that aims to identify structured triple relationships from unstructured text. In recent years, research on relation extraction has gradually advanced from the sentence level to the document level. Most existing document-level relation extraction (DocRE) models are fully supervised and their performance is limited by the dataset quality. However, existing DocRE datasets suffer from annotation omission, making fully supervised models unsuitable for real-world scenarios. To address this issue, we propose the DocRE method based on uncertainty pseudo-label selection. This method first trains a teacher model to annotate pseudo-labels for a dataset with incomplete annotations, trains a student model on the dataset with annotated pseudo-labels, and uses the trained student model to predict relations on the test set. To mitigate the confirmation bias problem in pseudo-label methods, we performed adversarial training on the teacher model and calculated the uncertainty of the model output to supervise the generation of pseudo-labels. In addition, to address the hard-easy sample imbalance problem, we propose an adaptive hard-sample focal loss. This loss can guide the model to reduce attention to easy-to-classify samples and outliers and to pay more attention to hard-to-classify samples. We conducted experiments on two public datasets, and the results proved the effectiveness of our method.

Document-Level Relation Extraction with Deep Gated Graph Reasoning

Extracting the relations of two entities on the sentence-level has drawn increasing attention in recent years but remains facing great challenges on document-level, due to the inherent difficulty in recognizing the relations of two entities across multiple sentences. Previous works show that employing the graph convolutional neural network can help the model capture unstructured dependent information of entities. However, they usually employed the non-adaptive weight edges to build the correlation weight matrix which suffered from the problem of information redundancy and gradient disappearance. To solve this problem, we propose a deep gated graph reasoning model for document-level relation extraction, namely, BERT-GGNNs, which employ an improved gated graph neural network with a learnable correlation weight matrix to establish multiple deep gated graph reason layers. The proposed deep gated graph reasoning layers make the model easier to reasoning the relations between entities hidden in the document. Experiments show that the proposed model outperforms most of strong baseline models, and our proposed model is 0.3% and 0.3% higher than the famous LSR-BERT model on the F1 and Ing F1, respectively.

Enhanced Heterogeneous Graph Attention Network with a Novel Multilabel Focal Loss for Document-Level Relation Extraction.

Recent years have seen a rise in interest in document-level relation extraction, which is defined as extracting all relations between entities in multiple sentences of a document. Typically, there are multiple mentions corresponding to a single entity in this context. Previous research predominantly employed a holistic representation for each entity to predict relations, but this approach often overlooks valuable information contained in fine-grained entity mentions. We contend that relation prediction and inference should be grounded in specific entity mentions rather than abstract entity concepts. To address this, our paper proposes a two-stage mention-level framework based on an enhanced heterogeneous graph attention network for document-level relation extraction. Our framework employs two different strategies to model intra-sentential and inter-sentential relations between fine-grained entity mentions, yielding local mention representations for intra-sentential relation prediction and global mention representations for inter-sentential relation prediction. For inter-sentential relation prediction and inference, we propose an enhanced heterogeneous graph attention network to better model the long-distance semantic relationships and design an entity-coreference path-based inference strategy to conduct relation inference. Moreover, we introduce a novel cross-entropy-based multilabel focal loss function to address the class imbalance problem and multilabel prediction simultaneously. Comprehensive experiments have been conducted to verify the effectiveness of our framework. Experimental results show that our approach significantly outperforms the existing methods.

Document-level relation extraction with global and path dependencies

Document-level relation extraction (RE) focuses on extracting relations for each entity pair in the same sentence or across different sentences of a document. Several existing methodologies aim to capture the intricate interactions among entities across a document by constructing diverse document graphs. However, these graphs frequently cannot sufficiently model the intricate global interactions and concurrent explicit path reasoning. Therefore, we introduce a distinctive graph-based model designed to assimilate global and path dependencies within a document for document-level RE, termed graph-based global and path dependencies (GGP). Specifically, the global dependency component captures interactions between mentions, entities, sentences and, the document through two interconnected graphs: the mention-level graph and the entity-level graph (ELG). To integrate relevant paths essential for the designated entity pair, the path dependency component consolidates information from various multi-hop paths of the target entity pair through an attention mechanism on the ELG. In addition, we devised an innovative method for learning path representation, which encapsulates relations and intermediate entities within the multi-hop path in the ELG. Comprehensive experiments conducted on standard document-level RE and CDR datasets reveal the following key findings: (i) GGP achieves an Ign F1 score of 59.98%, surpassing baselines by 0.61% on the test set; and (ii) the integration of various features derived from entities, sentences, documents, and paths enhances GGP's performance in document-level RE.

Incorporating Evidence Retrieval into Document-Level Relation Extraction by Guiding Attention

Incorporating Evidence Retrieval into Document-Level Relation Extraction by Guiding Attention

Document-level relation extraction with three channels

Relation extraction aims to identify the relation between two entities. Early works on relation extraction are performed within a sentence. Numerous entity pairs with relations are distributed across sentences, so the task of document-level relation extraction is proposed. The current works perform well on document-level relation extraction through graph neural network, transformer, etc. However, these works extract relation features from one perspective, resulting in limited extraction performance of the model. To break through the limitation, this paper proposes the DRETC model consisting of three channels. Channel one first constructs the mention matrix and then extracts the relation features between entities. Channel two performs sequence inference from mention to entity before generating the relation matrix. Moreover, the bridge method is proposed to introduce information in channel one that is beneficial to relation extraction. Channel three directly generates entity representations through text sequences to make up for the absence of text information related to the entity. The three channels extract relation features from three different perspectives, and the combination leads to better extraction performance. Experimental results show that the proposed DRETC model outperforms previous strong baseline models with one channel on the public dataset DocRED. Further analysis demonstrates that the combination of multiple channels has a better relation extraction performance than a single channel.

APIE: An information extraction module designed based on the pipeline method

Information extraction (IE) aims to discover and extract valuable information from unstructured text. This problem can be decomposed into two subtasks: named entity recognition (NER) and relation extraction (RE). Although the IE problem has been studied for years, most work efforts focused on jointly modeling these two subtasks, either by casting them into a structured prediction framework or by performing multitask learning through shared representations. However, since the contextual representations of entity and relation models inherently capture different feature information, sharing a single encoder to capture the information required by both subtasks in the same space would harm the accuracy of the model. Recent research (Zhong and Chen, 2020) has also proved that using two separate encoders for NER and RE tasks respectively through pipeline method are effective, with the model surpassing all previous joint models in accuracy. Thus, in this paper, we design An Pipeline method Information Extraction module called APIE, APIE combines the advantages of both pipeline methods and joint methods, demonstrating higher accuracy and powerful reasoning abilities. Specifically, we design a multi-level feature NER model based on attention mechanism and a document-level RE model based on local context pooling. To demonstrate the effectiveness of our proposed approach, we conducted tests on multiple datasets. Extensive experimental results have shown that our proposed model outperforms state-of-the-art methods and improves both accuracy and reasoning abilities.

Document-level Relation Extraction with Relation Correlations

Document-level relation extraction faces two often overlooked challenges: long-tail problem and multi-label problem. Previous work focuses mainly on obtaining better contextual representations for entity pairs, hardly address the above challenges. In this paper, we analyze the co-occurrence correlation of relations, and introduce it into the document-level relation extraction task for the first time. We argue that the correlations can not only transfer knowledge between data-rich relations and data-scarce ones to assist in the training of long-tailed relations, but also reflect semantic distance guiding the classifier to identify semantically close relations for multi-label entity pairs. Specifically, we use relation embedding as a medium, and propose two co-occurrence prediction sub-tasks from both coarse- and fine-grained perspectives to capture relation correlations. Finally, the learned correlation-aware embeddings are used to guide the extraction of relational facts. Substantial experiments on two popular datasets (i.e., DocRED and DWIE) are conducted, and our method achieves superior results compared to baselines. Insightful analysis also demonstrates the potential of relation correlations to address the above challenges. The data and code are released at https://github.com/RidongHan/DocRE-Co-Occur.

Document-level denoising relation extraction with false-negative mining and reinforced positive-class knowledge distillation

Many datasets for document-level relation extraction (RE) suffer from incomplete labeling, particularly the false negative problem, which induces improper biases during training. However, existing denoising methods are either limited to the scale of dataset or primarily focus on sentence-level RE. To tackle this prevalent issue of false negatives, we propose a denoising framework called FM-RKD for document-level RE. Firstly, a false-negative mining mechanism is introduced to identify and re-annotate false negative samples (FNs) within the original corpus, thereby producing a higher-quality pseudo corpus. Then, we propose a reinforced positive-class knowledge distillation method, where a teacher network trained with positive samples provides soft labels for a student network. This approach enables the student network to learn complete positive-class patterns and mitigate the overfitting issue caused by FNs. Extensive experiments on the Re-DocRED dataset show that FM-RKD outperforms the current state-of-the-art method by 1.36% in F1 score and 1.24% in Ign F1 score when the training data is incompletely annotated. Moreover, FM-RKD consistently achieves new peak performance with an F1 score of 78.38% even when the training data is well-annotated.

Document-level relation extraction with entity mentions deep attention

Document-level Relation Extraction(DocRE) aims to extract relations between entities from documents. In contrast to sentence-level relation extraction, it requires extracting semantic relations from multiple sentences. It is necessary to further improve the performance of the above algorithm in order to extract document-level relation. Therefore, the DocRE algorithms have to deal with more complex entity structure relationships and the need to unite semantic relationships between different sentences when reasoning about relationships between entities. The proposed algorithms fail to infer relationships between entities when dealing with complex entity structure relationships. In this paper, we propose an entity mentions deep attention framework that efficiently infers entity relationships through entity structure and contextual information. Firstly, a structural dependency module of entities is designed to achieve interaction between different mentions of the entity. Secondly, a deep contextual attention component proposed to enrich the semantic information between entities by entity-related contexts. Finally, we use a distance mapping component to solve the problem of entity pairs that are far away from each other. According to our implementation results, our model outperforms the state-ofthe-art models on three public datasets DocRED, DGA, and CDR.

Inter span learning for document-level relation extraction

The entity-relationship extraction model has a significant influence in relation extraction. The existing model cannot effectively identify the entity-relationship triples in overlapping relationships. It also has the problem of long-distance dependencies between entities. In this paper, an inter span learning for document-level relation extraction model is proposed. Firstly, the model converts input of the BERT pre-training model into word vectors. Secondly, it divides the word vectors to form span sequences by random initial span and uses convolutional neural networks to extract entity information in the span sequences. Dividing the word vector into span sequences can divide the entity pairs that may have overlapping relationships into the same span sequence, partially solving the overlapping relationship problem. Thirdly, the model uses inter span learning to obtain entity information in different span sequences. It fuses entity type features and uses Softmax regression to achieve entity recognition. Aiming at solving the problem of long-distance dependence between entities, inter span learning can fuse the information in different span sequences. Finally, it fuses text information and relationship type features, and uses Linear Layer to classify relationships. Experiments demonstrate that the model improves the F1-score of the DocRED dataset by 2.74% when compared to the baseline model.