Entity Pairs Research Articles

Document-level relation extraction with global and path dependencies

Document-level relation extraction (RE) focuses on extracting relations for each entity pair in the same sentence or across different sentences of a document. Several existing methodologies aim to capture the intricate interactions among entities across a document by constructing diverse document graphs. However, these graphs frequently cannot sufficiently model the intricate global interactions and concurrent explicit path reasoning. Therefore, we introduce a distinctive graph-based model designed to assimilate global and path dependencies within a document for document-level RE, termed graph-based global and path dependencies (GGP). Specifically, the global dependency component captures interactions between mentions, entities, sentences and, the document through two interconnected graphs: the mention-level graph and the entity-level graph (ELG). To integrate relevant paths essential for the designated entity pair, the path dependency component consolidates information from various multi-hop paths of the target entity pair through an attention mechanism on the ELG. In addition, we devised an innovative method for learning path representation, which encapsulates relations and intermediate entities within the multi-hop path in the ELG. Comprehensive experiments conducted on standard document-level RE and CDR datasets reveal the following key findings: (i) GGP achieves an Ign F1 score of 59.98%, surpassing baselines by 0.61% on the test set; and (ii) the integration of various features derived from entities, sentences, documents, and paths enhances GGP's performance in document-level RE.

Diverse Structure-Aware Relation Representation in Cross-Lingual Entity Alignment

Cross-lingual entity alignment (CLEA) aims to find equivalent entity pairs between knowledge graphs (KGs) in different languages. It is an important way to connect heterogeneous KGs and facilitate knowledge completion. Existing methods have found that incorporating relations into entities can effectively improve KG representation and benefit entity alignment, and these methods learn relation representation depending on entities, which cannot capture the diverse structures of relations. However, multiple relations in KG form diverse structures, such as adjacency structure and ring structure. This diversity of relation structures makes the relation representation challenging. Therefore, we propose to construct the weighted line graphs to model the diverse structures of relations and learn relation representation independently from entities. Especially, owing to the diversity of adjacency structures and ring structures, we propose to construct adjacency line graph and ring line graph, respectively, to model the structures of relations and to further improve entity representation. In addition, to alleviate the hubness problem in alignment, we introduce the optimal transport into alignment and compute the distance matrix in a different way. From a global perspective, we calculate the optimal 1-to-1 alignment bi-directionally to improve the alignment accuracy. Experimental results on two benchmark datasets show that our proposed method significantly outperforms state-of-the-art CLEA methods in both supervised and unsupervised manners.

A collaborative learning framework for knowledge graph embedding and reasoning

Knowledge graph embedding (KGE) and knowledge graph reasoning (KGR) aim to automatic completion of knowledge graph (KG). The difference is that most KGE models learn the embedded representation at a triple level. In contrast, KGR models focus more on optimizing decision-making and enhancing the interpretability of reasoning processes with multi-hop paths. As a result, KGE models are better at learning triplet embeddings, whereas KGR models can capture the multihop information between entity pairs. However, KGE and KGR models only focus on one aspect that affects the completion performance. This paper proposes a plug-and-play collaborative learning framework (CLF) for jointly enhancing knowledge graph embedding and reasoning, which can accommodate existing KGR and KGE models. The two models exchange training experiences in this framework to realize mutual learning through a collaborative learning module. In this module, a new distance function is designed to maintain the independence of candidate entities’ probabilities and avoid information loss. Furthermore, a knowledge augmentation module is designed to identify missing key triples to assist in the further iterative training of the framework. Extensive experiments on the benchmark datasets demonstrate that our framework significantly improves the performance of existing models.

FSN: Joint Entity and Relation Extraction Based on Filter Separator Network.

Joint entity and relation extraction methods have attracted an increasing amount of attention recently due to their capacity to extract relational triples from intricate texts. However, most of the existing methods ignore the association and difference between the Named Entity Recognition (NER) subtask features and the Relation Extraction (RE) subtask features, which leads to an imbalance in the interaction between these two subtasks. To solve the above problems, we propose a new joint entity and relation extraction method, FSN. It contains a Filter Separator Network (FSN) module that employs a two-direction LSTM to filter and separate the information contained in a sentence and merges similar features through a splicing operation, thus solving the problem of the interaction imbalance between subtasks. In order to better extract the local feature information for each subtask, we designed a Named Entity Recognition Generation (NERG) module and a Relation Extraction Generation (REG) module by adopting the design idea of the decoder in Transformer and average pooling operations to better capture the entity boundary information in the sentence and the entity pair boundary information for each relation in the relational triple, respectively. Additionally, we propose a dynamic loss function that dynamically adjusts the learning weights of each subtask in each epoch according to the proportionality between each subtask, thus narrowing down the difference between the ideal and realistic results. We thoroughly evaluated our model on the SciERC dataset and the ACE2005 dataset. The experimental results demonstrate that our model achieves satisfactory results compared to the baseline model.

Relation-oriented few-shot knowledge graph prototype networks

Knowledge graph (KG) embedding has been widely researched, but it suffers from some problems in the few-shot scenarios. The topological and semantic connection among entities cannot satisfy the assumption of samples’ independent identically distribution. Additionally, these relations between entity pairs are various and complex. But most of the previous methods are node-oriented, which is weak in modeling complex relations. In order to solve the above problems, we propose a few-shot relation prototype network (FRPN). In our method, each relation is regarded as a learning object, instead of just entity pairs’ concatenation. A relation-oriented two-channel embedding mechanism is designed to achieve multi-scale information aggregation at the entity level and the relation level respectively. For the entity level, it aggregates information at different scales according to the relation type and the neighbors under each relation. For the relation level, our model obtains each kind of relation’s prototype from the ontology and the entity layer. The multi-scale aggregation contributes to KG embedding in the few-shot scenario. Compared with existing models, our model has significantly improved the performance on both the link prediction and triple classification tasks in two few-shot datasets.

Node-degree aware edge sampling mitigates inflated classification performance in biomedical random walk-based graph representation learning.

Graph representation learning is a family of related approaches that learn low-dimensional vector representations of nodes and other graph elements called embeddings. Embeddings approximate characteristics of the graph and can be used for a variety of machine-learning tasks such as novel edge prediction. For many biomedical applications, partial knowledge exists about positive edges that represent relationships between pairs of entities, but little to no knowledge is available about negative edges that represent the explicit lack of a relationship between two nodes. For this reason, classification procedures are forced to assume that the vast majority of unlabeled edges are negative. Existing approaches to sampling negative edges for training and evaluating classifiers do so by uniformly sampling pairs of nodes. We show here that this sampling strategy typically leads to sets of positive and negative examples with imbalanced node degree distributions. Using representative heterogeneous biomedical knowledge graph and random walk-based graph machine learning, we show that this strategy substantially impacts classification performance. If users of graph machine-learning models apply the models to prioritize examples that are drawn from approximately the same distribution as the positive examples are, then performance of models as estimated in the validation phase may be artificially inflated. We present a degree-aware node sampling approach that mitigates this effect and is simple to implement. Our code and data are publicly available at https://github.com/monarch-initiative/negativeExampleSelection.

Relation-aware deep neural network enables more efficient biomedical knowledge acquisition from massive literature

Biomedical knowledge is typically organized in a relational scheme, such as chemical-disease relation, gene-disease relation, and gene-pathway relation. Biomedical scientists heavily rely on search engines to acquire up-to-date relational knowledge from massive biomedical articles. The navigation efficiency of the retrieval process, however, is significantly restricted by keyword matching techniques unaware of the biomedical relations of these keywords in articles. To bridge the gap between existing retrieval techniques and practical access demands for relational knowledge, we present a novel framework, Biomedical Relation-Aware Document Ranking (BioRADR), capable of retrieving articles expressing specific relations with respect to the queried entity pair. Based on a deep neural network, BioRADR can be trained from large-scale data automatically annotated via distant supervision, and empirical evaluation reveals that it outperforms the strongest baseline by over 8 points in NDCG@1. We implement an online system (http://bioradr.ai.thunlp.org/) based on BioRADR, enabling more efficient relation-oriented retrieval of biomedical articles.

Similarity propagation based semi-supervised entity alignment

Entity alignment aims to identify entities referring to the same real world object among multiple knowledge graphs. Current embedding based approaches suffer from the lack of labeled entity pairs as training data. Some works attempt to boost the training process with semi-supervised methods, which add confidently predicted entity pairs into training data iteratively. Though the effectiveness of this strategy has been confirmed, the current semi-supervised methods suffer from the problem of incorrect newly labeled entity pairs. This paper presents a novel semi-supervised entity alignment framework Similarity Propagation based Semi-supervised Entity Alignment (SPSEA), which improves the precision of labeled entity pairs by propagating alignment information from seed entity pairs to their direct neighbors. The key idea is to combine dependency between entities and entity embeddings to obtain entity similarities, which alleviates the problem of mislabeling when the entity embeddings are of low quality. And it finds aligned entities from direct neighbors of seed entity pairs with the help of relation alignment, which narrows the search space and not hurting the recall of true pairs. In addition, we propose novel deferred-acceptance algorithm and bilateral alignment strategy to further guarantee the quality of obtained entity pairs. Through extensive experiments, we show that the quality of labeled entity pairs obtained by SPSEA is high than current semi-supervised methods.

BioEGRE: a linguistic topology enhanced method for biomedical relation extraction based on BioELECTRA and graph pointer neural network

BackgroundAutomatic and accurate extraction of diverse biomedical relations from literature is a crucial component of bio-medical text mining. Currently, stacking various classification networks on pre-trained language models to perform fine-tuning is a common framework to end-to-end solve the biomedical relation extraction (BioRE) problem. However, the sequence-based pre-trained language models underutilize the graphical topology of language to some extent. In addition, sequence-oriented deep neural networks have limitations in processing graphical features.ResultsIn this paper, we propose a novel method for sentence-level BioRE task, BioEGRE (BioELECTRA and Graph pointer neural net-work for Relation Extraction), aimed at leveraging the linguistic topological features. First, the biomedical literature is preprocessed to retain sentences involving pre-defined entity pairs. Secondly, SciSpaCy is employed to conduct dependency parsing; sentences are modeled as graphs based on the parsing results; BioELECTRA is utilized to generate token-level representations, which are modeled as attributes of nodes in the sentence graphs; a graph pointer neural network layer is employed to select the most relevant multi-hop neighbors to optimize representations; a fully-connected neural network layer is employed to generate the sentence-level representation. Finally, the Softmax function is employed to calculate the probabilities. Our proposed method is evaluated on three BioRE tasks: a multi-class (CHEMPROT) and two binary tasks (GAD and EU-ADR). The results show that our method achieves F1-scores of 79.97% (CHEMPROT), 83.31% (GAD), and 83.51% (EU-ADR), surpassing the performance of existing state-of-the-art models.ConclusionThe experimental results on 3 biomedical benchmark datasets demonstrate the effectiveness and generalization of BioEGRE, which indicates that linguistic topology and a graph pointer neural network layer explicitly improve performance for BioRE tasks.

A Study of Minority Culture Outreach Translation Based on Cognitive Mapping Construction

Abstract In this paper, we first construct a cognitive neural network and use the knowledge representation learning technique for knowledge complementation and relation extraction to semantically match head-tail entity pairs with relations and accurately model the semantic relationships between entities and relations. Then, the vectors of each entity are mapped onto the hyperplane of a given relation by the cognitive mapping translation model, and vector translation operations are performed on the hyperplane. Finally, this paper’s translation model is compared with other models to compare the semantic representation translation, task processing function, and model complexity of each model so as to verify the superiority of this paper’s model for minority culture outreach translation. The results show that this paper’s method increases 3.4 and 1.2 BLEU values on average in the four translation tasks compared to the Imagination model and VAG model, respectively. When the number of iterations is 40, the MRR value of this paper’s model is 0.345, which is at least 0.1-0.5 more than the other models, thus indicating that this paper’s model has a better performance in outreach translation.

Document-level relation extraction with three channels

Relation extraction aims to identify the relation between two entities. Early works on relation extraction are performed within a sentence. Numerous entity pairs with relations are distributed across sentences, so the task of document-level relation extraction is proposed. The current works perform well on document-level relation extraction through graph neural network, transformer, etc. However, these works extract relation features from one perspective, resulting in limited extraction performance of the model. To break through the limitation, this paper proposes the DRETC model consisting of three channels. Channel one first constructs the mention matrix and then extracts the relation features between entities. Channel two performs sequence inference from mention to entity before generating the relation matrix. Moreover, the bridge method is proposed to introduce information in channel one that is beneficial to relation extraction. Channel three directly generates entity representations through text sequences to make up for the absence of text information related to the entity. The three channels extract relation features from three different perspectives, and the combination leads to better extraction performance. Experimental results show that the proposed DRETC model outperforms previous strong baseline models with one channel on the public dataset DocRED. Further analysis demonstrates that the combination of multiple channels has a better relation extraction performance than a single channel.

Document-level Relation Extraction with Relation Correlations

Document-level relation extraction faces two often overlooked challenges: long-tail problem and multi-label problem. Previous work focuses mainly on obtaining better contextual representations for entity pairs, hardly address the above challenges. In this paper, we analyze the co-occurrence correlation of relations, and introduce it into the document-level relation extraction task for the first time. We argue that the correlations can not only transfer knowledge between data-rich relations and data-scarce ones to assist in the training of long-tailed relations, but also reflect semantic distance guiding the classifier to identify semantically close relations for multi-label entity pairs. Specifically, we use relation embedding as a medium, and propose two co-occurrence prediction sub-tasks from both coarse- and fine-grained perspectives to capture relation correlations. Finally, the learned correlation-aware embeddings are used to guide the extraction of relational facts. Substantial experiments on two popular datasets (i.e., DocRED and DWIE) are conducted, and our method achieves superior results compared to baselines. Insightful analysis also demonstrates the potential of relation correlations to address the above challenges. The data and code are released at https://github.com/RidongHan/DocRE-Co-Occur.

Prompt Language Learner with Trigger Generation for Dialogue Relation Extraction

Dialogue relation extraction identifies semantic relations between entity pairs in dialogues. This research explores a methodology harnessing the potential of prompt-based fine-tuning paired with a trigger-generation approach. Capitalizing on the intrinsic knowledge of pre-trained language models, this strategy employs triggers that underline the relation between entities decisively. In particular, diverging from the conventional extractive methods seen in earlier research, our study leans towards a generative manner for trigger generation. The dialogue-based relation extraction (DialogeRE) benchmark dataset features multi-utterance environments of colloquial speech by multiple speakers, making it critical to capture meaningful clues for inferring relational facts. In the benchmark, empirical results reveal significant performance boosts in few-shot scenarios, where the availability of examples is notably limited. Nevertheless, the scarcity of ground-truth triggers for training hints at potential further refinements in the trigger-generation module, especially when ample examples are present. When evaluating the challenges of dialogue relation extraction, combining prompt-based learning with trigger generation offers pronounced improvements in both full-shot and few-shot scenarios. Specifically, integrating a meticulously crafted manual initialization method with the prompt-based model—considering prior distributional insights and relation class semantics—substantially surpasses the baseline. However, further advancements in trigger generation are warranted, especially in data-abundant contexts, to maximize performance enhancements.

Using Augmented Small Multimodal Models to Guide Large Language Models for Multimodal Relation Extraction

Multimodal Relation Extraction (MRE) is a core task for constructing Multimodal Knowledge images (MKGs). Most current research is based on fine-tuning small-scale single-modal image and text pre-trained models, but we find that image-text datasets from network media suffer from data scarcity, simple text data, and abstract image information, which requires a lot of external knowledge for supplementation and reasoning. We use Multimodal Relation Data augmentation (MRDA) to address the data scarcity problem in MRE, and propose a Flexible Threshold Loss (FTL) to handle the imbalanced entity pair distribution and long-tailed classes. After obtaining prompt information from the small model as a guide model, we employ a Large Language Model (LLM) as a knowledge engine to acquire common sense and reasoning abilities. Notably, both stages of our framework are flexibly replaceable, with the first stage adapting to multimodal related classification tasks for small models, and the second stage replaceable by more powerful LLMs. Through experiments, our EMRE2llm model framework achieves state-of-the-art performance on the challenging MNRE dataset, reaching an 82.95% F1 score on the test set.

Improving few-shot relation extraction through semantics-guided learning

Few-shot relation extraction (few-shot RE) aims to recognize relations between the entity pair in a given text by utilizing very few annotated instances. As a simple yet efficient approach, prototype network-based methods often directly incorporate relation information to enhance prototype representation or leverage contrastive learning to mitigate prediction confusion. Despite achieving good results, the above methods are still susceptible to false judgments of outlier samples and confusion of similar classes. To address these issues, we propose a novel Semantics-Guided Learning (SemGL) method that more effectively utilizes relation information to enhance both the representations of instances and prototypes for improving the performance of few-shot RE. First, SemGL employs the prompt encoder to encode various prompt templates of instances and relation information and obtains more accurate semantic representations of instances, instance prototypes, and concept prototypes via the prompt enhancement from large language models. Then, SemGL introduces a novel technique called relation graph learning, which leverages concept prototypes to cluster homogeneous instances together, emphasizing relation-specific features of concrete instances. Simultaneously, SemGL employs instance-level contrastive learning between instance prototypes and support instances to distinguish between intra-class instances and inter-class instances to promote shared features among intra-class instances. Additionally, prototype-level contrastive learning leverages concept prototypes to pull closer relation-specific features of the concept prototype and shared features of the instance prototype from the same relation. Finally, SemGL utilizes new relation prototypes that integrate interpretable features of concept prototypes and shared features of instance prototypes for prediction. Experimental results on two publicly available few-shot RE datasets demonstrate the effectiveness and efficiency of SemGL in introducing relation information, with particularly promising results for the domain adaptation challenge task.

A hierarchical convolutional model for biomedical relation extraction

Biomedical relation extraction aims to extract the interactive relations between biomedical entities in a sentence. In biomedical literature, the sentence lengths are longer compared to general domain. Because all entity pairs in a sentence share the same contextual features and each of them should be evaluated, the biomedical relation extract task suffers from more serious semantic overlapping and data imbalance problems. In this study, a hierarchical convolutional model is proposed to address these problems. This model organizes the convolution operation in a hierarchical structure. First, after a sentence is divided into several channels by using the semantic structure of a relation instance, a token-level convolution is used to encodes local contextual features of different channels. Then, a channel-level convolution is designed to encode global semantic dependencies of a sentence. The hierarchical convolution is effective to learn contextual features and semantic dependencies relevant to ordered named entities, which enhance the discriminability of a neural network for biomedical relation extraction. The proposed method was evaluated on seven public datasets for three tasks: including protein-protein, drug-drug, and chemical-protein interactions. It outperformed state-of-the-art performance about 3.7% in the F1 score.

CLRN: A reasoning network for multi-relation question answering over Cross-lingual Knowledge Graphs

Cross-lingual Knowledge Graphs-based Question Answering (CLKGQA) requires the question answering (QA) system to combine the knowledge graphs (KGs) in different languages to obtain answers to input questions. In previous works, the common idea is to merge Cross-lingual Knowledge Graphs (CLKGs) into a single KG through aligned entity pairs and then treat it as a traditional KG-based QA. However, as demonstrated by Tan et al. (2023), existing Entity Alignment (EA) models cannot generate highly accurate aligned entity pairs for CLKGs. Therefore, two issues need to be addressed in the CLKGQA task: (1) Remove the dependency of the QA model on the fused KG; (2) Improve the performance of the EA model in obtaining aligned entity pairs from locally isomorphic CLKGs. To solve the above two issues, this paper presents Cross-lingual Reasoning Network (CLRN), a novel multi-hop QA model that allows switching knowledge graphs at any stage of the multi-hop reasoning. Furthermore, we establish an iterative framework that combines CLRN and EA model, in which CLRN is used for extracting potential alignment triple pairs from CLKGs during the QA process. The extracted triple pairs provide pseudo-aligned entities, and the additional aligned entity pairs are used to mine missing relations between entities in CLKGs. These pseudo-aligned entity pairs and relations improve the performance of the EA model, resulting in higher accuracy in QA. Extensive experiments demonstrate the effectiveness of the proposed model, which outperforms the baseline approaches. Through iterative enhancement, the performance of the EA model has also been improved by > 1.0 % in [email protected] and [email protected], and the improvement is statistically significant in the confidence interval of p<0.01. Moreover, our work discusses the correlation between QA and EA from the side of QA, which has reference value for the follow-up exploration of related communities. We have open-sourced our dataset and code, which is available at the URL https://github.com/tan92hl/Cross-lingual-Reasoning-Network-for-CLKGQA.

Document-level relation extraction with entity mentions deep attention

Document-level Relation Extraction(DocRE) aims to extract relations between entities from documents. In contrast to sentence-level relation extraction, it requires extracting semantic relations from multiple sentences. It is necessary to further improve the performance of the above algorithm in order to extract document-level relation. Therefore, the DocRE algorithms have to deal with more complex entity structure relationships and the need to unite semantic relationships between different sentences when reasoning about relationships between entities. The proposed algorithms fail to infer relationships between entities when dealing with complex entity structure relationships. In this paper, we propose an entity mentions deep attention framework that efficiently infers entity relationships through entity structure and contextual information. Firstly, a structural dependency module of entities is designed to achieve interaction between different mentions of the entity. Secondly, a deep contextual attention component proposed to enrich the semantic information between entities by entity-related contexts. Finally, we use a distance mapping component to solve the problem of entity pairs that are far away from each other. According to our implementation results, our model outperforms the state-ofthe-art models on three public datasets DocRED, DGA, and CDR.

Learning entity-oriented representation for biomedical relation extraction

Biomedical Relation Extraction (BioRE) aims to automatically extract semantic relations for given entity pairs and is of great significance in biomedical research. Current popular methods often utilize pretrained language models to extract semantic features from individual input instances, which frequently suffer from overlapping semantics. Overlapping semantics refers to the situation in which a sentence contains multiple entity pairs that share the same context, leading to highly similar information between these entity pairs. In this study, we propose a model for learning Entity-oriented Representation (EoR) that aims to improve the performance of the model by enhancing the discriminability between entity pairs. It contains three modules: sentence representation, entity-oriented representation, and output. The first module learns the global semantic information of the input instance; the second module focuses on extracting the semantic information of the sentence from the target entities; and the third module enhances distinguishability among entity pairs and classifies the relation type. We evaluated our approach on four BioRE tasks with eight datasets, and the experiments showed that our EoR achieved state-of-the-art performance for PPI, DDI, CPI, and DPI tasks. Further analysis demonstrated the benefits of entity-oriented semantic information in handling multiple entity pairs in the BioRE task.

Multiple dependence representation of attention graph convolutional network relation extraction model

AbstractDependency analysis can better help neural network to capture semantic features in sentences, so as to extract entity relation. Currently, hard pruning strategies and soft pruning strategies based on dependency tree structure coding have been proposed to balance beneficial additional information and adverse interference in extraction tasks. A new model based on graph convolutional networks, which uses a variety of representations describing dependency trees from different perspectives and combining these representations to obtain a better sentence representation for relation classification is proposed. A newly defined module is added, and this module uses the attention mechanism to capture deeper semantic features from the context representation as the global semantic features of the input text, thus helping the model to capture deeper semantic information at the sentence level for relational extraction tasks. In order to get more information about a given entity pair from the input sentence, the authors also model implicit co‐references (references) to entities. This model can extract semantic features related to the relationship between entities from sentences to the maximum extent. The results show that the model in this paper achieves good results on SemEval2010‐Task8 and KBP37 datasets.