Entity Pairs Research Articles

Distant supervised relation extraction with position feature attention and selective bag attention

Distant supervision greatly reduces manual consumption by automatically labeling data. The relation extraction methods under distant supervision divide sentences with the same entity pair into a bag, and perform training and testing on these sentence bags. The existing distant supervised relation extraction methods ignore two facts. First, there are many sentences where the target entity pairs appear multiple times. Second, the noise between sentence bags is different, and the sentences of some bags are even all mislabeled. To solve these two problems, we propose a novel relation extraction method with position feature attention and selective bag attention. The position feature attention is employed to obtain the weighted sentence representation with different position features by calculating all position combinations of the target entity pair. A bag with large noise and a bag with small noise are selected through the selective bag attention mechanism to form a bag pair, and training is performed at the level of the bag pair, which denoises at the bag level and at the same time balances the noise between different bag pairs. The experimental results show that our method is effective and outperforms several competitive baseline methods.

Hypergraph reconstruction from network data

Networks can describe the structure of a wide variety of complex systems by specifying which pairs of entities in the system are connected. While such pairwise representations are flexible, they are not necessarily appropriate when the fundamental interactions involve more than two entities at the same time. Pairwise representations nonetheless remain ubiquitous, because higher-order interactions are often not recorded explicitly in network data. Here, we introduce a Bayesian approach to reconstruct latent higher-order interactions from ordinary pairwise network data. Our method is based on the principle of parsimony and only includes higher-order structures when there is sufficient statistical evidence for them. We demonstrate its applicability to a wide range of datasets, both synthetic and empirical.

HiAM: A Hierarchical Attention based Model for knowledge graph multi-hop reasoning

Learning to reason in large-scale knowledge graphs has attracted much attention from research communities recently. This paper targets a practical task of multi-hop reasoning in knowledge graphs, which can be applied in various downstream tasks such as question answering, and recommender systems. A key challenge in multi-hop reasoning is to synthesize structural information (e.g., paths) in knowledge graphs to perform deeper reasoning. Existing methods usually focus on connection paths between each entity pair. However, these methods ignore predecessor paths before connection paths and regard entities and relations within every single path as equally important. With our observations, predecessor paths before connection paths can provide more accurate semantic representations. Furthermore, entities and relations in a single path contribute variously to the right answers. To this end, we propose a novel model HiAM (Hierarchical Attention based Model) for knowledge graph multi-hop reasoning. HiAM makes use of predecessor paths to provide more accurate semantics for entities and explores the effects of different granularities. Firstly, we extract predecessor paths of head entities and connection paths between each entity pair. Then, a hierarchical attention mechanism is designed to capture the information of different granularities, including entity/relation-level and path-level features. Finally, multi-granularity features are fused together to predict the right answers. We go one step further to select the most significant path as the explanation for predicted answers. Comprehensive experimental results demonstrate that our method achieves competitive performance compared with the baselines on three benchmark datasets.

Document-Level Relation Extraction with Reconstruction

In document-level relation extraction (DocRE), graph structure is generally used to encode relation information in the input document to classify the relation category between each entity pair, and has greatly advanced the DocRE task over the past several years. However, the learned graph representation universally models relation information between all entity pairs regardless of whether there are relationships between these entity pairs. Thus, those entity pairs without relationships disperse the attention of the encoder-classifier DocRE for ones with relationships, which may further hind the improvement of DocRE. To alleviate this issue, we propose a novel encoder-classifier-reconstructor model for DocRE. The reconstructor manages to reconstruct the ground-truth path dependencies from the graph representation, to ensure that the proposed DocRE model pays more attention to encode entity pairs with relationships in the training. Furthermore, the reconstructor is regarded as a relationship indicator to assist relation classification in the inference, which can further improve the performance of DocRE model. Experimental results on a large-scale DocRE dataset show that the proposed model can significantly improve the accuracy of relation extraction on a strong heterogeneous graph-based baseline. The code is publicly available at https://github.com/xwjim/DocRE-Rec.

CAFE: Knowledge graph completion using neighborhood-aware features

Knowledge Graphs (KGs) currently contain a vast amount of structured information in the form of entities and relations. Because KGs are often constructed automatically by means of information extraction processes, they may miss information that was either not present in the original source or not successfully extracted. As a result, KGs might lack useful and valuable information. Current approaches that aim to complete missing information in KGs have two main drawbacks. First, some have a dependence on embedded representations, which impose a very expensive preprocessing step and need to be recomputed again as the KG grows. Second, others are based on long random paths that may not cover all relevant information, whereas exhaustively analyzing all possible paths between entities is very time-consuming. In this paper, we present an approach to complete KGs based on evaluating candidate triples using a set of neighborhood-based features. Our approach exploits the highly connected nature of KGs by analyzing the entities and relations surrounding any given pair of entities, while avoiding full recomputations as new entities are added. Our results indicate that our proposal is able to identify correct triples with a higher effectiveness than other state-of-the-art approaches, achieving higher average F1 scores in all tested datasets. Therefore, we conclude that the information present in the vicinities of the two entities within a candidate triple can be leveraged to determine whether that triple is missing from the KG or not.

MMKE: A Multi-Model Knowledge Extraction System from Unstructured Texts

In this work, we present a Multi-Model Knowledge Extraction (MMKE) System which consists of two unstructured text extraction models (RelationSO model and SubjectRO model) based on a multi-task learning framework. Instead of recognizing entity first and then predicting relationships between entity pairs in previous works, MMKE detects subject and corresponding relationships before extracting objects to cope with the diverse object-type problem, overlapping problem and non-predefined relation problem. Our system accepts unstructured text as input, from which it automatically extracts triplets knowledge (subject, relation, object). More importantly, we incorporate a number of user-friendly extraction functionalities, such as multi-format uploading, one-click extractions, knowledge editing and graphical displays. The demonstration video is available at this link: https://youtu.be/HtOPJrGhSxk.

Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling

Document-level relation extraction (RE) poses new challenges compared to its sentence-level counterpart. One document commonly contains multiple entity pairs, and one entity pair occurs multiple times in the document associated with multiple possible relations. In this paper, we propose two novel techniques, adaptive thresholding and localized context pooling, to solve the multi-label and multi-entity problems. The adaptive thresholding replaces the global threshold for multi-label classification in the prior work with a learnable entities-dependent threshold. The localized context pooling directly transfers attention from pre-trained language models to locate relevant context that is useful to decide the relation. We experiment on three document-level RE benchmark datasets: DocRED, a recently released large-scale RE dataset, and two datasets CDRand GDA in the biomedical domain. Our ATLOP (Adaptive Thresholding and Localized cOntext Pooling) model achieves an F1 score of 63.4, and also significantly outperforms existing models on both CDR and GDA. We have released our code at https://github.com/wzhouad/ATLOP.

A Unified Multi-Task Learning Framework for Joint Extraction of Entities and Relations

Joint extraction of entities and relations focuses on detecting entity pairs and their relations simultaneously with a unified model. Based on the extraction order, previous works mainly solve this task through relation-last, relation-first and relation-middle manner. However, these methods still suffer from the template-dependency, non-entity detection and non-predefined relation prediction problem. To overcome these challenges, in this paper, we propose a unified multi-task learning framework to divide the task into three interacted sub-tasks. Specifically, we first introduce the type-attentional method for subject extraction to provide prior type information explicitly. Then, the subject-aware relation prediction is presented to select useful relations based on the combination of global and local semantics. Third, we propose a question generation based QA method for object extraction to obtain diverse queries automatically. Notably, our method detects subjects or objects without relying on NER models and thus it is capable of dealing with the non-entity scenario. Finally, three sub-tasks are integrated into a unified model through parameter sharing. Extensive experiments demonstrate that the proposed framework outperforms all the baseline methods on two benchmark datasets, and further achieve excellent performance for non-predefined relations.

SRGCN: Graph-based multi-hop reasoning on knowledge graphs

Learning to infer missing links is one of the fundamental tasks in the knowledge graph. Instead of reasoning based on separate paths in the existing methods, in this paper, we propose a new model, Sequential Relational Graph Convolutional Network (SRGCN), which treats the multiple paths between an entity pair as a sequence of subgraphs. Specifically, to reason the relationship between two entities, we first construct a graph for the entities based on the knowledge graph and serialize the graph to a sequence. For each hop in the sequence, Relational Graph Convolutional Network (R-GCN) is then applied to update the embeddings of the entities. The updated embedding of the tail entity contains information of the entire graph, hence the relationship between two entities can be inferred from it. Compared to the existing approaches that deal with paths separately, SRGCN treats the graph as a whole, which can encode structural information and interactions between paths better. Experiments show that SRGCN outperforms path-based baselines on both link and fact prediction tasks. We also show that SRGCN is highly efficient in the sense that only one epoch of training is enough to achieve high accuracy, and even partial datasets can lead to competitive performance.

A Hybrid Model for Family History Information Identification and Relation Extraction: Development and Evaluation of an End-to-End Information Extraction System.

BackgroundFamily history information is important to assess the risk of inherited medical conditions. Natural language processing has the potential to extract this information from unstructured free-text notes to improve patient care and decision making. We describe the end-to-end information extraction system the Medical University of South Carolina team developed when participating in the 2019 National Natural Language Processing Clinical Challenge (n2c2)/Open Health Natural Language Processing (OHNLP) shared task.ObjectiveThis task involves identifying mentions of family members and observations in electronic health record text notes and recognizing the 2 types of relations (family member-living status relations and family member-observation relations). Our system aims to achieve a high level of performance by integrating heuristics and advanced information extraction methods. Our efforts also include improving the performance of 2 subtasks by exploiting additional labeled data and clinical text-based embedding models.MethodsWe present a hybrid method that combines machine learning and rule-based approaches. We implemented an end-to-end system with multiple information extraction and attribute classification components. For entity identification, we trained bidirectional long short-term memory deep learning models. These models incorporated static word embeddings and context-dependent embeddings. We created a voting ensemble that combined the predictions of all individual models. For relation extraction, we trained 2 relation extraction models. The first model determined the living status of each family member. The second model identified observations associated with each family member. We implemented online gradient descent models to extract related entity pairs. As part of postchallenge efforts, we used the BioCreative/OHNLP 2018 corpus and trained new models with the union of these 2 datasets. We also pretrained language models using clinical notes from the Medical Information Mart for Intensive Care (MIMIC-III) clinical database.ResultsThe voting ensemble achieved better performance than individual classifiers. In the entity identification task, our top-performing system reached a precision of 78.90% and a recall of 83.84%. Our natural language processing system for entity identification took 3rd place out of 17 teams in the challenge. We ranked 4th out of 9 teams in the relation extraction task. Our system substantially benefited from the combination of the 2 datasets. Compared to our official submission with F1 scores of 81.30% and 64.94% for entity identification and relation extraction, respectively, the revised system yielded significantly better performance (P<.05) with F1 scores of 86.02% and 72.48%, respectively.ConclusionsWe demonstrated that a hybrid model could be used to successfully extract family history information recorded in unstructured free-text notes. In this study, our approach to entity identification as a sequence labeling problem produced satisfactory results. Our postchallenge efforts significantly improved performance by leveraging additional labeled data and using word vector representations learned from large collections of clinical notes.

TPmod: A Tendency-Guided Prediction Model for Temporal Knowledge Graph Completion

Temporal knowledge graphs (TKGs) have become useful resources for numerous Artificial Intelligence applications, but they are far from completeness. Inferring missing events in temporal knowledge graphs is a fundamental and challenging task. However, most existing methods solely focus on entity features or consider the entities and relations in a disjoint manner. They do not integrate the features of entities and relations in their modeling process. In this paper, we propose TPmod, a tendency-guided prediction model, to predict the missing events for TKGs (extrapolation). Differing from existing works, we propose two definitions for TKGs: the Goodness of relations and the Closeness of entity pairs. More importantly, inspired by the attention mechanism, we propose a novel tendency strategy to guide our aggregated process. It integrates the features of entities and relations, and assigns varying weights to different past events. What is more, we select the Gate Recurrent Unit (GRU) as our sequential encoder to model the temporal dependency in TKGs. Besides, the Softmax function is employed to generate the final decreasing group of candidate entities. We evaluate our model on two TKG datasets: GDELT-5 and ICEWS-250. Experimental results show that our method has a significant and consistent improvement compared to state-of-the-art baselines.

DiS-ReX: A Multilingual Dataset for Distantly Supervised Relation Extraction

Distant supervision (DS) is a well established technique for creating large-scale datasets for relation extraction (RE) without using human annotations. However, research in DS-RE has been mostly limited to the English language. Constraining RE to a single language inhibits utilization of large amounts of data in other languages which could allow extraction of more diverse facts. Very recently, a dataset for multilingual DS-RE has been released. However, our analysis reveals that the proposed dataset exhibits unrealistic characteristics such as 1) lack of sentences that do not express any relation, and 2) all sentences for a given entity pair expressing exactly one relation. We show that these characteristics lead to a gross overestimation of the model performance. In response, we propose a new dataset, DiS-ReX, which alleviates these issues. Our dataset has more than 1.5 million sentences, spanning across 4 languages with 36 relation classes + 1 no relation (NA) class. We also modify the widely used bag attention models by encoding sentences using mBERT and provide the first benchmark results on multilingual DS-RE. Unlike the competing dataset, we show that our dataset is challenging and leaves enough room for future research to take place in this field.

Document-level relation extraction with Entity-Selection Attention

Document-level relation extraction is a complex natural language processing task that predicts relations of entity pairs by capturing the critical semantic features on entity pairs from the document. However, current methods usually consider that the entity pairs contain the vast majority of information which can represent relational facts, and thus focus on modeling the entity pair, ignoring features on whole document and sentences. In the document-level relation extraction, the distance between entity pairs is relatively long. Judging the relation between entities usually requires reading many sentences or the whole document. Therefore, sentences and documents are particularly crucial for document-level relation extraction. In order to make full use of the multi-level information of sentences and documents, this paper proposes a document-level relation extraction framework with two advantages. First, we use the encoder to obtain the semantic features about the document and use the inter-sentence attention based on entity pairs to dynamically capture the features of multiple vital sentences. Second, we design a document gating that combines sentence-level features with document-level features to predict relations. Extensive experiments on a benchmark dataset have well-validated the effectiveness of the proposed method.

D‐BERT: Incorporating dependency‐based attention into BERT for relation extraction

Relation extraction between entity pairs is an increasingly critical area in natural language processing. Recently, the pre-trained bidirectional encoder representation from transformer (BERT) performs excellently on the text classification or sequence labelling tasks. Here, the high-level syntactic features that consider the dependency between each word and the target entities into the pre-trained language models are incorporated. Our model also utilizes the intermediate layers of BERT to acquire different levels of semantic information and designs multi-granularity features for final relation classification. Our model offers a momentous improvement over the published methods for the relation extraction on the widely used data sets.

Entity Relation Extraction Based on Entity Indicators

Relation extraction aims to extract semantic relationships between two specified named entities in a sentence. Because a sentence often contains several named entity pairs, a neural network is easily bewildered when learning a relation representation without position and semantic information about the considered entity pair. In this paper, instead of learning an abstract representation from raw inputs, task-related entity indicators are designed to enable a deep neural network to concentrate on the task-relevant information. By implanting entity indicators into a relation instance, the neural network is effective for encoding syntactic and semantic information about a relation instance. Organized, structured and unified entity indicators can make the similarity between sentences that possess the same or similar entity pair and the internal symmetry of one sentence more obviously. In the experiment, a systemic analysis was conducted to evaluate the impact of entity indicators on relation extraction. This method has achieved state-of-the-art performance, exceeding the compared methods by more than 3.7%, 5.0% and 11.2% in F1 score on the ACE Chinese corpus, ACE English corpus and Chinese literature text corpus, respectively.

A Proximity Approach to Understanding University-Industry Collaborations for Innovation in Non-Local Context: Exploring the Catch-Up Role of Regional Absorptive Capacity

The article aims to explore which types of proximity approach can foster university–industry (U-I) collaborations for innovation and discuss the role of different dimension regional absorptive capacity in cooperation to promote inter-regional partnerships from “unbalance” to “coordination”. Therefore, we intercept 484 pairs of cooperative entities and analyze proximity effects and heterogeneity cross-regional U-I collaborations by hierarchical regression. The results show: (1) In non-local contexts, geographic distance is not a hamper for improving innovation performance. The economic development level has no significantly different effects on such a role. (2) Technological proximity plays a negative role in increasing innovative performance, and the eastern region has the most noticeable results. (3) The closer in social distance can get more innovation performance in eastern and western, but the central area negatively affects. (4) The U-I collaborations for innovation performance-enhancing advantages are not equal for all regions but are moderating by specific regional absorptive capacity dimensions. The areas with a higher level of internal human capital can get more catch-up effects. The lagging regions should increase talents to promote cross-regional cooperation for catching up. In contrast, the prosperous areas should take advantage of the talent-gathering effects to promote knowledge spillover.

Unsupervised DNF Blocking for Efficient Linking of Knowledge Graphs and Tables

Entity Resolution (ER) is the problem of identifying co-referent entity pairs across datasets, including knowledge graphs (KGs). ER is an important prerequisite in many applied KG search and analytics pipelines, with a typical workflow comprising two steps. In the first ’blocking’ step, entities are mapped to blocks. Blocking is necessary for preempting comparing all possible pairs of entities, as (in the second ‘similarity’ step) only entities within blocks are paired and compared, allowing for significant computational savings with a minimal loss of performance. Unfortunately, learning a blocking scheme in an unsupervised fashion is a non-trivial problem, and it has not been properly explored for heterogeneous, semi-structured datasets, such as are prevalent in industrial and Web applications. This article presents an unsupervised algorithmic pipeline for learning Disjunctive Normal Form (DNF) blocking schemes on KGs, as well as structurally heterogeneous tables that may not share a common schema. We evaluate the approach on six real-world dataset pairs, and show that it is competitive with supervised and semi-supervised baselines.

Knowledge Graph Reasoning Based on Tensor Decomposition and MHRP-Learning

In the process of learning and reasoning knowledge graph, the existing tensor decomposition technology only considers the direct relationship between entities in knowledge graph. However, it ignores the characteristics of the graph structure of knowledge graph. To solve this problem, a knowledge graph reasoning algorithm based on multihop relational paths learning (MHRP-learning) and tensor decomposition is proposed in this paper. Firstly, MHRP-learning is adopted to obtain the relationship path between entity pairs in the knowledge graph. Then, the tensor decomposition is performed to get a novel learning framework. Finally, experiments show that the proposed method achieves advanced results, and it is applicable to knowledge graph reasoning.

A Relational Adaptive Neural Model for Joint Entity and Relation Extraction.

Relation extraction is a popular subtask in natural language processing (NLP). In the task of entity relation joint extraction, overlapping entities and multi-type relation extraction in overlapping triplets remain a challenging problem. The classification of relations by sharing the same probability space will ignore the correlation information among multiple relations. A relational-adaptive entity relation joint extraction model based on multi-head self-attention and densely connected graph convolution network (which is called MA-DCGCN) is proposed in the paper. In the model, the multi-head attention mechanism is specifically used to assign weights to multiple relation types among entities so as to ensure that the probability space of multiple relation is not mutually exclusive. This mechanism also predicts the strength of the relationship between various relationship types and entity pairs flexibly. The structure information of deeper level in the text graph is extracted by the densely connected graph convolution network, and the interaction information of entity relation is captured. To demonstrate the superior performance of our model, we conducted a variety of experiments on two widely used public datasets, NYT and WebNLG. Extensive results show that our model achieves state-of-the-art performance. Especially, the detection effect of overlapping triplets is significantly improved compared with the several existing mainstream methods.

Joint extraction of entities and overlapping relations using source-target entity labeling

Joint extraction of entities and overlapping relations has attracted considerable attention in recent research. Existing relation extraction methods rely on a training set that is labeled by the distant supervision method for supervised relation extraction. However, the drawbacks of these methods are that large-scale unlabeled data cannot be used and the quality of labeled data cannot be guaranteed. Moreover, owing to the relatively complex overlapping relations, it is difficult to perform joint entity-relation extraction accurately. In this study, we propose an end-to-end neural network model (BERT-JEORE) for the joint extraction of entities and overlapping relations. First, we use the BERT-based parameter-sharing layer to capture the joint features of entities and overlapping relations. Then, we implement the source-target BERT model to assign entity labels to each token in a sentence, thereby expanding the amount of labeled data and improving their quality. Finally, we design a three-step overlapping relations extraction model and use it to predict the relations between all entity pairs. Experiments conducted on two public datasets show that BERT-JEORE achieves the best current performance and outperforms the baseline models by a significant margin. Further analysis shows that our model can effectively capture different types of overlapping relational triplets in a sentence.