Entity Pairs Research Articles

Multi-Relation Extraction via A Global-Local Graph Convolutional Network

Relation extraction (RE) extracts the semantic relations among entities in a sentence, which converts the unstructured text into structured and easy-to-understand information. Although RE has been studied over decades, it still faces two kinds of research challenges that are not well addressed thus far: 1) joint consideration of the global sentence structure and the local entity interaction, and 2) effective solution to the overlapping triplets within the same sentence. To tackle these issues, in this paper, we present global-local graph-based convolutional network towards multi-relation extraction, GAME for short. In particular, we devise two layers of graph convolutional network (GCN) with different structures to complete the feature extraction, which effectively improves the capability of relation extraction. Moreover, we implement the GCN layers via the pure GCN model and graph attention network respectively for further comparison. Besides, we adopt a classification strategy to extract relation among entity pairs, assisting in solving the more complicated problem of overlapping triplets in RE. Extensive experiments have been conducted on two widely-used benchmark datasets, demonstrating that our model significantly outperforms several state-of-the-art methods. As a side product, we have released our data, codes and parameter settings to facilitate other researchers.

Span-based model for overlapping entity recognition and multi-relations classification in the food domain

<abstract><p>Information extraction (IE) is an important part of the entire knowledge graph lifecycle. In the food domain, extracting information such as ingredient and cooking method from Chinese recipes is crucial to safety risk analysis and identification of ingredient. In comparison with English, due to the complex structure, the richness of information in word combination, and lack of tense, Chinese IE is much more challenging. This dilemma is particularly prominent in the food domain with high-density knowledge, imprecise syntactic structure. However, existing IE methods focus only on the features of entities in a sentence, such as context and position, and ignore features of the entity itself and the influence of self attributes on prediction of inter entity relationship. To solve the problems of overlapping entity recognition and multi-relations classification in the food domain, we propose a span-based model known as SpIE for IE. The SpIE uses the span representation for each possible candidate entity to capture span-level features, which transforms named entity recognition (NER) into a classification mission. Besides, SpIE feeds extra information about the entity into the relation classification (RC) model by considering the effect of entity's attributes (both the entity mention and entity type) on the relationship between entity pairs. We apply SpIE on two datasets and observe that SpIE significantly outperforms the previous neural approaches due to capture the feature of overlapping entity and entity attributes, and it remains very competitive in general IE.</p></abstract>

Matching Network of Ontologies: A Random Walk and Frequent Itemsets Approach

A System of Systems (SoS) is a complex set of IS (Information Systems) created by the aggregation and interconnection of ISs. SoS brings unexpected behavior and functionality during its construction to the benefit of its users and the SoS itself. The integration of SoSs is a matter of time. However, as SoSs can behave as an organization, it may be inappropriate to try to integrate individual IS members separately. On the other hand, manually integrating the SoS as a whole can be unfeasible due to its complexity. If an SoS has ontologies modeling the knowledge, the integration of SoSs can be translated into a problem of a network of ontologies alignment. However, it creates another challenge: computing each possible pair of entities inside each network’s ontology can have unfeasible execution time, even using the best matchers available. In this article, we propose to mine the data from the networks using random walks and frequent item sets algorithm and discover relevant nodes elected as candidate entities. Next, the networks are pruned by an algebraic method eliminating identical entities. The relevant nodes are reinserted in the network to avoid losing essential correspondences. After the pre-processing step, data is sent to two matchers to obtain metrics and compare the results with the pairwise brute force approach and previous work. We identified relevant nodes with recall up to 0.75. The results are promising since precision and recall are closer to the force brute, and execution time is shorter, even more, when the size of the networks and the number of ontologies to be compared increases. We validate our approach using ontologies created from the OAEI (Ontology Alignment Evaluation Initiative).

Graph Attention Networks for Multiple Pairs of Entities and Aspects Sentiment Analysis in Long Texts

Graph Attention Networks for Multiple Pairs of Entities and Aspects Sentiment Analysis in Long Texts

BERT gated multi-window attention network for relation extraction

Entity relation extraction aims to identify the semantic relation between entity pairs in a sentence and is an important technical support for downstream tasks such as question answering systems and semantic searching. Existing relation extraction models mainly rely on neural networks to extract the semantic information of sentences, ignoring the critical role of important phrase information in relation extraction. For this problem, this paper proposes a relation extraction model based on BERT gated multi-window attention network (BERT-GMAN). The model first uses BERT to extract the semantic representation features of the sentence and its constraint information. Secondly, it constructs the key phrases extraction network to obtain multi-granularity phrase information and uses element-wise max pooling to select key phrases features. Thirdly, it adopts classification feature perception network to further filter and globally perceive key phrase feature to form the overall features of relation classification. Finally, it combines with Softmax classifier to perform relation extraction. The experimental results on the Semeval-2010 Task 8 dataset show that the performance of the model in this paper is further improved compared with the existing methods, and the F1-score reaches 90.25%.

One-Shot Relation Learning for Knowledge Graphs via Neighborhood Aggregation and Paths Encoding

The relation learning between two entities is an essential task in knowledge graph (KG) completion that has received much attention recently. Previous work almost exclusively focused on relations widely seen in the original KGs, which means that enough training data are available for modeling. However, long-tail relations that only show in a few triples are actually much more common in practical KGs. Without sufficiently large training data, the performance of existing models on predicting long-tail relations drops impressively. This work aims to predict the relation under a challenging setting where only one instance is available for training. We propose a path-based one-shot relation prediction framework, which can extract neighborhood information of an entity based on the relation query attention mechanism to learn transferable knowledge among the same relation. Simultaneously, to reduce the impact of long-tail entities on relation prediction, we selectively fuse path information between entity pairs as auxiliary information of relation features. Experiments in three one-shot relation learning datasets show that our proposed framework substantially outperforms existing models on one-shot link prediction and relation prediction.

NLIRE: A Natural Language Inference method for Relation Extraction

Relation extraction task aims at detecting the semantic relation between a pair of entities in a given target sentence. However, previous methods lack the description of the relation definition, thus needing to model the implication of relations during training. To tackle this issue, we propose a natural language inference method for relation extraction. Given a premise and a hypothesis, the natural language inference task refers to predicting whether the facts in the premise necessarily imply the facts in the hypothesis. Specifically, for each relation type, we construct a relation description. These relation descriptions are the definition of relation, containing prior knowledge that helps model understand the meaning of relation. The given target sentence is viewed as the premise, and these descriptions are viewed as the hypotheses. Then model infers whether these hypotheses can be concluded from the premise. Based on the inference results, our model selects the relation corresponding to the most confident hypothesis as the prediction. Substantial experiments on SemEval2010 Task8 dataset demonstrate that the proposed method achieves state-of-the-art performance.

Combining embedding-based and symbol-based methods for entity alignment

The objective of entity alignment is to judge whether entities refer to the same object in the real world. Methods for entity alignment can be grossly divided into two groups: conventional symbol-based entity alignment methods and embedding-based entity alignment methods. Both groups of methods have advantages and disadvantages (which are detailed in Section 1). Therefore, combining the advantages of both methods might be a promising strategy. However, to the best of our knowledge, only the RTEA algorithm that was proposed in our previous conference paper (Proceeding of Pacific Rim International Conference on Artificial Intelligence, pp. 162–175, 2019) utilizes this strategy for entity alignment. This manuscript is an extended version of that conference paper, in which an improved algorithm, namely, ESEA (combining embedding-based and symbol-based methods for entity alignment), is proposed based on the following steps. First, a novel method for combining embedding models with symbol-based models is proposed. Entities with high vector similarities are obtained through a hybrid embedding model, and the final aligned entity pairs are calculated via symbol-based methods. Second, a series of symbol-based methods, instead of only the edit distance method in the original version, are combined with embedding-based methods for relation alignment. Third, we combine symbol-based and embedding-based methods in a more complicated framework with the objective of better exploiting the advantages of both methods. The experimental results on real-world datasets demonstrate that the proposed method outperformed several state-of-the-art embedding-based entity alignment approaches and outperformed our previous RTEA method.

COVID-19 Knowledge Extractor (COKE): A Curated Repository of Drug-Target Associations Extracted from the CORD-19 Corpus of Scientific Publications on COVID-19.

The COVID-19 pandemic has catalyzed a widespread effort to identify drug candidates and biological targets of relevance to SARS-COV-2 infection, which resulted in large numbers of publications on this subject. We have built the COVID-19 Knowledge Extractor (COKE), a web application to extract, curate, and annotate essential drug–target relationships from the research literature on COVID-19. SciBiteAI ontological tagging of the COVID Open Research Data set (CORD-19), a repository of COVID-19 scientific publications, was employed to identify drug–target relationships. Entity identifiers were resolved through lookup routines using UniProt and DrugBank. A custom algorithm was used to identify co-occurrences of the target protein and drug terms, and confidence scores were calculated for each entity pair. COKE processing of the current CORD-19 database identified about 3000 drug–protein pairs, including 29 unique proteins and 500 investigational, experimental, and approved drugs. Some of these drugs are presently undergoing clinical trials for COVID-19. The COKE repository and web application can serve as a useful resource for drug repurposing against SARS-CoV-2. COKE is freely available at https://coke.mml.unc.edu/, and the code is available at https://github.com/DnlRKorn/CoKE.

Three birds, one stone: A novel translation based framework for joint entity and relation extraction

Joint entity and relation extraction is an important task in natural language processing and knowledge graph construction. Existing studies mainly focus on three issues: redundant predictions, overlapping triples and relation connections. However, as far as we know, none of them is able to solve the three problems simultaneously in a unified architecture. To address this issue, in this paper, we propose a novel translation based unified framework. Specifically, the proposed framework contains two components: an entity tagger and a relation extractor. The former is used to recognize all candidate head entities and tail entities respectively. The latter predicts relations for every entity pair dynamically through ranking with translation mechanism. To show the superiority of the proposed framework, we instantiate it through the simplest binary entity tagger and TransE algorithm. Extensive experiments over two widely used datasets demonstrate that, even with the simplest components, the proposed framework can still achieve competitive performance with most previous baselines. Moreover, the framework is flexible. It enjoys further performance boost when employing more powerful entity tagger and knowledge graph embedding algorithm.

Modeling Relation Paths for Knowledge Graph Completion

Knowledge graphs (KG) often encounter knowledge incompleteness. The path reasoning that predicts the unknown path relation between pairwise entities based on existing facts is one of the most promising approaches to the knowledge graph completion. However, most conventional path reasoning methods exclusively consider the entity description included in fact triples, ignoring both the type information of entities and the interaction between different semantic representations. In this study, we propose a novel method, Type-aware Attentive Path Reasoning (TAPR), to complete the knowledge graph by simultaneously considering KG structural information, textual information, and type information. More specifically, we first leverage types to enrich the representational learning of entities and relationships. Next, we describe a type-level attention to select the most relevant type of given entity in a specific triple without any predefined rules or patterns to reduce the impact of noisy types. After learning the distributed representation of all paths, path-level attention assigns different weights to paths, from which relations among entity pairs are calculated. We conduct a series of experiments on a real-world dataset to demonstrate the effectiveness of TAPR. Experimental results show that our method significantly outperforms all baselines on link prediction and entity prediction tasks.

Matching Sensor Ontologies With Multi-Context Similarity Measure and Parallel Compact Differential Evolution Algorithm

Sensor ontology is able to resolve the sensor data heterogeneity issue among the Cybertwin-driven 6G based Internet of Everything (IoE) systems. However, due to human subjectivity, the sensor ontologies also suffer from the heterogeneity problem. To address this problem, it is necessary to execute the sensor Ontology Matching (OM) process, i.e., finding the identical entity pairs between two ontologies. To this end, we first propose a Multi-Context based ESM (MC-ESM) to measure the similarity of two entities by taking into consideration their semantic contexts. After that, a parallel compact Differential Evolution with Adaptive Step Length (pcDE-ASL) is proposed to find all entity mappings, which uses the probability presentation on the population to save the memory consumption, and the parallel processing mechanism and ASL to help the algorithm efficiently converge on the global optima. The experimental results show that pcDE-ASL is both effective and efficient.

Hybrid attention-based transformer block model for distant supervision relation extraction

With an exponential explosive growth of various digital text information, it is challenging to efficiently obtain specific knowledge from massive unstructured text information. As one basic task for natural language processing (NLP), relation extraction (RE) aims to extract semantic relations between entity pairs based on the given text. To avoid manual labeling of datasets, distant supervision relation extraction (DSRE) has been widely used, aiming to utilize knowledge base to automatically annotate datasets. Unfortunately, this method heavily suffers from wrong labelling due to its underlying strong assumptions. To address this issue, we propose a new framework using hybrid attention-based Transformer block with multi-instance learning for DSRE. More specifically, the Transformer block is, for the first time, used as a sentence encoder, which mainly utilizes multi-head self-attention to capture syntactic information at the word level. Then, a novel sentence-level attention mechanism is proposed to calculate the bag representation, aiming to exploit all useful information in each sentence. Experimental results on the public dataset New York Times (NYT) demonstrate that the proposed approach can outperform the state-of-the-art algorithms on the adopted dataset, which verifies the effectiveness of our model on the DSRE task.

Multi-Graph Cooperative Learning Towards Distant Supervised Relation Extraction

The Graph Convolutional Network (GCN) is a universal relation extraction method that can predict relations of entity pairs by capturing sentences’ syntactic features. However, existing GCN methods often use dependency parsing to generate graph matrices and learn syntactic features. The quality of the dependency parsing will directly affect the accuracy of the graph matrix and change the whole GCN’s performance. Because of the influence of noisy words and sentence length in the distant supervised dataset, using dependency parsing on sentences causes errors and leads to unreliable information. Therefore, it is difficult to obtain credible graph matrices and relational features for some special sentences. In this article, we present a Multi-Graph Cooperative Learning model (MGCL), which focuses on extracting the reliable syntactic features of relations by different graphs and harnessing them to improve the representations of sentences. We conduct experiments on a widely used real-world dataset, and the experimental results show that our model achieves the state-of-the-art performance of relation extraction.

Relation classification via BERT with piecewise convolution and focal loss.

Recent relation extraction models’ architecture are evolved from the shallow neural networks to natural language model, such as convolutional neural networks or recurrent neural networks to Bert. However, these methods did not consider the semantic information in the sequence or the distance dependence problem, the internal semantic information may contain the useful knowledge which can help relation classification. Focus on these problems, this paper proposed a BERT-based relation classification method. Compare with the existing Bert-based architecture, the proposed model can obtain the internal semantic information between entity pair and solve the distance semantic dependence better. The pre-trained BERT model after fine tuning is used in this paper to abstract the semantic representation of sequence, then adopt the piecewise convolution to obtain semantic information which influence the extraction results. Compare with the existing methods, the proposed method can achieve a better accuracy on relational extraction task because of the internal semantic information extracted in the sequence. While, the generalization ability is still a problem that cannot be ignored, and the numbers of the relationships are difference between different categories. In this paper, the focal loss function is adopted to solve this problem by assigning a heavy weight to less number or hard classify categories. Finally, comparing with the existing methods, the F1 metric of the proposed method can reach a superior result 89.95% on the SemEval-2010 Task 8 dataset.

Improving Distant Supervised Relation Extraction using shortest dependency path

Distant supervised relation extraction is the problem of classifying the semantic relation between a pair of entities in a given sentence, where data is created by aligning unlabelled text with knowledge base automatically. Prior researches indicated that most of the sentences in the distant supervised relation extraction setting are very long and benefit from the attention over words. We note that the shortest dependency path between the entity pair in the syntax analysis tree of a sentence can help word based attention. In this paper, we propose a noval distant supervised neural relation extraction method, which makes use of the shortest dependency path to supervise the learning of word attention. Through extensive experiments on benchmark datasets, we demonstrate effectiveness of our model.

Neural symbolic reasoning with knowledge graphs: Knowledge extraction, relational reasoning, and inconsistency checking

Knowledge graphs (KGs) express relationships between entity pairs, and many real-life problems can be formulated as knowledge graph reasoning (KGR). Conventional approaches to KGR have achieved promising performance but still have some drawbacks. On the one hand, most KGR methods focus only on one phase of the KG lifecycle, such as KG completion or refinement, while ignoring reasoning over other stages, such as KG extraction. On the other hand, traditional KGR methods, broadly categorized as symbolic and neural, are unable to balance both scalability and interpretability. To resolve these two problems, we take a more comprehensive perspective of KGR with regard to the whole KG lifecycle, including KG extraction, completion, and refinement, which correspond to three subtasks: knowledge extraction, relational reasoning, and inconsistency checking. In addition, we propose the implementation of KGR using a novel neural symbolic framework, with regard to both scalability and interpretability. Experimental results demonstrate that our proposed methods outperform traditional neural symbolic models.

Multi-granularity sequential neural network for document-level biomedical relation extraction

Document-level biomedical relation extraction aims to extract the relation between multiple mentions of entities throughout an entire document. However, most methods suffer from long-distance context dependency and complex semantics causing by numerous biomedical entities and inter-sentence relations. In this paper, we propose a multi-granularity sequential network (MGSN) for document-level relation extraction to solve above problems. The proposed method learns to extract the document-level entity relation by the accumulation of document-level information and entity-level information including global and local entity information. In addition, some target entity pairs that reflect target entity relations can be extracted and paid more attention by CNN-based bi-affine structure. Experimental results on three document-level biomedical datasets demonstrate the effectiveness of the proposed model. Our code is available from http://github.com/SCUT-CCNL/MGSN.

Learning graph attention-aware knowledge graph embedding

The knowledge graph, which utilizes graph structure to represent multi-relational data, has been widely used in the reasoning and prediction tasks, attracting considerable research efforts recently. However, most existing works still concentrate on learning knowledge graph embeddings straightforwardly and intuitively without subtly considering the context of knowledge. Specifically, recent models deal with each single triple independently or consider contexts indiscriminately, which is one-sided as each knowledge unit (i.e., triple) can be derived from its partial surrounding triples. In this paper, we propose a graph-attention-based model to encode entities, which formulates a knowledge graph as an irregular graph and explores a number of concrete and interpretable knowledge compositions by integrating the graph-structured information via multiple independent channels. To measure the correlation between entities from different angles (i.e., entity pair, relation, and structure), we respectively develop three attention metrics. By making use of our enhanced entity embeddings, we further introduce several improved factorization functions for updating relation embeddings and evaluating candidate triples. We conduct extensive experiments on downstream tasks including entity classification, entity typing, and link prediction to validate our methods. Empirical results validate the importance of our introduced attention metrics and demonstrate that our proposed method can improve the performance of factorization models on large-scale knowledge graphs.

Analysis of Big Data Embedded Mapping Model Oriented to Prediction of Subordinate Relationship

With the rapid development of information technology, research on big data information has gone deep into all walks of life, and massive amounts of data spew out anytime and anywhere. This paper proposes a domain-subordinate relationship acquisition that combines multiple strategies and word representations. From semi-structured text and unstructured text, the candidate entity upper-lower relationship entity pairs are extracted, and then the obtained candidate upper-lower relationship entity pairs are verified by the support vector machine method to obtain high-quality candidate entity pairs. In the training phase, an exploratory method is designed to search for the optimal solution, and a greedy mechanism is introduced to evaluate the effectiveness of the reinforcement learning agent, so that the virtual node mapping scheme can continuously move towards higher system returns. Evolution has finally achieved the optimal virtual network mapping decision, so that large-scale tasks can be deployed to task processing nodes in the appropriate underlying network to achieve efficient task execution in a big data environment. The experimental results show that the Tard method can effectively avoid model overfitting and improve the task recognition processing ability in the actual application process under the premise of meeting large-scale task requests.