Entity Pairs Research Articles

SimE4KG: Distributed and Explainable Multi-Modal Semantic Similarity Estimation for Knowledge Graphs

In recent years, exciting sources of data have been modeled as knowledge graphs (KGs). This modeling represents both structural relationships and the entity-specific multi-modal data in KGs. In various data analytics pipelines and machine learning (ML), the task of semantic similarity estimation plays a significant role. Assigning similarity values to entity pairs is needed in recommendation systems, clustering, classification, entity matching/disambiguation and many others. Efficient and scalable frameworks are needed to handle the quadratic complexity of all-pair semantic similarity on Big Data KGs. Moreover, heterogeneous KGs demand multi-modal semantic similarity estimation to cover the versatile contents like categorical relations between classes or their attribute literals like strings, timestamps or numeric data. In this paper, we propose the SimE4KG framework as a resource providing generic open-source modules that perform semantic similarity estimation in multi-modal KGs. To justify the computational costs of similarity estimation, the SimE4KG generates reproducible, reusable and explainable results. The pipeline results are a native semantic RDF KG, including the experiment results, hyper-parameter setup and explanation of the results, like the most influential features. For fast and scalable execution in memory, we implemented the distributed approach using Apache Spark. The entire development of this framework is integrated into the holistic distributed Semantic ANalytics StAck (SANSA).

TransAM: Transformer appending matcher for few-shot knowledge graph completion

Few-shot knowledge graph completion (FSKGC) refers to predicting new facts for a new relation with only few-shot observed entity pairs (triples) as support set. Existing solutions to FSKGC mainly conduct the matching process over entity pair representations. Although effective, a major concern of these models is that the entity interactions are not fully explored, based on the observation that they usually generate the pair representation before the matching stage. Such a design inherently overlooks the fine-grained information from entity interactions, leading to performance decrements in one or three shot, which require matching models to capture more sufficient semantic meanings for prediction. To remedy this issue, in this paper, we explore the entity interactions within and between different instances, i.e., the co-occurrence of two entities, for FSKGC and propose our model named TransAM, Transformer Appending Matcher. TransAM solves the FSKGC problem by computing the probability of entity sequence with a well-designed transformer matching network. Specifically, TransAM appends query entity pair to serialized reference entity sequence and utilizes transformer to calculate the probability by capturing intra- and inter- triple entity interactions. To bridge the gap between transformer and the triple structure, TransAM introduces rotary operation to preserve the head and tail roles of entity within the triple and distinguishes different triples by a separated triple position encoding. Empirical studies on two public benchmark datasets NELL-One and Wiki-One show that TransAM outperforms existing metric-learning solutions in MRR and Hits@1 with both one- and three- shot settings, and achieves comparable results on five-shot setting. Datasets and code will be public available at https://github.com/gawainx/TransAM.

BTDM: A Bi-Directional Translating Decoding Model-Based Relational Triple Extraction

The goal of relational triple extraction is to extract knowledge-rich relational triples from unstructured text. Although the previous methods obtain considerable performance, there are still some problems, such as error propagation, the overlapping triple problem, and suboptimal subject–object alignment. To address the shortcomings above, in this paper, we decompose this task into three subtasks from a fresh perspective: entity extraction, subject–object alignment and relation judgement, as well as propose a novel bi-directional translating decoding model (BTDM). Specifically, a bidirectional translating decoding structure is designed to perform entity extraction and subject–object alignment, which decodes entity pairs from both forward and backward extraction. The bidirectional structure effectively mitigates the error propagation problem and aligns the subject–object pairs. The translating decoding approach handles the overlapping triple problem. Finally, a (entity pair, relation) bipartite graph is designed to achieve practical relationship judgement. Experiments show that our model outperforms previous methods and achieves state-of-the-art performance on NYT and WebNLG. We achieved F1-scores of 92.7% and 93.8% on the two datasets. Meanwhile, in various complementary experiments on complex scenarios, our model demonstrates consistent performance gain in various complex scenarios.

A joint training network for learning more distinguishable relation features in relation classification

Relation classification is an important task in natural language processing, which aims to predict the semantic relation between a given entity pair in a sentence. There are datasets, like TACRED, that contain a large number of “no_relation” type samples. Most existing methods treat “no_relation” and normal relation types equally, and directly apply the softmax function over all relation types. In this paper, we propose a novel joint training network to learn more distinguishable relation features for relation classification. Specially, we convert the original multi-class classification problem into two joint optimized modules, binary classification of whether a relation is “no_relation” and multi-class classification of normal relation types. To further differentiate between similar normal relation types, we introduce a self-supervised contrastive learning method to learn more distinguishable features for them. We jointly optimize the above modules. Experimental results agree well with our design intention and demonstrate that our joint training network not only achieves superior performance against existing competitive models, but also is robust to “no_relation” problem.

Semantic piecewise convolutional neural network with adaptive negative training for distantly supervised relation extraction

Distantly Supervised Relation Extraction (DSRE) aligns existing knowledge bases with unstructured text to extract relation facts, and its automatically generated training data is inevitably noisy. Most existing works identify and reduce the impact of noise by enhancing semantic features. However, they only consider the semantic information in a single instance and ignore the semantic information between different instances. In this work, we propose a Semantic Piecewise Convolutional Neural Network (SPCNN), which uses the similarity between different entity pairs as semantic information to improve relation extraction. Specifically, to learn better semantic vector representations, we combine position features with entity pair features and entity similarity features in a high-dimensional space respectively, and generating two different semantic-aware representations. Then we unify these two representations to form a high-quality bag representation for training. Moreover, we design an Adaptive Negative Training (ANT) strategy, which facilitates the network to further exploit the rich semantic features to reduce the interference of noisy labels. Extensive experimental results on a large-scale benchmark dataset show that our method significantly outperforms other baselines.

Adaptive Prototypical Networks With Label Words and Joint Representation Learning for Few-Shot Relation Classification.

Relation classification (RC) task is one of fundamental tasks of information extraction, aiming to detect the relation information between entity pairs in unstructured natural language text and generate structured data in the form of entity-relation triple. Although distant supervision methods can effectively alleviate the problem of lack of training data in supervised learning, they also introduce noise into the data and still cannot fundamentally solve the long-tail distribution problem of the training instances. In order to enable the neural network to learn new knowledge through few instances such as humans, this work focuses on few-shot relation classification (FSRC), where a classifier should generalize to new classes that have not been seen in the training set, given only a number of samples for each class. To make full use of the existing information and get a better feature representation for each instance, we propose to encode each class prototype in an adaptive way from two aspects. First, based on the prototypical networks, we propose an adaptive mixture mechanism to add label words to the representation of the class prototype, which, to the best of our knowledge, is the first attempt to integrate the label information into features of the support samples of each class so as to get more interactive class prototypes. Second, to more reasonably measure the distances between samples of each category, we introduce a loss function for joint representation learning (JRL) to encode each support instance in an adaptive manner. Extensive experiments have been conducted on FewRel under different few-shot (FS) settings, and the results show that the proposed adaptive prototypical networks with label words and JRL has not only achieved significant improvements in accuracy but also increased the generalization ability of FSRC.

Document-level relation extraction with two-stage dynamic graph attention networks

Document-level Relation Extraction (RE) aims to infer complex semantic relations between entities in a document. Previous approaches leverage a multi-classification model to predict relation types between each entity pair. However, in contrast to sentence-level RE, document-level RE contains various entities expressed by mentions appearing across multiple sentences in a document. Therefore, the amount of negative instances (‘no relationship’) significantly outnumbers that of other positive instances in document-level RE. In addition, most existing methods construct static graphs with heuristic rules to capture the interactions among entities. However, these heuristic rules ignore the specificities of the documents. In this study, we propose a novel two-stage framework to extract document-level relations based on dynamic graph attention networks, namely TDGAT. In the first stage, we capture the relational links of the entity pairs using a binary classification model. In the second stage, we extract fine-grained relations among entities, including the type of ‘NA (no relationship)’. To reduce error propagation, we regard the entity pair links predicted in the first stage as the prior information and leverage them to reconstruct the document-level graphs of the second stage. In this manner, we can provide extra head and tail entity connection information for predicting relations in a document. Furthermore, we propose a dynamic graph strategy to explore the multi-hop interactions between related information. The experimental results show that our framework outperforms most existing models on the public document-level dataset DocRED. The extensive analysis demonstrates the effectiveness of our TDGAT in extracting inter-sentence relations.

Active Temporal Knowledge Graph Alignment

Entity alignment aims to identify equivalent entity pairs from different knowledge graphs (KGs). Recently, aligning temporal knowledge graphs (TKGs) that contain time information has aroused increasingly more interest, as the time dimension is widely used in real-life applications. The matching between TKGs requires seed entity pairs, which are lacking in practice. Hence, it is of great significance to study TKG alignment under scarce supervision. In this work, the authors formally formulate the problem of TKG alignment with limited labeled data and propose to solve it under the active learning framework. As the core of active learning is to devise query strategies to select the most informative instances to label, the authors propose to make full use of time information and put forward novel time-aware strategies to meet the requirement of weakly supervised temporal entity alignment. Extensive experimental results on multiple real-world datasets show that it is important to study TKG alignment with scarce supervision, and the proposed time-aware strategy is effective.

Relational distance and document-level contrastive pre-training based relation extraction model

Document-level relation extraction has multi-entity and multi-mention compared to sentence-level, existing sentence-level relation extraction models cannot meet the requirements of document-level relation extraction. Existing graph-based document-level models usually design points and edges manually, which often introduces man-made noise; while the Transformer-based models cannot deeply solve the difficulties such as coreference resolution by designing pre-training tasks or other methods. In this paper, we propose a new Relational Distance and Document-level Contrastive Pre-training (RDDCP) based relation extraction model, which achieves coreference resolution by simple and effective mention replacement; we also introduce the concept of relational distance to achieve document-level contrastive pre-training, and find the most likely relational mention pairs from the plural mention pairs existing in the document-level dataset for contrastive learning; for the relation information in distant mentions ignored by the relational distance, we quantified the distances as weights and incorporated the information with weights into the embedding representation of entities. Each entity presents different entity embedding representations in different entity pairs. We conducted experiments on three popular datasets and the RDDCP model outperformed GAIN, SSAN and ATLOP as well as other baseline models in terms of performance and time complexity.

Generative adversarial network for unsupervised multi-lingual knowledge graph entity alignment

Entity alignment is an essential process in knowledge graph (KG) fusion, which aims to link entities representing the same real-world object in different KGs, to achieve entity expansion and graph fusion. Recently, embedding-based entity pair similarity evaluation has become mainstream in entity alignment research. However, these methods heavily rely on labelled entity pairs, which are often unavailable. Some self-supervised methods exploit features of KGs regardless of noise when generating aligned entity pairs. To resolve this issue, we propose a generative adversarial entity alignment method, which is more robust to noise data. The proposed method then exploits both attribute and structure information in the KGs and applies a BERT-based contrastive loss function to embed entities in KGs. Experimental results on several benchmark datasets demonstrate the superiority of our framework compared with most existing state-of-the-art entity alignment methods.

Few-shot link prediction for temporal knowledge graphs based on time-aware translation and attention mechanism

Few-shot knowledge graph completion (KGC) is an important and common task in real applications, which aims to predict unseen facts when only few samples are available for each relation in the knowledge graph (KG). Previous methods on few-shot KGC mainly focus on static KG, however, many KG in real-world applications are dynamic and develop over time. In this work, we consider few-shot KGC in temporal knowledge graphs (TKGs), where the fact may only hold for a specific timestamp. We propose a Few-Shot Completion model in TKG (TFSC), which compare the input query to the given few-shot references to make predictions. Specifically, in order to enhance the representation of entities in the case of few samples, we use the attention mechanism to model the neighbor entities of the task entity with timestamp information, and generate expressive time-aware entity pair representations through the Transformer encoder. A comprehensive set of experiments is finally carried out to demonstrate the effectiveness a of our proposed model TFSC.

Dialogue Relation Extraction with Document-Level Heterogeneous Graph Attention Networks

We propose a heterogeneous graph attention network to address the problem of dialogue relation extraction. Compared with several popular sequence-based and graph-based models, our method shows superior performance on the benchmark dataset DialogRE. The implementation of this work can be found at https://github.com/declare-lab/dialog-HGAT Dialogue relation extraction aims to detect the relation between pairs of entities mentioned in a multi-party dialogue. It plays an essential role in understanding the deep logic of dialogues and facilitating the development of intelligent dialogue systems. We introduce a heterogeneous graph attention network to model the cross-sentence relations in a conversation. This heterogeneous graph attention network has modeled multi-type features of the conversation, such as utterance, word, speaker, argument, and entity type information. We compare our method with several popular baselines such as convolutional neural networks and long short-term memory, experimental results show our model outperforms the state-of-the-art method by 9.4%/7.8% F1 scores, and 6.6%/3.9% $$F1_c$$ scores in both validation and test sets with only 4.0M parameters. In this work, we present an attention-based heterogeneous graph network to deal with the dialogue relation extraction task in an inductive manner. Experimental results on the dataset DialogRE confirm the effectiveness of our method.

Embedding knowledge graph of patent metadata to measure knowledge proximity

AbstractKnowledge proximity refers to the strength of association between any two entities in a structural form that embodies certain aspects of a knowledge base. In this work, we operationalize knowledge proximity within the context of the US Patent Database (knowledge base) using a knowledge graph (structural form) named “PatNet” built using patent metadata, including citations, inventors, assignees, and domain classifications. We train various graph embedding models using PatNet to obtain the embeddings of entities and relations. The cosine similarity between the corresponding (or transformed) embeddings of entities denotes the knowledge proximity between these. We compare the embedding models in terms of their performances in predicting target entities and explaining domain expansion profiles of inventors and assignees. We then apply the embeddings of the best‐preferred model to associate homogeneous (e.g., patent–patent) and heterogeneous (e.g., inventor–assignee) pairs of entities.

Similarity Feature Construction for Matching Ontologies through Adaptively Aggregating Artificial Neural Networks

Ontology is the kernel technique of Semantic Web (SW), which enables the interaction and cooperation among different intelligent applications. However, with the rapid development of ontologies, their heterogeneity issue becomes more and more serious, which hampers communications among those intelligent systems built upon them. Finding the heterogeneous entities between two ontologies, i.e., ontology matching, is an effective method of solving ontology heterogeneity problems. When matching two ontologies, it is critical to construct the entity pair’s similarity feature by comprehensively taking into consideration various similarity features, so that the identical entities can be distinguished. Due to the ability of learning complex calculating model, recently, Artificial Neural Network (ANN) is a popular method of constructing similarity features for matching ontologies. The existing ANNs construct the similarity feature in a single perspective, which could not ensure its effectiveness under diverse heterogeneous contexts. To construct an accurate similarity feature for each entity pair, in this work, we propose an adaptive aggregating method of combining different ANNs. In particular, we first propose a context-based ANN and syntax-based ANN to respectively construct two similarity feature matrices, which are then adaptively integrated to obtain a final similarity feature matrix through the Ordered Weighted Averaging (OWA) and Analytic hierarchy process (AHP). Ontology Alignment Evaluation Initiative (OAEI)’s benchmark and anatomy track are used to verify the effectiveness of our method. The experimental results show that our approach’s results are better than single ANN-based ontology matching techniques and state-of-the-art ontology matching techniques.

Contextual Path Retrieval: A Contextual Entity Relation Embedding-based Approach

Contextual path retrieval (CPR) refers to the task of finding contextual path(s) between a pair of entities in a knowledge graph that explains the connection between them in a given context. For this novel retrieval task, we propose the Embedding-based Contextual Path Retrieval (ECPR) framework. ECPR is based on a three-component structure that includes a context encoder and path encoder that encode query context and path, respectively, and a path ranker that assigns a ranking score to each candidate path to determine the one that should be the contextual path. For context encoding, we propose two novel context encoding methods, i.e., context-fused entity embeddings and contextualized embeddings. For path encoding, we propose PathVAE, an inductive embedding approach to generate path representations. Finally, we explore two path-ranking approaches. In our evaluation, we construct a synthetic dataset from Wikipedia and two real datasets of Wikinews articles constructed through crowdsourcing. Our experiments show that methods based on ECPR framework outperform baseline methods, and that our two proposed context encoders yield significantly better performance than baselines. We also analyze a few case studies to show the distinct features of ECPR-based methods.

Critical Relation Path Aggregation-Based Industrial Control Component Exploitable Vulnerability Reasoning

With the growing discovery of exposed vulnerabilities in the Industrial Control Components (ICCs), identification of the exploitable ones is urgent for Industrial Control System (ICS) administrators to proactively forecast potential threats. However, it is not a trivial task due to the complexity of the multi-source heterogeneous data and the lack of automatic analysis methods. To address these challenges, we propose an exploitability reasoning method based on the ICC-Vulnerability Knowledge Graph (KG) in which relation paths contain abundant potential evidence to support the reasoning. The reasoning task in this work refers to determining whether a specific relation is valid between an attacker entity and a possible exploitable vulnerability entity with the help of a collective of the critical paths. The proposed method consists of three primary building blocks: KG construction, relation path representation, and query relation reasoning. A security-oriented ontology combines exploit modeling, which provides a guideline for the integration of the scattered knowledge while constructing the KG. We emphasize the role of the aggregation of the attention mechanism in representation learning and ultimate reasoning. In order to acquire a high-quality representation, the entity and relation embeddings take advantage of their local structure and related semantics. Some critical paths are assigned corresponding attentive weights and then they are aggregated for the determination of the query relation validity. In particular, similarity calculation is introduced into a critical path selection algorithm, which improves search and reasoning performance. Meanwhile, the proposed algorithm avoids redundant paths between the given pairs of entities. Experimental results show that the proposed method outperforms the state-of-the-art ones in the aspects of embedding quality and query relation reasoning accuracy.

A numerical comparative study of completion methods for pairwise comparison matrices

In the context of some multi-criteria decision-making methods, such as the Analytic Hierarchy Process, an expert is required to compare entities, e.g. alternatives and criteria. However, often, for various reasons, the expert cannot provide judgments on all pairs of entities. For these cases, several completion methods have been proposed in the literature to estimate the missing values of pairwise comparison matrices. In this paper, we study the similarity of eleven completion methods on the basis of numerical simulations and hierarchical clustering. We perform simulations for matrices of different orders considering various numbers of missing comparisons. Finally, the results suggest the existence of a cluster of five extremely similar methods, and a method significantly dissimilar from all the others.

Models and Techniques for Domain Relation Extraction: A Survey

As the significant subtask of information extraction, relation extraction (RE) aims to identify and classify semantic relations between pairs of entities and is widely adopted as the foundation of downstream applications including knowledge graphs, intelligent question answering, text mining, and sentiment analysis. Different from general knowledge, domain knowledge is pertinent to specific fields which include a wealth of proprietary entities and relations. Besides, most of the data are formed as documents rather than sentences. In this paper, the task of domain RE is defined, and the common domains are presented. Furthermore, we provide a systematic review of state-of-the-art techniques as well as the latest trends. We survey different neural network-based techniques for RE and describe the overall framework, training procedures, as well as the pros and cons of these techniques. Then, we introduce and compare the corpus and metrics used for domain RE tasks. Finally, we conclude and discuss future research issues of domain RE.

Pairwise Metric Learning With Angular Margin for Tag Recommendation

The users on the Internet have been growing exponentially, and tag recommendation can automatically provide users with a selection of tags of interest to meet their personalized needs. Users can utilize these tags to freely annotate their favorite resources, making them efficient and fast in retrieving related resources. Tensor factorization methods are commonly used in tag recommendation at present. These methods model the user × item × tag interactions, transform the latent feature representations of users, items and tags into low-rank matrices and use inner products for prediction. However, the problem of using inner product is that it does not satisfy the triangle inequality, it ignores the distance relationship among entity pairs and cannot capture the fine-grained preference information. Metric learning in recommendation domains focus on using pairwise loss, which assumes that different categories (such as users, items and tags) have fixed margins. Different categories often have different intra-class variations. With fixed margins, it is often difficult to accurately distinguish between positive and negative samples, thus reducing recommendation performance and limiting the expression ability of the model. In this study, the metric learning method is used to explore the distance relationship among user × item × tag triplet, and the existing metric learning based methods (namely LRML, CML, SML) are applied to the tag recommendation. A pairwise metric learning method with angular margin is proposed, named PMLT. The pairwise distance relationship between user-tag and item-tag is modeled for the information of different entities. And an extra angular margin regularizer is added to the original pairwise loss to control the size of angular margin for user-tag and item-tag respectively. The strength of the constrained angular margin regularizer is controlled to dynamically adjust the distance changes of entity. This method constrains the fixed margin and also the angular margin of user-tag and item-tag. Compared with the traditional metric learning method, this method can capture additional relationship structure and has good recommendation performance. Finally, we conducted extensive experiments on two datasets, LastFm and Movielens, and the experimental results showed that the proposed method PMLT outperform the state-of-the-art baseline in the evaluation metrics Recall@N and NDCG@N, and obtain better prediction quality. We also analyze the influence of different parameters and internal components on the performance of the proposed method, which improves the interpretability of the proposed method.

Integrating Symbol Similarities with Knowledge Graph Embedding for Entity Alignment: An Unsupervised Framework

Entity alignment refers to discovering identical entity pairs in 2 knowledge graphs, which is a significant task in knowledge fusion. Early automated entity alignment techniques are based mainly on similarity calculation and comparing symbolic features, i.e., entity names, between entities. Nevertheless, such methods’ performance would reduce significantly when the difference between knowledge graphs is enormous because of relying on predefined comparison rules. Recently, embedding-based methods calculate the similarity between entity pairs through vector embeddings and thus can deal with different knowledge graphs. However, embedding-based methods mostly require humans to annotate data, which is laborious. Therefore, we learn from each other to propose an unsupervised entity alignment framework in this work, which can generate initial alignment seeds automatically by considering symbolic similarities. It can effectively avoid the waste of human resources and is suitable for handling multiple types of knowledge graphs. In addition, we investigate improving the quality and quantity of initial alignment by integrating multiple symbolic similarity features of entities and dealing with the situation of entity information missing better. Experimental results on 3 real datasets demonstrate its state-of-the-art performance.