Multi-relational knowledge graph contrastive learning for link prediction

A knowledge graph (KG) is used to store knowledge in the form of connected facts. Facts in KG are represented in the form of a triple (subject, predicate, object) or (head, relation, tail). KG is widely used in question answering, information retrieval, classification, recommender systems, and so on. However, a common problem with KG is incomplete KG. A KG is called incomplete if there is a missing relationship between two entities. An incomplete KG can have an impact on decreasing the accuracy of a task that uses the KG. One solution to the incomplete KG is to use link prediction. Link prediction aims to predict the missing relationship between two entities in a KG. Another problem is that the size of KG is large, consisting of hundreds or millions of entities and relationships. Handling large KG also needs to be considered. Therefore, link prediction on large KG also needs to be considered so that the link prediction process is more efficient. This paper discusses link prediction using embedding to overcome the incomplete KG problem. In addition, it is proposed to use clustering to increase the efficiency of the link prediction process. Clustering is used to group the embedding results. After the embedding results are grouped, scoring and loss function calculations to predict missing links are carried out in groups that are considered appropriate. It is expected that with this grouping, the time of link prediction process can be more efficient because there is no need to check all the vectors in the embedding space.

Knowledge graphs (KG) include large amounts of structured data in many different domains. Knowledge or information is captured by entities and relationships between them in KG. One of the open problems in knowledge graphs area is prediction, that is predicting new relationships or links between the given existing entities in KG. A recent approach in graph-based learning problems is embedding, in which graphs are represented as low-dimensional vectors. Then, it is easier to make link predictions using these vector representations. We also use graph embedding for graph representations. A sub-problem of link prediction in KG is the link prediction in the presence of literal values, and specifically numeric values, on the receiving end of links. This is a harder problem because of the numeric literal values taking arbitrary values. For such entries link prediction models cannot work, because numeric entities are not embedded in the vector space. There are several studies in this area, but they are all complex approaches. In this study, we propose a novel approach for link prediction in KG in the presence of numerical values. To overcome the embedding problem of numeric values, we used a clustering approach for clustering these numerical values in a knowledge graph and then used the clusters for performing link prediction. Then we clustered the numerical values to enhance the prediction rates and evaluated our method on a part of Freebase knowledge graph, which includes entities, relations, and numerical literals. Test results show that a considerable increase in link prediction rate can be achieved in comparison to previous studies.

Community knowledge graph abstraction for enhanced link prediction: A study on PubMed knowledge graph

Improving embedding-based link prediction performance using clustering

Knowledge Graphs (KGs) are relational knowledge bases that represent facts as a set of labelled nodes and the labelled relations between them. Their machine learning counterpart, Knowledge Graph Embeddings (KGEs), learn to predict new facts based on the data contained in a KG – the so-called link prediction task. To date, almost all forms of link prediction for KGs rely on some form of embedding model, and KGEs hold state-of-the-art status for link prediction. In this paper, we present TWIG-I (Topologically-Weighted Intelligence Generation for Inference), a novel link prediction system that can represent the features of a KG in latent space without using node or edge embeddings. TWIG-I shows mixed performance relative to state-of-the-art KGE models – at times exceeding or falling short of baseline performance. However, unlike KGEs, TWIG-I can be natively used for transfer learning across distinct KGs. We show that using transfer learning with TWIG-I can lead to increases in performance in some cases both over KGE baselines and over TWIG-I models trained without finetuning. While these results are still mixed, TWIG-I clearly demonstrates that structural features are sufficient to solve the link prediction task in the absence of embeddings. Finally, TWIG-I opens up cross-KG transfer learning as a new direction in link prediction research and application.

Complex networks can be modeled as knowledge graphs (KGs) with nodes and edges denoting entities and relations among those entities, respectively. A knowledge graph embedding assigns to each node and edge in a KG a low-dimensional semantic vector such that the original structure and relations in the KG are approximately preserved in these learned semantic vectors. KG embeddings support downstream applications such as KG completion, classification, entity resolution, link prediction, question answering, and recommendation. In the real world, KGs are dynamic and evolve over time. State-of-the-art KG embedding models deal with static KGs. To support dynamic updates (even local), they must be retrained on the whole KG from scratch, which is inefficient. To this end, we propose a new context-aware Online Updates of Knowledge Graph Embedding (OUKE) method, which supports embedding updates in an online manner. OUKE learns two different vectors for each node and edge, i.e., knowledge embedding and context embedding. This strategy effectively limits the impacts of a local update in a smaller region, so that OUKE is able to efficiently update the KG embedding. Experiments on the link prediction in dynamic KGs demonstrate both effectiveness and efficiency of our solution.KeywordsKnowledge graphsEmbeddingDynamic updates

Knowledge Graphs (KGs) comprise of interlinked information in the form of entities and relations between them in a particular domain and provide the backbone for many applications. However, the KGs are often incomplete as the links between the entities are missing. Link Prediction is the task of predicting these missing links in a KG based on the existing links. Recent years have witnessed many studies on link prediction using KG embeddings which is one of the mainstream tasks in KG completion. To do so, most of the existing methods learn the latent representation of the entities and relations whereas only a few of them consider contextual information as well as the textual descriptions of the entities. This paper introduces an attentive encoder-decoder based link prediction approach considering both structural information of the KG and the textual entity descriptions. Random walk based path selection method is used to encapsulate the contextual information of an entity in a KG. The model explores a bidirectional Gated Recurrent Unit (GRU) based encoder-decoder to learn the representation of the paths whereas SBERT is used to generate the representation of the entity descriptions. The proposed approach outperforms most of the state-of-the-art models and achieves comparable results with the rest when evaluated with FB15K, FB15K-237, WN18, WN18RR, and YAGO3-10 datasets.

Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors. These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities. For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding. Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning.

KGEL: A novel end-to-end embedding learning framework for knowledge graph completion

Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors. These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities. For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding. Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning.

Link prediction over a knowledge graph aims to predict the missing head entities <inline-formula><tex-math notation="LaTeX">$h$</tex-math></inline-formula> or tail entities <inline-formula><tex-math notation="LaTeX">$t$</tex-math></inline-formula> and missing relations <inline-formula><tex-math notation="LaTeX">$r$</tex-math></inline-formula> for a triple <inline-formula><tex-math notation="LaTeX">$(h,r,t)$</tex-math></inline-formula> . Recent years have witnessed great advance of knowledge graph embedding based link prediction methods, which represent entities and relations as elements of a continuous vector space. Most methods learn the embedding vectors by optimizing a margin-based loss function, where the margin is used to separate negative and positive triples in the loss function. The loss function utilizes the general structures of knowledge graphs, e.g., the vector of <inline-formula><tex-math notation="LaTeX">$r$</tex-math></inline-formula> is the translation of the vector of <inline-formula><tex-math notation="LaTeX">$h$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$t$</tex-math></inline-formula> , and the vector of <inline-formula><tex-math notation="LaTeX">$t$</tex-math></inline-formula> should be the nearest neighbor of the vector of <inline-formula><tex-math notation="LaTeX">$h+r$</tex-math></inline-formula> . However, there are many particular structures, and can be employed to promote the performance of link prediction. One typical structure in knowledge graphs is hierarchical structure, which existing methods have much unexplored. We argue that the hierarchical structures also contain rich inference patterns, and can further enhance the link prediction performance. In this paper, we propose a hierarchy-constrained link prediction method, called hTransM, on the basis of the translation-based knowledge graph embedding methods. It can adaptively determine the optimal margin by detecting the single-step and multi-step hierarchical structures. Moreover, we prove the effectiveness of hTransM theoretically, and experiments over three benchmark datasets and two sub-tasks of link prediction demonstrate the superiority of hTransM.

A probabilistic ensemble approach for knowledge graph embedding

DKGR: A Distributed Framework for Geometric Knowledge Graph Embedding with Ray

An open-domain knowledge graph (KG) has entities as nodes and natural language relations as edges, and is constructed by extracting (subject, relation, object) triples from text. The task of open-domain link prediction is to infer missing relations in the KG. Previous work has used standard link prediction for the task. Since triples are extracted from text, we can ground them in the larger textual context in which they were originally found. However, standard link prediction methods only rely on the KG structure and ignore the textual context that each triple was extracted from. In this paper, we introduce the new task of open-domain contextual link prediction which has access to both the textual context and the KG structure to perform link prediction. We build a dataset for the task and propose a model for it. Our experiments show that context is crucial in predicting missing relations. We also demonstrate the utility of contextual link prediction in discovering context-independent entailments between relations, in the form of entailment graphs (EG), in which the nodes are the relations. The reverse holds too: context-independent EGs assist in predicting relations in context.

An open-domain knowledge graph (KG) has entities as nodes and natural language relations as edges, and is constructed by extracting (subject, relation, object) triples from text. The task of open-domain link prediction is to infer missing relations in the KG. Previous work has used standard link prediction for the task. Since triples are extracted from text, we can ground them in the larger textual context in which they were originally found. However, standard link prediction methods only rely on the KG structure and ignore the textual context that each triple was extracted from. In this paper, we introduce the new task of open-domain contextual link prediction which has access to both the textual context and the KG structure to perform link prediction. We build a dataset for the task and propose a model for it. Our experiments show that context is crucial in predicting missing relations. We also demonstrate the utility of contextual link prediction in discovering context-independent entailments between relations, in the form of entailment graphs (EG), in which the nodes are the relations. The reverse holds too: context-independent EGs assist in predicting relations in context.