Measuring the Inferential Values of Relations in Knowledge Graphs

Knowledge graph representation learning aims to obtain its vector representation by mapping entities and relations in knowledge graphs to a continuous low-dimensional vector space by learning methods. Most of the existing knowledge graph representation learning methods only consider the single-step relation between entities from the perspective of triples and fail to effectively utilize important information such as ordered multi-step relation paths and entity descriptions, thus affecting the ability of knowledge representation learning. We propose a knowledge graph representation learning model that integrates ordered relation paths and entity descriptions in response to the above problems. The model can integrate the triple representation in the knowledge graph, the semantic representation of entity description, and the representation of ordered relation paths for training. On the FB15K, WN18, FB15K-237, and WN18RR datasets, the proposed model and baselines are run on the link prediction task. Experimental results show that the model has higher accuracy than existing baselines, demonstrating the effectiveness and superiority of the method.

Las técnicas avanzadas de extracción de información y la creciente disponibilidad de datos vinculados han dado a luz a la noción de Grafo de Conocimiento (Knowledge Graph, KG) de gran escala. Con la creciente popularidad de KGs que contienen millones de conceptos y entidades, la investigación de herramientas fundamentales que estudian características semánticas de KGs es crítica para el desarrollo de aplicaciones basadas en KG, aparte del estudio de las técnicas de población de KG. Con este enfoque, esta tesis explora la similitud semántica en KGs teniendo en cuenta el concepto de taxonomía, concepto de distribución, la entidad descripciones y las categorías. La similitud semántica captura la cercanía de significados. A través del estudio de la red semántica de conceptos y entidades con relaciones significativas en KGs, hemos propuesto una nueva métrica de semántica WPath semántica, y un nuevo método de computación basado en información gráfica (IC). Con el WPath y el IC basado en gráfos, la similitud semántica de los conceptos se puede calcular directamente, basándose únicamente en el conocimiento estructural y el conocimiento estadístico contenido en KGs. Los experimentos en similitud de palabras han demostrado que la mejora de los métodos propuestos es estadísticamente significativa en comparación con los métodos convencionales. Por otra parte, observando que los conceptos suelen ser colocados con descripciones textuales, proponemos un nuevo enfoque de incorporación para formar el concepto y incorporación de palabras conjuntamente. El espacio vectorial compartido de conceptos y palabras ha proporcionado una computación de la similitud conveniente entre conceptos y palabras a través de similitud vectorial. De manera adicional, se ilustran algunas aplicaciones de modelos basados en el conocimiento, en corpus y en embeddings en la tarea de desambiguación y clasificación semántica, con el fin de demostrar la capacidad e idoneidad de diferentes métodos de similitud en aplicaciones específicas. Por último, la búsqueda de entidad semántica se utiliza como una demostración ilustrativa de un nivel más alto de la aplicación que consiste en similitud basado en el texto de concordancia, la desambiguación y la expansión de la consulta. Para implementar la demostración completa de la consulta de información centrada en la entidad, también proponemos un enfoque basado en reglas para construir y ejecutar automáticamente consultas SPARQL. ABSTRACT The advanced information extraction techniques and increasing availability of linked data have given birth to the notion of large scale Knowledge Graph (KG). With the increasing popularity of KGs containing millions of concepts and entities, the research of fundamental tools studying semantic features of KGs is critical for the development of KG-based applications, apart from the study of KG population techniques. With such focus, this thesis exploits semantic similarity in KGs taking into consideration of concept taxonomy, concept distribution, entity descriptions and categories. Semantic similarity captures the closeness of meanings. Through studying the semantic network of concepts and entities with meaningful relations in KGs, we proposed a novel WPath semantic similarity metric and new graph-based Information Content (IC) computation method. With the WPath and graph-based IC, semantic similarity of concepts can be computed directly and only based on the structural and statistical knowledge contained in KG. The word similarity experiments have shown that the improvement of the proposed methods is statistical significant comparing to conventional methods. Moreover, observing that concepts are usually collocated with textual descriptions, we propose a novel embedding approach to train concept and word embedding jointly. The shared vector space of concepts and words, has provided convenient similarity computation between concepts and words through vector similarity. Furthermore, the applications of knowledge-based, corpus-based and embedding-based similarity methods are shown and compared in the task of semantic disambiguation and classification, in order to demonstrate the capability and suitability of different similarity methods in specific application. Finally, semantic entity search is used as an illustrative showcase to demonstrate higher level of the application consisting of text matching, disambiguation and query expansion. To implement the complete demonstration of entity-centric information querying, we also propose a rule-based approach for constructing and executing SPARQL queries automatically. In summary, the thesis exploits various similarity methods and illustrates their corresponding applications for KGs. The proposed similarity methods and presented similaritybased applications would help in facilitating the research and development of applications in KGs.

Knowledge Graphs (KGs) model statements as head-relation-tail triples. Intrinsically, KGs are assumed incomplete especially when knowledge is represented under the Open World Assumption. The problem of KG completeness aims at identifying missing values. While some approaches focus on predicting relations between pairs of known nodes in a graph, other solutions have studied the problem of predicting missing entity properties or relations even in the presence of unknown tails. In this work, we address the latter research problem: for a given head entity in a KG, obtain the set of relations which are missing for the entity. To tackle this problem, we present an approach that mines latent information about head entities and their relations in KGs. Our solution combines in a novel way, state-of-the-art techniques from association rule learning and community detection to discover latent groups of relations in KGs. These latent groups are used for predicting missing relations of head entities in a KG. Our results on ten KGs show that our approach is complementary state-of-the-art solutions.

Knowledge graph (KG) representation learning which aims to encode entities and relations into low-dimensional spaces, has been widely used in KG completion and link prediction. Although existing KG representation learning models have shown promising performance, the theoretical mechanism behind existing models is much less well-understood. It is challenging to accurately portray the internal connections between models and build a competitive model systematically. To overcome this problem, a unified KG representation learning framework, called GrpKG, is proposed in this paper to model the KG representation learning from a generic groupoid perspective. We discover that many existing models are essentially the same in the sense of groupoid isomorphism and further provide transformation methods between different models. Moreover, we explore the applications of GrpKG in the model classification as well as other processes. The experiments on several benchmark data sets validate the effectiveness and superiority of our framework by comparing two proposed models (GrpQ8 and GrpM2) with the state-of-the-art models.

Knowledge graphs are repositories providing factual knowledge about entities. They are a great source of knowledge to support modern AI applications for Web search, question answering, digital assistants, and online shopping. The advantages of machine learning techniques and the Web's growth have led to colossal knowledge graphs with billions of facts about hundreds of millions of entities collected from a large variety of sources. While integrating independent knowledge sources promises rich information, it inherently leads to heterogeneities in representation due to a large variety of different conceptualizations. Thus, real-world knowledge graphs are threatened in their overall utility. Due to their sheer size, they are hardly manually curatable anymore. Automatic and semi-automatic methods are needed to cope with these vast knowledge repositories. We first address the general topic of representation heterogeneity by surveying the problem throughout various data-intensive fields: databases, ontologies, and knowledge graphs. Different techniques for automatically resolving heterogeneity issues are presented and discussed, while several open problems are identified. Next, we focus on entity heterogeneity. We show that automatic matching techniques may run into quality problems when working in a multi-knowledge graph scenario due to incorrect transitive identity links. We present four techniques that can be used to improve the quality of arbitrary entity matching tools significantly. Concerning relation heterogeneity, we show that synonymous relations in knowledge graphs pose several difficulties in querying. Therefore, we resolve these heterogeneities with knowledge graph embeddings and by Horn rule mining. All methods detect synonymous relations in knowledge graphs with high quality. Furthermore, we present a novel technique for avoiding heterogeneity issues at query time using implicit knowledge storage. We show that large neural language models are a valuable source of knowledge that is queried similarly to knowledge graphs already solving several heterogeneity issues internally.

Nowadays, personalized recommendation based on knowledge graphs has become a hot spot for researchers due to its good recommendation effect. In this paper, we researched personalized recommendation based on knowledge graphs. First of all, we study the knowledge graphs’ construction method and complete the construction of the movie knowledge graphs. Furthermore, we use Neo4j graph database to store the movie data and vividly display it. Then, the classical translation model TransE algorithm in knowledge graph representation learning technology is studied in this paper, and we improved the algorithm through a cross-training method by using the information of the neighboring feature structures of the entities in the knowledge graph. Furthermore, the negative sampling process of TransE algorithm is improved. The experimental results show that the improved TransE model can more accurately vectorize entities and relations. Finally, this paper constructs a recommendation model by combining knowledge graphs with ranking learning and neural network. We propose the Bayesian personalized recommendation model based on knowledge graphs (KG-BPR) and the neural network recommendation model based on knowledge graphs (KG-NN). The semantic information of entities and relations in knowledge graphs is embedded into vector space by using improved TransE method, and we compare the results. The item entity vectors containing external knowledge information are integrated into the BPR model and neural network, respectively, which make up for the lack of knowledge information of the item itself. Finally, the experimental analysis is carried out on MovieLens-1M data set. The experimental results show that the two recommendation models proposed in this paper can effectively improve the accuracy, recall, F1 value and MAP value of recommendation.

Knowledge Graph Embedding methods learn low-dimensional representations for entities and relations in knowledge graphs, which can be used to infer previously unknown relations between pairs of entities in the knowledge graph. This is particularly useful for expanding otherwise sparse knowledge graphs. However, the relation types that can be predicted using knowledge graph embeddings are confined to the set of relations that already exists in the KG. Often the set of relations that exist between two entities are not independent, and it is possible to predict what other relations are likely to exist between two entities by composing the embeddings of the relations in which each entity participates. We introduce relation composition as the task of inferring embeddings for unseen relations by combining existing relations in a knowledge graph. Specifically, we propose a supervised method to compose relational embeddings for novel relations using pre-trained relation embeddings for existing relations. Our experimental results on a previously proposed benchmark dataset for relation composition ranking and triple classification show that the proposed supervised relation composition method outperforms several unsupervised relation composition methods.

Knowledge graphs (KGs) have provided a better approach to organize and manage multi-source knowledge in the field of civilian transportation equipment. However, due to the existence of isolated data islands, there are still problems such as unidentified attribute relations in the KGs for civilian transportation equipment. An effective solution to these problems can be achieved through entity alignment. In this paper, we propose two entity alignment strategies for different purposes. Specifically, aiming at the incompleteness of single-source knowledge in the construction of KGs for civilian transportation equipment, we employ an instance alignment strategy to enable multi-source fusion of attribute relationships. In addition, aiming at the inconsistency between different attribute values of the same entity, we propose a conflict resolution strategy based on multi-source data quality assessment, and achieve the unity of attribute values. Compared with previous KGs, our proposed entity alignment method is able to moderately enrich the attribute relations in KGs for civilian transportation equipment.

Building and exploiting spatial–temporal knowledge graph for next POI recommendation

Knowledge graph embedding aims at learning low-dimensional representations for entities and relations in knowledge graph. Previous knowledge graph embedding methods use just one score to measure the plausibility of a fact, which can’t fully utilize the latent semantics of entities and relations. Meanwhile, they ignore the type of relations in knowledge graph and don’t use fact type explicitly. We instead propose a model to fuse different scores of a fact and utilize relation and fact type information to supervise the training process. Specifically, scores by inner product of a fact and scores by neural network are fused with different weights to measure the plausibility of a fact. For each fact, besides modeling the plausibility, the model learns to classify different relations and differentiate positive facts from negative ones which can be seen as a muti-task method. Experiments show that our model achieves better link prediction performance than multiple strong baselines on two benchmark datasets WN18 and FB15k.

Knowledge graphs (KGs) are of great importance in various artificial intelligence systems, such as question answering, relation extraction, and recommendation. Nevertheless, most real-world KGs are highly incomplete, with many missing relations between entities. To discover new triples (i.e., head entity, relation, tail entity), many KG completion algorithms have been proposed in recent years. However, a vast majority of existing studies often require a large number of training triples for each relation, which contradicts the fact that the frequency distribution of relations in KGs often follows a long tail distribution, meaning a majority of relations have only very few triples. Meanwhile, since most existing large-scale KGs are constructed automatically by extracting information from crowd-sourcing data using heuristic algorithms, plenty of errors could be inevitably incorporated due to the lack of human verification, which greatly reduces the performance for KG completion. To tackle the aforementioned issues, in this paper, we study a novel problem of error-aware few-shot KG completion and present a principled KG completion framework REFORM. Specifically, we formulate the problem under the few-shot learning framework, and our goal is to accumulate meta-knowledge across different meta-tasks and generalize the accumulated knowledge to the meta-test task for error-aware few-shot KG completion. To address the associated challenges resulting from insufficient training samples and inevitable errors, we propose three essential modules neighbor encoder, cross-relation aggregation, and error mitigation in each meta-task. Extensive experiments on three widely used KG datasets demonstrate the superiority of the proposed framework REFORM over competitive baseline methods.

The representation learning of knowledge graph refers to embedding entities and relations in knowledge graph into a low-dimensional dense vector space. Existing knowledge graph embedding models mostly chose Euclidean Space as their vector space and consider each fact triple in knowledge graph independently. However, Euclidean Space is unable to represent knowledge effectively due to its strict constraints and mathematical expression. Besides, entities in knowledge graph are not isolated, while these models ignore the association between entities. To solve the problems above, we proposed a text-enhanced knowledge graph representation learning model in Hyperbolic Space. We utilize rich semantic information of entity description by using Transformer Encoder to enhance the ability of knowledge representation. Besides, we embed entities and relations into Hyperbolic Space, which can better capture hierarchical information of the knowledge graph. Experiments on benchmark dataset show that our method achieves better performance compared with other state-of-art methods.KeywordsKnowledge graphRepresentation learningEmbedding modelHyperbolic space

Knowledge graph (KG) embedding aims to encode the entities and relations in KG into low dimensional vector space while preserving the inherent structure of KG. To learn better embeddings, three aspects of KG area, i.e., negative sampling, semantic information in single triplets and structural information in relational paths, are extensively studied in the literature. However, since different KGs have complex and distinct patterns, a single model is usually hard to adapt well to different KGs and different downstream tasks. Recently, automated machine learning (AutoML) has exhibited its power in many machine learning tasks. Inspired by the success of AutoML in both academia and industry, we propose AutoKGE in this thesis to address the three aspects in KG area in different ways. AutoKGE is not only new to the literature, but also opens up new directions in analyzing and designing the KG embedding models. In detail, we propose NSCaching, a simply but very efficient method in sampling high-quality negative triplets. In order to keep track of the dynamic distribution of negative triplets in different KGs, we develop an automated version NSCaching (auto) to adapt the sampling schemes. To capture the different semantic information in each triplet, we propose AutoSF to automatically design SFs for distinct KGs regarding the relation patterns. Novel, better and KG-dependent scoring functions are designed through our algorithm. To explore the structural information, we propose NRASE to distill structural information and combine it with semantic information based on the relational path. Formed as a neural architecture search (NAS) problem, the searched models adaptively combine the structural and semantic information in various KG tasks. Extensive experiments demonstrate the effectiveness of the searched models and efficiency of each search algorithms.

Knowledge graph (KG) is a structured semantic knowledge base, which is widely used in the fields of semantic search, such as intelligent Q&A and intelligent recommendation. Representation learning, as a key issue of KG, aims to vectorize entities and relations in KG to reduce data sparseness and improve computational efficiency. Translation-based representation learning model shows great knowledge representation ability, but there also are limitations in complex relations modeling and representation accuracy. To address these problems, this paper proposes a novel representation learning model with semantic vectors, called TransV, which makes full use of external text corpus and KG’s context to accurately represent entities and complex relations. Entity semantic vectors and relation semantic vectors are constructed, which can not only deeply extend semantic structure of KG, but also transform complex relations into precise simple relations from a semantic perspective. Link prediction and triple classification tasks are performed on TransV with public datasets. Experimental results show that TransV can outperform other translation-based models. Mean Rank is reduced by 66 and Hits@10 is increased by 20% on average for link prediction task on FB15K.

Knowledge graph representation learning aims to embed entities and relationships into low-dimensional space through knowledge graph embedding methods. Because knowledge graphs are incomplete, it is often necessary to complete the knowledge graph through representation learning methods. With the development of pre-trained language models, more and more research applies them to the field of knowledge graph representation learning, using the powerful semantic representation capabilities of pre-trained language models to improve the performance of knowledge graph embedding. Most of the existing methods make use of the semantic information of the triple text but do not fully consider the structural information of the triple and the graph structure information of the knowledge graph. The triple structure reflects the semantic information and relationship pattern of the triple, and the graph structure reflects the surrounding entity's semantic features. To address the above issues, this paper proposes a knowledge graph representation learning method named PREGSE, which is based on pre-trained language models and integrates graph structure information. Firstly, pre-trained language models are employed to encode triplets through text encoding, obtaining vectors for the triplets. Secondly, a graph attention network is utilized to learn various local graph structure information. Lastly, a multi-task learning strategy is applied to simultaneously learn triplet structure information and semantic information. We trained our model on the FB15k-237 and WN18RR datasets, and the results show that on the FB15k-237 dataset, our model improved the MRR metric by 27 % and the Hits@10 metric by 8 % compared to the StAR model. The experiments show that our model can further improve the performance of knowledge graph representation learning.