Knowledge Graph Embedding Research Articles

Universal Knowledge Graph Embedding Framework Based on High-Quality Negative Sampling and Weighting

The traditional model training approach based on negative sampling randomly samples a portion of negative samples for training, which can easily overlook important negative samples and adversely affect the training of knowledge graph embedding models. Some researchers have explored non-sampling model training frameworks that use all unobserved triples as negative samples to improve model training performance. However, both training methods inevitably introduce false negative samples and easy-to-separate negative samples that are far from the model’s decision boundary, and they do not consider the adverse effects of long-tail entities and relations during training, thus limiting the improvement of model training performance. To address this issue, we propose a universal knowledge graph embedding framework based on high-quality negative sampling and weighting, called HNSW-KGE. First, we conduct pre-training based on the NS-KGE non-sampling training framework to quickly obtain an initial set of relatively high-quality embedding vector representations for all entities and relations. Second, we design a candidate negative sample set construction strategy that samples a certain number of negative samples that are neither false negatives nor easy-to-separate negatives for all positive triples, based on the embedding vectors obtained from pre-training. This ensures the provision of high-quality negative samples for model training. Finally, we apply weighting to the loss function based on the frequency of the entities and relations appearing in the triples to mitigate the adverse effects of long-tail entities and relations on model training. Experiments conducted on benchmark datasets FB15K237 and WN18RR using various knowledge graph embedding models demonstrate that our proposed framework HNSW-KGE, based on high-quality negative sampling and weighting, achieves better training performance and exhibits versatility, making it applicable to various types of knowledge embedding models.

Large-Scale Knowledge Graphs as a Tool for Enhanced Robotic Perception

Autonomous robotic systems depend on their perception and understanding of their environment for informed decision-making. One of the goals of the Semantic Web is to make knowledge on the Web machine-readable, which can significantly aid robots by providing background knowledge, and thereby support their understanding. In this paper, we present a reasoning system that uses the Ontology for Robotic Knowledge Acquisition (ORKA) to integrate the sensory data and perception algorithms of the robot, thereby enhancing its autonomous capabilities. This reasoning system is subsequently employed to retrieve and integrate information from the Semantic Web, thereby improving the robot's comprehension of its environment. To achieve this, the system employs a Perceived-Entity Linking (PEL) pipeline that associates regions in the sensory data of the robotic agent with concepts in a target knowledge graph. As a use-case for the linking process, the Perceived-Entity Typing task is used to determine the more fine-grained subclass of the perceived entities. Specifically, we provide an analysis of the performance of different knowledge graph embedding methods on the task using a annotated observations and WikiData as a target knowledge graph. The experiments indicate that relying on pre-trained embedding methods results in an increased performance when using TransE as the embedding method for the observations of the robot. This contribution advances the field by demonstrating the potential of integrating Semantic Web technologies with robotic perception, thereby enabling more nuanced and context-aware decision-making in autonomous systems.

Uncovering Multi-step Attacks with Threat Knowledge Graph Reasoning

The rapid advancement of information technologies has significantly intensified the focus on cyberspace security across various sectors. In this evolving landscape, attackers deploy many of techniques- including exploits, weakness identification, and complex multi-step attacks- to gain unauthorized access to systems. Conversely, defenders harness insights from a variety of sources to pinpoint potential threats. Prominent public cybersecurity databases such as the Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK), Common Attack Pattern Enumeration and Classification (CAPEC), Common Vulnerabilities and Exposures (CVE), Common Weakness Enumeration (CWE), and Common Platform Enumeration (CPE) provide extensive data on security entities and their interrelations, playing a pivotal role in enriching the understanding of cybersecurity challenges and assisting in comprehensive defensive analyses. However, the semantic cross-analysis of these databases, crucial for identifying obscure threat patterns, remains underexploited. In this study, we amalgamate data from these disparate sources into a cohesive threat knowledge graph and introduce a novel knowledge representation learning approach, A4CKGE (ATT&CK-CAPEC-CWE-CVE-CPE Knowledge Graph Embedding). This method utilizes advanced structural and textual analytics to predict interactions among security entities such as products, vulnerabilities, weaknesses, and multi-step attack sequences, employing complex attack templates generated through a Large Language Model (LLM). Our extensive experiments demonstrate that this approach significantly outperforms existing state-of-the-art methods in effectively predicting these relationships. The findings validate the efficacy of our threat knowledge graph in unveiling hidden connections, thereby highlighting its potential to strengthen cybersecurity defenses substantially.

Integrated Knowledge Graph and Drug Molecular Graph Fusion via Adversarial Networks for Drug-Drug Interaction Prediction.

The Co-administration of multiple drugs can enhance the efficacy of disease treatment by reducing drug resistance and side effects. However, it also raises the risk of adverse drug interactions, presenting a challenging problem in healthcare. Various approaches have been developed to predict drug-drug interactions (DDIs) by leveraging both knowledge graphs and drug attribute information. While these methods have shown promise, they often fail to effectively capture correlations between biomedical information in the knowledge graph and drug properties. This work introduces a novel end-to-end DDI predictor framework based on generative adversarial networks. This framework utilizes a message-passing neural network to capture molecular structure information while employing the knowledge-aware graph attention network to capture the representation of drugs in the knowledge graph through considering the importance of different multihop neighbor nodes and relationships. The dual generative adversarial networks employ two generators and two discriminators in knowledge graph embedding and molecular topology embedding for adversarial training to capture the interrelations and complementary knowledge between molecular structure information and semantic information from the knowledge graph. comprehensive experiments have demonstrated that the proposed method outperforms state-of-the-art algorithms in binary classification, with improvements of 1.0% in accuracy, 0.45% in area under the receiver operating characteristic curve (AUC), 0.24% in area under the precision-recall curve (AUPR), and 0.98% in F1 score. Furthermore, for multiclass classification tasks, improvements were observed across various evaluation metrics, including 0.9% in accuracy, 0.25% in macro precision, 0.2% in macro recall, and 0.28% in macro F1. Additionally, ablation studies were conducted to showcase the effectiveness and robustness of our method in DDI prediction tasks.

Knowledge Graph Embedding for Hierarchical Entities Based on Auto-Embedding Size

Knowledge graph embedding represents entities and relations as low-dimensional continuous vectors. Recently, researchers have attempted to leverage the potential semantic connections between entities with hierarchical relationships in the knowledge graph. However, existing knowledge embedding methods that model entity hierarchical structures face the following challenges: (1) Setting the same embedding dimension for entities at different hierarchical levels can lead to overfitting issues and the waste of storage and computational resources; (2) Manually searching the embedding dimension in discrete space makes it easy to miss the optimal parameters and increases training costs. Therefore, we propose a knowledge embedding method, HEAES, that automatically learns the embedding dimensions for entities based on their hierarchical structure. Specifically, we first propose a new modeling method to capture the hierarchical relationships of entities, and we then train the model on a triple classification task. Then, we adaptively learn the pruning threshold to trim the embedding vectors, automatically learning different embedding dimensions for entities at different hierarchical levels. Experiments on the YAGO26K and DB111K datasets verify that introducing entity hierarchical information helps boost the model performance.

GeoEntity-type constrained knowledge graph embedding for predicting natural-language spatial relations

Natural-language spatial relations between geographic entities (geoentities) reflect diverse perceptions influenced by factors like location, culture, and linguistic conventions. These relations play a crucial role in supporting geospatial tasks, such as question answering and cognitive reasoning. While prior studies focused on a limited set of human-selected spatial terms and geometric attributes, they often overlooked essential semantic attributes. To overcome this limitation, we developed a Spatial Relation-based Knowledge Graph Embedding framework, SR-KGE, with new KG fusion functions to predict spatial relation terms among distinct geoentities. This method not only considers graph structures and the diversity of natural language expressions in the embedding and learning process, but also incorporates geoentity types as a constraint to capture spatial and semantic relations more accurately. Our experiments on two knowledge graph datasets, one small-scale and one large-scale, have both shown its superior performance in spatial relation inference compared to popular KGE models, including TransE, RotatE, and HAKE. We hope our research will advance the classic study of natural language described spatial relations in a more automated and intelligent way.

A knowledge graph-based inspection items recommendation method for port state control inspection of LNG carriers

The safe navigation and operation of Liquified natural gas (LNG) carriers is crucial in maritime transportation. The Port State Control (PSC) inspection is the primary method for identifying deficiencies in vessels. Different from other vessels, the PSC inspection of LNG carriers requires knowledge of specialized transport equipment. Therefore, effective management of inspection knowledge and precise targeting of inspection items are particularly crucial. Knowledge graph is an efficient way to manage knowledge in the context of artificial intelligence. In this study, an LNG carrier PSC inspection knowledge graph is constructed for fusing multisource knowledge data from PSC inspection. Additionally, a knowledge graph-based recommendation model, namely the improved knowledge graph convolutional network combined with pretrained translation embedding (PT-KGCN), is developed to recommend inspection items and knowledge. The PT-KGCN model first carries out knowledge graph embedding. It then predicts possible deficiencies on the basis of historical PSC inspection data. Finally, it recommends inspection items by correlating the predicted deficiencies with knowledge from the knowledge graph. The results show that more than 87% of defective items in historical data are correctly predicted. The research findings can provide ideas for the practical application of knowledge data in the inspection fields.

HGCGE: hyperbolic graph convolutional networks-based knowledge graph embedding for link prediction

HGCGE: hyperbolic graph convolutional networks-based knowledge graph embedding for link prediction

TarKG: a comprehensive biomedical knowledge graph for target discovery.

Target discovery is a crucial step in drug development, as it directly affects the success rate of clinical trials. Knowledge graphs (KGs) offer unique advantages in processing complex biological data and inferring new relationships. Existing biomedical KGs primarily focus on tasks such as drug repositioning and drug-target interactions, leaving a gap in the construction of KGs tailored for target discovery. We established a comprehensive biomedical KG focusing on target discovery, termed TarKG, by integrating seven existing biomedical KGs, nine public databases, and traditional Chinese medicine knowledge databases. TarKG consists of 1143313 entities and 32806467 relations across 15 entity categories and 171 relation types, all centered around 3 core entity types: Disease, Gene, and Compound. TarKG provides specialized knowledges for the core entities including chemical structures, protein sequences, or text descriptions. By using different KG embedding algorithms, we assessed the knowledge completion capabilities of TarKG, particularly for disease-target link prediction. In case studies, we further examined TarKG's ability to predict potential protein targets for Alzheimer's disease (AD) and to identify diseases potentially associated with the metallo-deubiquitinase CSN5, using literature analysis for validation. Furthermore, we provided a user-friendly web server (https://tarkg.ddtmlab.org) that enables users to perform knowledge retrieval and relation inference using TarKG. TarKG is accessible at https://tarkg.ddtmlab.org.

A multi-entity reinforced main path analysis: Heterogeneous network embedding considering knowledge proximity

Main path analysis (MPA) is an important approach in detecting the trajectory of knowledge diffusion in a specific research domain. Previous studies always focus on citation-based relationships, overlooking other structural forms in citation network. This study introduces a multi-entity reinforced MPA model by constructing a knowledge graph from paper metadata, including citations, authors, journals, and keywords. We construct heterogeneous network to reveal relationships among various entities. Different knowledge graph embedding models are employed to train the network, thereby obtaining entity and relation embeddings. The cosine similarity algorithm is adopted to measure the knowledge proximity between these embeddings. We take the Internet of Thing domain as an example to verify the performance of the multi-entity reinforced MPA through both quantitative and qualitative analysis. Our findings indicate that the adjusted MPA exhibits stronger topic relevance, demonstrating the effectiveness of the method in capturing complex knowledge relationships.

Recommender systems based on neuro-symbolic knowledge graph embeddings encoding first-order logic rules

Recommender systems based on neuro-symbolic knowledge graph embeddings encoding first-order logic rules

FuseLinker: Leveraging LLM’s pre-trained text embeddings and domain knowledge to enhance GNN-based link prediction on biomedical knowledge graphs

ObjectiveTo develop the FuseLinker, a novel link prediction framework for biomedical knowledge graphs (BKGs), which fully exploits the graph’s structural, textual and domain knowledge information. We evaluated the utility of FuseLinker in the graph-based drug repurposing task through detailed case studies. MethodsFuseLinker leverages fused pre-trained text embedding and domain knowledge embedding to enhance the graph neural network (GNN)-based link prediction model tailored for BKGs. This framework includes three parts: a) obtain text embeddings for BKGs using embedding-visible large language models (LLMs), b) learn the representations of medical ontology as domain knowledge information by employing the Poincaré graph embedding method, and c) fuse these embeddings and further learn the graph structure representations of BKGs by applying a GNN-based link prediction model. We evaluated FuseLinker against traditional knowledge graph embedding models and a conventional GNN-based link prediction model across four public BKG datasets. Additionally, we examined the impact of using different embedding-visible LLMs on FuseLinker’s performance. Finally, we investigated FuseLinker’s ability to generate medical hypotheses through two drug repurposing case studies for Sorafenib and Parkinson’s disease. ResultsBy comparing FuseLinker with baseline models on four BKGs, our method demonstrates superior performance. The Mean Reciprocal Rank (MRR) and Area Under receiver operating characteristic Curve (AUROC) for KEGG50k, Hetionet, SuppKG and ADInt are 0.969 and 0.987, 0.548 and 0.903, 0.739 and 0.928, and 0.831 and 0.890, respectively. ConclusionOur study demonstrates that FuseLinker is an effective novel link prediction framework that integrates multiple graph information and shows significant potential for practical applications in biomedical and clinical tasks. Source code and data are available at https://github.com/YKXia0/FuseLinker.

Research Progresses and Applications of Knowledge Graph Embedding Technique in Chemistry.

A knowledge graph (KG) is a technique for modeling entities and their interrelations. Knowledge graph embedding (KGE) translates these entities and relationships into a continuous vector space to facilitate dense and efficient representations. In the domain of chemistry, applying KG and KGE techniques integrates heterogeneous chemical information into a coherent and user-friendly framework, enhances the representation of chemical data features, and is beneficial for downstream tasks, such as chemical property prediction. This paper begins with a comprehensive review of classical and contemporary KGE methodologies, including distance-based models, semantic matching models, and neural network-based approaches. We then catalogue the primary databases employed in chemistry and biochemistry that furnish the KGs with essential chemical data. Subsequently, we explore the latest applications of KG and KGE in chemistry, focusing on risk assessment, property prediction, and drug discovery. Finally, we discuss the current challenges to KG and KGE techniques and provide a perspective on their potential future developments.

A robotic manipulation framework for industrial human–robot collaboration based on continual knowledge graph embedding

A robotic manipulation framework for industrial human–robot collaboration based on continual knowledge graph embedding

Identifying compound-protein interactions with knowledge graph embedding of perturbation transcriptomics

The emergence of perturbation transcriptomics provides a new perspective for drug discovery, but existing analysis methods suffer from inadequate performance and limited applicability. In this work, we present PertKGE, a method designed to deconvolute compound-protein interactions from perturbation transcriptomics with knowledge graph embedding. By considering multi-level regulatory events within biological systems that share the same semantic context, PertKGE significantly improves deconvoluting accuracy in two critical "cold-start" settings: inferring targets for new compounds and conducting virtual screening for new targets. We further demonstrate the pivotal role of incorporating multi-level regulatory events in alleviating representational biases. Notably, it enables the identification of ectonucleotide pyrophosphatase/phosphodiesterase-1 as the target responsible for the unique anti-tumor immunotherapy effect of tankyrase inhibitor K-756 and the discovery of five novel hits targeting the emerging cancer therapeutic targetaldehyde dehydrogenase 1B1 with a remarkable hit rate of 10.2%. These findings highlight the potential of PertKGE to accelerate drug discovery.

Decoupled semantic graph neural network for knowledge graph embedding

Knowledge graph embedding (KGE) learns an embedding space for a more accurate representation of entities and relations. Although the KGE model has proliferated, it often fails to fully capture the rich semantics in knowledge graphs. Some studies attempt to apply decoupling methods to decompose node representations, yet they neglect the semantic noise generated during the decoupled process. To address these issues, we introduce the decoupled semantic graph network for KGE (named DSGNet), which employs a novel approach that combines semantic decoupling with structural awareness. DSGNet begins by projecting the knowledge graph into distinct semantic spaces, ensuring minimal correlation between them to achieve effective semantic decoupling. To mitigate semantic noise, a top-k sampling technique is applied to the decoupled graphs. DSGNet then aggregates these different semantic graphs using a relation-aware aggregation module, followed by a multi-layer aggregation process for enhanced node representation. The extensive experiments demonstrate that DSGNet performs competitively on two popular datasets. We make our code publicly available at https://github.com/HubuKG/DSGNet.

Intelligent Asset Allocation Portfolio Division and Recommendation

With the continuous development of financial markets, intelligent asset allocation has become a topic of great concern in the investment field. However, traditional asset allocation methods often face difficulties in grasping the relationship between diversity, risk and return, which limits its application in complex market environments. To solve this problem, this study introduces deep learning and knowledge graphs and proposes an intelligent asset allocation model. Our model makes full use of the advantages of the Knowledge Graph Embedding Model (KGE), LSTM, and Genetic Algorithm (GA) to build a multi-level and multi-dimensional asset allocation model. KGE helps capture the complex relationships between different assets, LSTM is used to learn key patterns of historical portfolio performance, and GA finds the optimal asset allocation combination by simulating natural selection and genetic mechanisms. Experimental findings indicate that our model has demonstrated substantial improvements across various performance metrics and outperforms conventional approaches.

Knowledge Graph Embedding Using a Multi-Channel Interactive Convolutional Neural Network with Triple Attention

Knowledge Graph Embedding Using a Multi-Channel Interactive Convolutional Neural Network with Triple Attention

Community knowledge graph abstraction for enhanced link prediction: A study on PubMed knowledge graph

Objective:As new knowledge is produced at a rapid pace in the biomedical field, existing biomedical Knowledge Graphs (KGs) cannot be manually updated in a timely manner. Previous work in Natural Language Processing (NLP) has leveraged link prediction to infer the missing knowledge in general-purpose KGs. Inspired by this, we propose to apply link prediction to existing biomedical KGs to infer missing knowledge. Although Knowledge Graph Embedding (KGE) methods are effective in link prediction tasks, they are less capable of capturing relations between communities of entities with specific attributes (Fanourakis et al., 2023). Methods:To address this challenge, we proposed an entity distance-based method for abstracting a Community Knowledge Graph (CKG) from a simplified version of the pre-existing PubMed Knowledge Graph (PKG) (Xu et al., 2020). For link prediction on the abstracted CKG, we proposed an extension approach for the existing KGE models by linking the information in the PKG to the abstracted CKG. The applicability of this extension was proved by employing six well-known KGE models: TransE, TransH, DistMult, ComplEx, SimplE, and RotatE. Evaluation metrics including Mean Rank (MR), Mean Reciprocal Rank (MRR), and Hits@k were used to assess the link prediction performance. In addition, we presented a backtracking process that traces the results of CKG link prediction back to the PKG scale for further comparison. Results:Six different CKGs were abstracted from the PKG by using embeddings of the six KGE methods. The results of link prediction in these abstracted CKGs indicate that our proposed extension can improve the existing KGE methods, achieving a top-10 accuracy of 0.69 compared to 0.5 for TransE, 0.7 compared to 0.54 for TransH, 0.67 compared to 0.6 for DistMult, 0.73 compared to 0.57 for ComplEx, 0.73 compared to 0.63 for SimplE, and 0.85 compared to 0.76 for RotatE on their CKGs, respectively. These improved performances also highlight the wide applicability of the extension approach. Conclusion:This study proposed novel insights into abstracting CKGs from the PKG. The extension approach indicated enhanced performance of the existing KGE methods and has applicability. As an interesting future extension, we plan to conduct link prediction for entities that are newly introduced to the PKG.

A survey on temporal knowledge graph embedding: Models and applications

Knowledge graph embedding (KGE), as a pivotal technology in artificial intelligence, plays a significant role in enhancing the logical reasoning and management efficiency of downstream tasks in knowledge graphs (KGs). It maps the intricate structure of a KG to a continuous vector space. Conventional KGE techniques primarily focus on representing static data within a KG. However, in the real world, facts frequently change over time, as exemplified by evolving social relationships and news events. The effective utilization of embedding technologies to represent KGs that integrate temporal data has gained significant scholarly interest. This paper comprehensively reviews the existing methods for learning KG representations that incorporate temporal data. It offers a highly intuitive perspective by categorizing temporal KGE (TKGE) methods into seven main classes based on dynamic evolution models and extensions of static KGE. The review covers various aspects of TKGE, including the background, problem definition, symbolic representation, training process, commonly used datasets, evaluation schemes, and relevant research. Furthermore, detailed descriptions of related embedding models are provided, followed by an introduction to typical downstream tasks in temporal KG scenarios. Finally, the paper concludes by summarizing the challenges faced in TKGE and outlining future research directions.