Knowledge Graph Embedding Research Articles

Towards assessing the quality of knowledge graphs via differential testing

Knowledge graphs (KG) can aggregate data and make information resources easier to calculate and understand. With tremendous advancements in knowledge graphs, they have been incorporated into plenty of software systems to assist various tasks. However, while KGs determine the performance of downstream software systems, their quality is often measured by the accuracy of test data. Considering the limitation of accessible high-quality test data, an automated quality assessment technique could fundamentally improve the testing efficiency of KG-driven software systems and save plenty of manual labeling resources.In this paper, we propose an automated approach to quantify the quality of KGs via differential testing. It first constructs multiple Knowledge Graph Embedding Models (KGEM) and conducts head prediction tasks on models. Then, it can produce a differential score that reflects the quality of KGs by comparing the proximity of output results. To validate the effectiveness of this approach, we experiment with four open-sourced knowledge graphs. The experiment results show that our approach is capable of accurately evaluating the quality of KGs and producing reliable results on different datasets. Moreover, we compared our method with existing methods and achieved certain advantages. The potential usefulness of our approach sheds light on the development of various KG-driven software systems.

Integrating adaptive fuzzy embedding with topology and property hypergraphs: Enhancing membership degree-aware knowledge graph reasoning

Knowledge Graphs (KG) leverage the Resource Description Framework (RDF) to model clear and static resources, enabling the revelation of latent meanings through embedding and reasoning. However, their effectiveness diminishes when facing the inherent vagueness and dynamism of real-world information. To overcome this limitation, we introduce a novel adaptive fuzzy RDF KG embedding approach. This method represents fuzzy knowledge by integrating topological structures and descriptive properties while generating membership degrees to enhance the reasoning capabilities of KG. Specifically, our approach introduces a way to create embeddings based on local and global topological pattern mining. Then, path quality-based adaptive learning is implemented. Next, we incorporate entity descriptive information and construct a property hypergraph from another perspective to learn embeddings. By creatively combining these three components, our method can generate a fuzzy membership degree to aid knowledge reasoning. To thoroughly assess the method's effectiveness, experiments were conducted around four key questions, evaluating algorithm performance compared with other work and assessing knowledge reasoning effectiveness based on logical inference rules on Unmanned Aerial Vehicle (UAV) datasets.

Spatio-temporal knowledge embedding method considering the lifecycle of geographical entities

With the emergence of substantial amounts of data featuring spatio-temporal characteristics, research on spatio-temporal knowledge has garnered widespread attention. However, the existing knowledge graph embedding methods are difficult to embed spatio-temporal knowledge in continuous time, failing to meet the unique embedding requirements of geographic entities for uninterrupted, constantly changing, and smoothly continuous embeddings. To address this issue, we begin by outlining a version-based representation of spatio-temporal knowledge. Building on this foundation, we propose a novel and effective knowledge graph embedding model called Version Embedding (VerE), which embeds continuous time into vector space, transforms geographic entities into corresponding versions using an attention mechanism, and calculates spatio-temporal knowledge scores under the constraint of version similarity regularization. Subsequently, we introduce a method for link prediction based on unknown time with the aim of assessing the model’s generalization capabilities in time representation. Finally, experiments conducted on two real-world datasets demonstrate significant performance improvement of VerE compared to most existing models, and its ability to maintain stability and continuity in link prediction under unknown time. These experiments confirmed the effectiveness of the proposed model and provided new perspectives and methods for spatio-temporal knowledge embedding.

Towards knowledge-infused automated disease diagnosis assistant

With the advancement of internet communication and telemedicine, people are increasingly turning to the web for various healthcare activities. With an ever-increasing number of diseases and symptoms, diagnosing patients becomes challenging. In this work, we build a diagnosis assistant to assist doctors, which identifies diseases based on patient–doctor interaction. During diagnosis, doctors utilize both symptomatology knowledge and diagnostic experience to identify diseases accurately and efficiently. Inspired by this, we investigate the role of medical knowledge in disease diagnosis through doctor–patient interaction. We propose a two-channel, knowledge-infused, discourse-aware disease diagnosis model (KI-DDI), where the first channel encodes patient–doctor communication using a transformer-based encoder, while the other creates an embedding of symptom-disease using a graph attention network (GAT). In the next stage, the conversation and knowledge graph embeddings are infused together and fed to a deep neural network for disease identification. Furthermore, we first develop an empathetic conversational medical corpus comprising conversations between patients and doctors, annotated with intent and symptoms information. The proposed model demonstrates a significant improvement over the existing state-of-the-art models, establishing the crucial roles of (a) a doctor’s effort for additional symptom extraction (in addition to patient self-report) and (b) infusing medical knowledge in identifying diseases effectively. Many times, patients also show their medical conditions, which acts as crucial evidence in diagnosis. Therefore, integrating visual sensory information would represent an effective avenue for enhancing the capabilities of diagnostic assistants.

Predicting gene disease associations with knowledge graph embeddings for diseases with curtailed information.

Knowledge graph embeddings (KGE) are a powerful technique used in the biomedical domain to represent biological knowledge in a low dimensional space. However, a deep understanding of these methods is still missing, and, in particular, regarding their applications to prioritize genes associated with complex diseases with reduced genetic information. In this contribution, we built a knowledge graph (KG) by integrating heterogeneous biomedical data and generated KGE by implementing state-of-the-art methods, and two novel algorithms: Dlemb and BioKG2vec. Extensive testing of the embeddings with unsupervised clustering and supervised methods showed that KGE can be successfully implemented to predict genes associated with diseases and that our novel approaches outperform most existing algorithms in both scenarios. Our findings underscore the significance of data quality, preprocessing, and integration in achieving accurate predictions. Additionally, we applied KGE to predict genes linked to Intervertebral Disc Degeneration (IDD) and illustrated that functions pertinent to the disease are enriched within the prioritized gene set.

KEMoS: A knowledge-enhanced multi-modal summarizing framework for Chinese online meetings

The demand for “online meetings” and “collaborative office work” keeps surging recently, producing an abundant amount of relevant data. How to provide participants with accurate and fast summarizing service has attracted extensive attention. Existing meeting summarizing models overlook the utilization of multi-modal information and the information offsetting during summarizing. In this paper, we develop a knowledge-enhanced multi-modal summarizing framework. Firstly, we construct a three-layer multi-modal meeting knowledge graph, including basic, knowledge, and multi-modal layer, to integrate meeting information thoroughly. Then, we raise a topic-based hierarchical clustering approach, which considers information entropy and difference simultaneously, to capture the semantic evolution of meetings. Next, we devise a multi-modal enhanced encoding strategy, including a sentence-level cross-modal encoder, a joint loss function, and a knowledge graph embedding module, to learn the meeting and topic-level presentations. Finally, when generating summaries, we design a topic-enhanced decoding strategy for the Transformer decoder which mitigates semantic offsetting with the aid of topic information. Extensive experiments show that our proposed work consistently outperforms state-of-the-art solutions on the Chinese meeting dataset, where the ROUGE-1, ROUGE-2, and ROUGE-L are 49.98%, 21.03%, and 32.03% respectively.

Multichannel Multimodal Emotion Analysis of Cross-Modal Feedback Interactions Based on Knowledge Graph

Multimodal sentiment analysis is a downstream branch task of sentiment analysis with high attention at present. Previous work in multimodal sentiment analysis have focused on the representation and fusion of modalities, capturing the underlying semantic relationships between modalities by considering contextual information. While this approach is feasible for simple contextual comments, more complex comments require the integration of external knowledge to obtain more accurate sentiment information. However, incorporating external knowledge into sentiment analysis to enhance information complementarity has not been thoroughly investigated. To address this, we propose a multichannel cross-modal feedback interaction model that incorporates the knowledge graph into multimodal sentiment analysis. Our proposed model consists of two main components: the cross-modal feedback recurrent interaction module and the external knowledge module for capturing latent information. The cross-modal interaction employs a self-feedback mechanism during network training, extracting feature representations of each modality and using these representations to mask sensory inputs, allowing the model to perform feedback-based feature masking. The external knowledge graph captures potential semantic information representations in the textual data through knowledge graph embedding. Finally, a global feature fusion module is employed for multichannel multimodal information integration. On two publicly available datasets, our method demonstrates good performance in terms of accuracy and F1 scores, compared to state-of-the-art models and several baselines.

Improving static and temporal knowledge graph embedding using affine transformations of entities

To find a suitable embedding for a knowledge graph (KG) remains a big challenge nowadays. By measuring the distance or plausibility of triples and quadruples in static and temporal knowledge graphs, many reliable knowledge graph embedding (KGE) models are proposed. However, these classical models may not be able to represent and infer various relation patterns well, such as TransE cannot represent symmetric relations, DistMult cannot represent inverse relations, RotatE cannot represent multiple relations, etc.. In this paper, we improve the ability of these models to represent various relation patterns by introducing the affine transformation framework. Specifically, we first utilize a set of affine transformations related to each relation or timestamp to operate on entity vectors, and then these transformed vectors can be applied not only to static KGE models, but also to temporal KGE models. The main advantage of using affine transformations is their good geometry properties with interpretability. Our experimental results demonstrate that the proposed intuitive design with affine transformations provides a statistically significant increase in performance with adding a few extra processing steps and keeping the same number of embedding parameters. Taking TransE as an example, we employ the scale transformation (the special case of an affine transformation). Surprisingly, it even outperforms RotatE to some extent on various datasets. We also introduce affine transformations into RotatE, Distmult, ComplEx, TTransE and TComplEx respectively, and experiments demonstrate that affine transformations consistently and significantly improve the performance of state-of-the-art KGE models on both static and temporal knowledge graph benchmarks.

Natural Language Processing (NLP) in Recommendation Systems

This paper delves into the application of Natural Language Processing (NLP) techniques in recommendation systems, specifically focusing on novel approaches to enhance recommendation accuracy and user satisfaction. The utilization of NLP algorithms has revolutionized how content is recommended to users, leveraging linguistic analysis and machine learning to understand user preferences and provide tailored suggestions. Our research explores various NLP methodologies, including sentiment analysis, topic modelling, and semantic analysis, to extract meaningful insights from textual data. Furthermore, we investigate the integration of deep learning models, such as neural networks and transformer architectures, to capture complex patterns and improve recommendation precision. A key highlight of our study is the introduction of a novel recommendation method termed "Knowledge Graph Embedding for Contextual Recommendation." This innovative approach combines knowledge graph representation with contextual understanding, allowing for more nuanced and personalized recommendations based on user interactions, historical data, and contextual relevance. We delve into the intricacies of this technique, detailing its implementation, training process, and evaluation metrics.

ShallowBKGC: a BERT-enhanced shallow neural network model for knowledge graph completion

Knowledge graph completion aims to predict missing relations between entities in a knowledge graph. One of the effective ways for knowledge graph completion is knowledge graph embedding. However, existing embedding methods usually focus on developing deeper and more complex neural networks, or leveraging additional information, which inevitably increases computational complexity and is unfriendly to real-time applications. In this article, we propose an effective BERT-enhanced shallow neural network model for knowledge graph completion named ShallowBKGC. Specifically, given an entity pair, we first apply the pre-trained language model BERT to extract text features of head and tail entities. At the same time, we use the embedding layer to extract structure features of head and tail entities. Then the text and structure features are integrated into one entity-pair representation via average operation followed by a non-linear transformation. Finally, based on the entity-pair representation, we calculate probability of each relation through multi-label modeling to predict relations for the given entity pair. Experimental results on three benchmark datasets show that our model achieves a superior performance in comparison with baseline methods. The source code of this article can be obtained from https://github.com/Joni-gogogo/ShallowBKGC.

The RDF2vec family of knowledge graph embedding methods

Knowledge graph embeddings represent a group of machine learning techniques which project entities and relations of a knowledge graph to continuous vector spaces. RDF2vec is a scalable embedding approach rooted in the combination of random walks with a language model. It has been successfully used in various applications. Recently, multiple variants to the RDF2vec approach have been proposed, introducing variations both on the walk generation and on the language modeling side. The combination of those different approaches has lead to an increasing family of RDF2vec variants. In this paper, we evaluate a total of twelve RDF2vec variants on a comprehensive set of benchmark models, and compare them to seven existing knowledge graph embedding methods from the family of link prediction approaches. Besides the established GEval benchmark introducing various downstream machine learning tasks on the DBpedia knowledge graph, we also use the new DLCC (Description Logic Class Constructors) benchmark consisting of two gold standards, one based on DBpedia, and one based on synthetically generated graphs. The latter allows for analyzing which ontological patterns in a knowledge graph can actually be learned by different embedding. With this evaluation, we observe that certain tailored RDF2vec variants can lead to improved performance on different downstream tasks, given the nature of the underlying problem, and that they, in particular, have a different behavior in modeling similarity and relatedness. The findings can be used to provide guidance in selecting a particular RDF2vec method for a given task.

M[formula omitted]ixKG: Mixing for harder negative samples in knowledge graph

Knowledge graph embedding (KGE) involves mapping entities and relations to low-dimensional dense embeddings, enabling a wide range of real-world applications. The mapping is achieved via distinguishing the positive and negative triplets in knowledge graphs. Therefore, how to design high-quality negative triplets is critical in the effectiveness of KEG models. Existing KGE models face challenges in generating high-quality negative triplets. Some models employ simple static distributions, i.e. uniform or Bernoulli distribution, and it is difficult for these methods to be trained distinguishably because of the sampled uninformative negative triplets. Furthermore, current methods are confined to constructing negative triplets from existing entities within the knowledge graph, limiting their ability to explore harder negatives. We introduce a novel mixing strategy in knowledge graphs called M2ixKG. M2ixKG adopts mixing operation in generating harder negative samples from two aspects: one is mixing among the heads and tails in triplets with the same relation to strengthen the robustness and generalization of the entity embeddings; the other is mixing the negatives with high scores to generate harder negatives. Our experiments, utilizing three datasets and four classical score functions, highlight the exceptional performance of M2ixKG in comparison to previous negative sampling algorithms.

Enhancing Knowledge Graph Embedding with Hierarchical Self-Attention and Graph Neural Network Techniques for Drug-Drug Interaction Prediction in Virtual Reality Environments

In biomedicine, the critical task is to decode Drug–Drug Interactions (DDIs) from complex biomedical texts. The scientific community employs Knowledge Graph Embedding (KGE) methods, enhanced with advanced neural network technologies, including capsule networks. However, existing methodologies primarily focus on the structural details of individual entities or relations within Biomedical Knowledge Graphs (BioKGs), overlooking the overall structural context of BioKGs, molecular structures, positional features of drug pairs, and their critical Relational Mapping Properties. To tackle the challenges identified, this study presents HSTrHouse an innovative hierarchical self-attention BioKGs embedding framework. This architecture integrates self-attention mechanisms with advanced neural network technologies, including Convolutional Neural Network (CNN) and Graph Neural Network (GNN), for enhanced computational modeling in biomedical contexts. The model bifurcates the BioKGs into entity and relation layers for structural analysis. It employs self-attention across these layers, utilizing PubMedBERT and CNN for position feature extraction, and a GNN for drug pair molecular structure analysis. Then, we connect the position and molecular structure features to integrate them into the self-attention calculation of entity and relation. After that, the output of the self-attention layer is combined with the connected vectors of the position feature and molecular structure feature to obtain the final representation vector, and finally, to model the Relational Mapping Properties (RMPs), the representation vector is embedded into the complex vector space using Householder projections to obtain the BioKGs model. The paper validates HSTrHouse’s efficacy by comparing it with advanced models on three standard BioKGs for DDIs research.

A knowledge-driven graph convolutional network for abnormal electrocardiogram diagnosis

The electrocardiogram (ECG) signal comprising P-, Q-, R-, S-, and T-waves is an indispensable noninvasive diagnostic tool for analyzing physiological conditions of the heart. In general, traditional ECG intelligent diagnosis methods gradually extract features of the signal from input data until they can classify the ECG signal. However, the decision-making process of ECG intelligence models is implicit to clinicians. Clinical experts rely on clear and specific features extracted from ECG data to diagnose cardiac diseases effectively. Inspired by this clinical diagnosis mechanism, we propose an ECG knowledge graph (ECG-KG) framework primarily to improve ECG classification by presenting knowledge of ECG clinical diagnosis. In particular, the ECG-KG framework contains an ECG semantic feature extraction module, a knowledge graph construction module, and an ECG classification module. First, the ECG semantic feature extraction module locates the key points using the difference value method and further calculates the ECG attribute features. Further, the knowledge graph construction module utilizes attribute features to design entities and relationships for constructing abnormal ECG triples. The triples vectorize ECG abnormalities through the strategy of knowledge graph embedding strategy. Finally, the ECG classification module combines the ECG knowledge graph with the graph convolutional network model and adequately integrates expert knowledge to identify ECG abnormalities. Experiments conducted on the benchmark QT, the CPSC-2018, and the ZZU-ECG datasets show that the ECG-KG framework considerably outperforms other ECG diagnosis models, indicating the effectiveness of the ECG-KG framework for ECG abnormality diagnosis.

A knowledge graph embedding model based on multi-level analogical reasoning

A knowledge graph embedding model based on multi-level analogical reasoning

An embedding model for temporal knowledge graphs with long and irregular intervals

As the basis of various downstream applications such as automated question answering and knowledge reasoning, Temporal Knowledge Graph (TKG) embedding learning has attracted many interests. Recent works have explored several methods to learn dynamic embedding for TKGs and led to success in many applications, however, they are difficult to be optimized. In order to accurately catch semantic information propagation of entity-to-entity and entity-to-relation, as well as track dynamics of entities and relations in continuous time, we proposed a novel TKG embedding model ODETKGE. Instead of using recurrent neural network to model dynamics of semantic representations in continuous time, we leverage neural ordinary differential equations to model the dynamic evolution, which enable us to capture long-range dependencies between embeddings at different timestamps, track the dynamics of TKGs with irregular intervals and remain computationally efficient. Furthermore, to effectively model semantics of knowledge graph structure, we combine entity-relation collaboratively updated graph convolutional networks and neural ordinary differential equations. Additionally, to acquire important semantics such as creation and disappearance of relations, we utilize a graph transition layer. Comparing with RNN based models, the proposed ODETKGE can use fewer parameters to achieve deeper layers, and make the transformation of representations between two timestamps smoother. Extensive experiments have been executed on real-world datasets and the results verify that ODETKGE is superior than the state-of-the-art models in link forecasting, long-interval link forecasting, irregular-interval link forecasting, ablation study and training time.

Relation correlations-aware graph convolutional network with text-enhanced for knowledge graph embedding

Relation correlations-aware graph convolutional network with text-enhanced for knowledge graph embedding

Hierarchical knowledge graph relationship prediction leverage of axiomatic fuzzy set graph structure

Knowledge graph embedding has found widespread application across various fields due to its inherent structured data. However, some non-graph-based models necessitate improvement in accurately describing complex hierarchical structures and connecting intricate relational paths. Graph neural networks embedding methods provide a promising direction for exploring complex hierarchical modeling, facilitating the flow of information from nodes along relational paths. Nevertheless, the deficient interpretable characterization by the saliency estimation in entities, which contributes to the efficient performance of the model, is difficult to comprehend. This paper proposes a novel model to mitigate the described phenomenon by intensifying the original structure of knowledge graphs. Specifically, it integrates the Axiomatic Fuzzy Set entity semantic extraction framework with a heterogeneous relationship self-attention mechanism to bolster node representation. Next, this framework incorporates a convolutional neural network model to increase the interaction capabilities of entities and relations. This interpretable entity-level attention mechanism complements the structural information of the knowledge graphs, enabling more comprehensive neighbor information capture and improving the ability to identify the correct entity location. Experimental validation on two relationally complex datasets demonstrates superior performance compared to current methods, obtaining a new Hit@1 value of 45.6% on WN18RR.

Link Prediction Based on Data Augmentation and Metric Learning Knowledge Graph Embedding

Link Prediction Based on Data Augmentation and Metric Learning Knowledge Graph Embedding

Repurposing non-pharmacological interventions for Alzheimer's disease through link prediction on biomedical literature

Non-pharmaceutical interventions (NPI) have great potential to improve cognitive function but limited investigation to discover NPI repurposing for Alzheimer's Disease (AD). This is the first study to develop an innovative framework to extract and represent NPI information from biomedical literature in a knowledge graph (KG), and train link prediction models to repurpose novel NPIs for AD prevention. We constructed a comprehensive KG, called ADInt, by extracting NPI information from biomedical literature. We used the previously-created SuppKG and NPI lexicon to identify NPI entities. Four KG embedding models (i.e., TransE, RotatE, DistMult and ComplEX) and two novel graph convolutional network models (i.e., R-GCN and CompGCN) were trained and compared to learn the representation of ADInt. Models were evaluated and compared on two test sets (time slice and clinical trial ground truth) and the best performing model was used to predict novel NPIs for AD. Discovery patterns were applied to generate mechanistic pathways for high scoring candidates. The ADInt has 162,212 nodes and 1,017,284 edges. R-GCN performed best in time slice (MR = 5.2054, Hits@10 = 0.8496) and clinical trial ground truth (MR = 3.4996, Hits@10 = 0.9192) test sets. After evaluation by domain experts, 10 novel dietary supplements and 10 complementary and integrative health were proposed from the score table calculated by R-GCN. Among proposed novel NPIs, we found plausible mechanistic pathways for photodynamic therapy and Choerospondias axillaris to prevent AD, and validated psychotherapy and manual therapy techniques using real-world data analysis. The proposed framework shows potential for discovering new NPIs for AD prevention and understanding their mechanistic pathways.