Knowledge Graph Embedding Approaches Research Articles

A Knowledge Graph Embedding Model for Answering Factoid Entity Questions

Factoid entity questions (FEQ), which seek answers in the form of a single entity from knowledge sources such as DBpedia and Wikidata, constitute a substantial portion of user queries in search engines. This paper introduces the Knowledge Graph Embedding model for Factoid Entity Question answering (KGE-FEQ). Leveraging a textual knowledge graph derived from extensive text collections, KGE-FEQ encodes textual relationships between entities. The model employs a two-step process: (1) Triple Retrieval, where relevant triples are retrieved from the textual knowledge graph based on semantic similarities to the question, and (2) Answer Selection, where a knowledge graph embedding approach is utilized for answering the question. This involves positioning the embedding for the answer entity close to the embedding of the question entity, incorporating a vector representing the question and textual relations between entities. Extensive experiments evaluate the performance of the proposed approach, comparing KGE-FEQ to state-of-the-art baselines in factoid entity question answering and the most advanced open-domain question answering techniques applied to FEQs. The results show that KGE-FEQ outperforms existing methods across different datasets. Ablation studies highlights the effectiveness of KGE-FEQ when both the question and textual relations between entities are considered for answering questions.

Momentum portfolio selection based on learning-to-rank algorithms with heterogeneous knowledge graphs

Artificial intelligence techniques for financial time series analysis have been used to enhance momentum trading methods. However, most previous studies, which have treated stocks as independent entities, have overlooked the significance of correlations among individual stocks, thus compromising portfolio performance. To address this gap, a momentum trading framework is proposed that combines heterogeneous data, such as corporate governance factors and financial domain knowledge, to model the relationships between stocks. Our approach involves adopting a knowledge graph embedding approach to map relations among heterogeneous relationships in the data, which is then utilized to train a multitask supervised learning approach based on a learning-to-rank algorithm. This method culminates in a robust portfolio selection method on the basis of the framework. Experimental results using data from the Taiwan Stock Exchange demonstrate that our proposed method outperforms traditional linear models and other machine learning methods in predictive ability. The investment portfolio constructed serves as an invaluable aid to investment decision-making.

An Entity Ontology-Based Knowledge Graph Embedding Approach to News Credibility Assessment

An Entity Ontology-Based Knowledge Graph Embedding Approach to News Credibility Assessment

RoCS: Knowledge Graph Embedding Based on Joint Cosine Similarity

Knowledge graphs usually have many missing links, and predicting the relationships between entities has become a hot research topic in recent years. Knowledge graph embedding research maps entities and relations to a low-dimensional continuous space representation to predict links between entities. The present research shows that the key to the knowledge graph embedding approach is the design of scoring functions. According to the scoring function, knowledge graph embedding methods can be classified into dot product models and distance models. We find that the triple scores obtained using the dot product model or the distance model were unbounded, which leads to large variance. In this paper, we propose RotatE Cosine Similarity (RoCS), a method to compute the joint cosine similarity of complex vectors as a scoring function to make the triple scores bounded. Our approach combines the rotational properties of the complex vector embedding model RotatE to model complex relational patterns. The experimental results demonstrate that the newly introduced RoCS yields substantial enhancements compared to RotatE across various knowledge graph benchmarks, improving up to 4.0% in hits at 1 (Hits@1) on WN18RR and improving up to 3.3% in Hits@1 on FB15K-237. Meanwhile, our method achieves some new state-of-the-art (SOTA), including Hits@3 of 95.6%, Hits@10 of 96.4% on WN18, and mean reciprocal rank (MRR) of 48.9% and Hits@1 of 44.5% on WN18RR.

KE-X: Towards subgraph explanations of knowledge graph embedding based on knowledge information gain

Over the past years, knowledge graph embedding approaches have proven effective for knowledge graph completion tasks. However, most existing models are either built on a certain embedding space or black-box neural networks, making it hard to access explanations for prediction results and resulting in limited explainability. In this paper, we propose to leverage information entropy to quantify the importance of explanation candidates, then build a framework KE-X, for explaining results from knowledge graph embedding approaches by generating explainable subgraphs. Specifically, by performing a modified message passing mechanism on a partially masked knowledge subgraph and maximizing knowledge information gain, KE-X can extract the most valuable subgraph explanation for a link prediction query. To evaluate KE-X, we conduct experiments on three real-world knowledge graphs with two representative KGE models, TransE and DistMult. Both quantitative and case study results show that our framework can extract high-quality explanations.

Using Knowledge Graph Embedding for Fault Detection

Using Knowledge Graph Embedding for Fault Detection

DrugRep-KG: Toward Learning a Unified Latent Space for Drug Repurposing Using Knowledge Graphs.

Drug repurposing or repositioning (DR) refers to finding new therapeutic applications for existing drugs. Current computational DR methods face data representation and negative data sampling challenges. Although retrospective studies attempt to operate various representations, it is a crucial step for an accurate prediction to aggregate these features and bring the associations between drugs and diseases into a unified latent space. In addition, the number of unknown associations between drugs and diseases, which is considered negative data, is much higher than the number of known associations, or positive data, leading to an imbalanced dataset. In this regard, we propose the DrugRep-KG method, which applies a knowledge graph embedding approach for representing drugs and diseases, to address these challenges. Despite the typical DR methods that consider all unknown drug-disease associations as negative data, we select a subset of unknown associations, provided the disease occurs because of an adverse reaction to a drug. DrugRep-KG has been evaluated based on different settings and achieves an AUC-ROC (area under the receiver operating characteristic curve) of 90.83% and an AUC-PR (area under the precision-recall curve) of 90.10%, which are higher than in previous works. Besides, we checked the performance of our framework in finding potential drugs for coronavirus infection and skin-related diseases: contact dermatitis and atopic eczema. DrugRep-KG predicted beclomethasone for contact dermatitis, and fluorometholone, clocortolone, fluocinonide, and beclomethasone for atopic eczema, all of which have previously been proven to be effective in other studies. Fluorometholone for contact dermatitis is a novel suggestion by DrugRep-KG that should be validated experimentally. DrugRep-KG also predicted the associations between COVID-19 and potential treatments suggested by DrugBank, in addition to new drug candidates provided with experimental evidence. The data and code underlying this article are available at https://github.com/CBRC-lab/DrugRep-KG.

Prediction of Drug–Target Interaction Using Dual-Network Integrated Logistic Matrix Factorization and Knowledge Graph Embedding

Nowadays, drug–target interactions (DTIs) prediction is a fundamental part of drug repositioning. However, on the one hand, drug–target interactions prediction models usually consider drugs or targets information, which ignore prior knowledge between drugs and targets. On the other hand, models incorporating priori knowledge cannot make interactions prediction for under-studied drugs and targets. Hence, this article proposes a novel dual-network integrated logistic matrix factorization DTIs prediction scheme (Ro-DNILMF) via a knowledge graph embedding approach. This model adds prior knowledge as input data into the prediction model and inherits the advantages of the DNILMF model, which can predict under-studied drug–target interactions. Firstly, a knowledge graph embedding model based on relational rotation (RotatE) is trained to construct the interaction adjacency matrix and integrate prior knowledge. Secondly, a dual-network integrated logistic matrix factorization prediction model (DNILMF) is used to predict new drugs and targets. Finally, several experiments conducted on the public datasets are used to demonstrate that the proposed method outperforms the single base-line model and some mainstream methods on efficiency.

Learning structured embeddings of knowledge graphs with generative adversarial framework

Many large knowledge graphs are now available and ready to provide semantically structured information that is regarded as an important resource for question answering and decision support tasks. However, they are built on rigid symbolic frameworks which makes them hard to be used in other intelligent systems. Knowledge graph embedding approaches are gaining increasing attention, which embeds symbolic entities and relations into continuous vector spaces. Such graph embeddings are often learned by training a model to distinguish true triples from negative ones. Unfortunately, the negative triples created by replacing their heads or tails with randomly selected entities are easily identified by the model, which makes them insufficient to train useful models. To this end, we propose a method under a generative adversarial architecture to learn graph embeddings, in which a generative network is trained to provide continually improved “plausible” triples whereas a discriminative network learns to distinguish truth triples from the others by competing with the generator in a two-player minimax game. When arriving at a convergence, the generative network recovers the training data and can be used for knowledge graph completion, while the discriminative network is trained to be a good triple classifier. Extensive experiments demonstrate our method can improve multiple graph embedding models with a significant margin on both link prediction and triple classification tasks.

TransModE: Translational Knowledge Graph Embedding Using Modular Arithmetic

In order to improve link prediction process in a knowledge graph, we tackle the problem of learning representations of entities and relations. The strength of the existing models in this domain mainly relies on its ability of modeling and inferring the patterns of the relations. In this paper, we propose a new knowledge graph embedding approach called TransModE that has the ability to represent all simple and complex relation patterns. Inspired by TransE, TransModE represents the relations between the entities of a knowledge graph as a transition in the modulus space. Experimenting our model on multiple benchmark knowledge graphs shows the simplicity and the scalability of TransModE. It also shows that TransModE outperforms existing state-of-the-art models by being able to infer and model all types of relations.

Knowledge graph embeddings for dealing with concept drift in machine learning

Data stream learning has been largely studied for extracting knowledge structures from continuous and rapid data records. As data is evolving on a temporal basis, its underlying knowledge is subject to many challenges. Concept drift,11This work is addressing the challenge concept drift in machine learning as opposed to concept drift in the Semantic Web community where “concept” (class) meaning in ontology TBox shifts from versioning, iterations or modifications. Note that changes in ABox alone can also lead to concept drift in learning from ontology streams.. as one of core challenge from the stream learning community, is described as changes of statistical properties of the data over time, causing most of machine learning models to be less accurate as changes over time are in unforeseen ways. This is particularly problematic as the evolution of data could derive to dramatic change in knowledge. We address this problem by studying the semantic representation of data streams in the Semantic Web, i.e., ontology streams. Such streams are ordered sequences of data annotated with ontological vocabulary. In particular we exploit three levels of knowledge encoded in ontology streams to deal with concept drifts: i) existence of novel knowledge gained from stream dynamics, ii) significance of knowledge change and evolution, and iii) (in)consistency of knowledge evolution. Such knowledge is encoded as knowledge graph embeddings through a combination of novel representations: entailment vectors, entailment weights, and a consistency vector. We illustrate our approach on classification tasks of supervised learning. Key contributions of the study include: (i) an effective knowledge graph embedding approach for stream ontologies, and (ii) a generic consistent prediction framework with integrated knowledge graph embeddings for dealing with concept drifts. The experiments have shown that our approach provides accurate predictions towards air quality in Beijing and bus delay in Dublin with real world ontology streams.

DFraud³: Multi-Component Fraud Detection Free of Cold-Start

Fraud review detection is a hot research topic in recent years. The Cold-start is a particularly new but significant problem referring to the failure of a detection system to recognize the authenticity of a new user. State-of-the-art solutions employ a translational knowledge graph embedding approach ( TransE ) to model the interaction of the components of a review system. However, these approaches suffer from the limitation of TransE in handling N-1 relations and the narrow scope of a single classification task, i.e., detecting fraudsters only. In this paper, we model a review system as a Heterogeneous Information Network (HIN) which enables a unique representation to every component and performs graph inductive learning on the review data through aggregating features of nearby nodes. HIN with graph induction helps to address the camouflage issue (fraudsters with genuine reviews) which has shown to be more severe when it is coupled with cold-start, i.e., new fraudsters with genuine first reviews. In this research, instead of focusing only on one component, detecting either fraud reviews or fraud users (fraudsters), vector representations are learned for each component, enabling multi-component classification. In other words, we can detect fraud reviews, fraudsters, and fraud-targeted items, thus the name of our approach DFraud3. DFraud3 demonstrates a significant accuracy increase of 13% over the state of the art on Yelp.

Data stream event prediction based on timing knowledge and state transitions

We study a practical problem of predicting the upcoming events in data streams using a novel approach. Treating event time orders as relationship types between event entities, we build a dynamic knowledge graph and use it to predict future event timing. A unique aspect of this knowledge graph embedding approach for prediction is that we enhance conventional knowledge graphs with the notion of "states"---in what we call the ephemeral state nodes---to characterize the state of a data stream over time. We devise a complete set of methods for learning relevant events, for building the event-order graph stream from the original data stream, for embedding and prediction, and for theoretically bounding the complexity. We evaluate our approach with four real world stream datasets and find that our method results in high precision and recall values for event timing prediction, ranging between 0.7 and nearly 1, significantly outperforming baseline approaches. Moreover, due to our choice of efficient translation-based embedding, the overall throughput that the stream system can handle, including continuous graph building, training, and event predictions, is over one thousand to sixty thousand tuples per second even on a personal computer---which is especially important in resource constrained environments, including edge computing.

Evaluation of knowledge graph embedding approaches for drug-drug interaction prediction in realistic settings

BackgroundCurrent approaches to identifying drug-drug interactions (DDIs), include safety studies during drug development and post-marketing surveillance after approval, offer important opportunities to identify potential safety issues, but are unable to provide complete set of all possible DDIs. Thus, the drug discovery researchers and healthcare professionals might not be fully aware of potentially dangerous DDIs. Predicting potential drug-drug interaction helps reduce unanticipated drug interactions and drug development costs and optimizes the drug design process. Methods for prediction of DDIs have the tendency to report high accuracy but still have little impact on translational research due to systematic biases induced by networked/paired data. In this work, we aimed to present realistic evaluation settings to predict DDIs using knowledge graph embeddings. We propose a simple disjoint cross-validation scheme to evaluate drug-drug interaction predictions for the scenarios where the drugs have no known DDIs.ResultsWe designed different evaluation settings to accurately assess the performance for predicting DDIs. The settings for disjoint cross-validation produced lower performance scores, as expected, but still were good at predicting the drug interactions. We have applied Logistic Regression, Naive Bayes and Random Forest on DrugBank knowledge graph with the 10-fold traditional cross validation using RDF2Vec, TransE and TransD. RDF2Vec with Skip-Gram generally surpasses other embedding methods. We also tested RDF2Vec on various drug knowledge graphs such as DrugBank, PharmGKB and KEGG to predict unknown drug-drug interactions. The performance was not enhanced significantly when an integrated knowledge graph including these three datasets was used.ConclusionWe showed that the knowledge embeddings are powerful predictors and comparable to current state-of-the-art methods for inferring new DDIs. We addressed the evaluation biases by introducing drug-wise and pairwise disjoint test classes. Although the performance scores for drug-wise and pairwise disjoint seem to be low, the results can be considered to be realistic in predicting the interactions for drugs with limited interaction information.

Adaptive knowledge subgraph ensemble for robust and trustworthy knowledge graph completion

Knowledge graph (KG) embedding approaches are widely used to infer underlying missing facts based on intrinsic structure information. However, the presence of noisy facts in automatically extracted or crowdsourcing KGs significantly reduces the reliability of various embedding learners. In this paper, we thoroughly study the underlying reasons for the performance drop in dealing with noisy knowledge graphs, and we propose an ensemble framework, Adaptive Knowledge Subgraph Ensemble (AKSE), to enhance the robustness and trust of knowledge graph completion. By employing an effective knowledge subgraph extraction approach to re-sample the sub-components from the original knowledge graph, AKSE generates different representations for learning diversified base learners (e.g., TransE and DistMult), which substantially alleviates the noise effect of KG embedding. All embedding learners are integrated into a unified framework to reduce generalization errors via our simple or adaptive weighting schemes, where the weight is allocated based on each individual learner’s prediction capacity. Experimental results show that the robustness of our ensemble framework outperforms exiting knowledge graph embedding approaches on manually injected noise as well as inherent noisy extracted KGs.