Entity Embeddings Research Articles

Routing User-Interest Markov Tree for Scalable Personalized Knowledge-Aware Recommendation.

To facilitate more accurate and explainable recommendation, it is crucial to incorporate side information into user-item interactions. Recently, knowledge graph (KG) has attracted much attention in a variety of domains due to its fruitful facts and abundant relations. However, the expanding scale of real-world data graphs poses severe challenges. In general, most existing KG-based algorithms adopt exhaustively hop-by-hop enumeration strategy to search all the possible relational paths, this manner involves extremely high-cost computations and is not scalable with the increase of hop numbers. To overcome these difficulties, in this article, we propose an end-to-end framework Knowledge-tree-routed UseR-Interest Trajectories Network (KURIT-Net). KURIT-Net employs the user-interest Markov trees (UIMTs) to reconfigure a recommendation-based KG, striking a good balance for routing knowledge between short-distance and long-distance relations between entities. Each tree starts from the preferred items for a user and routes the association reasoning paths along the entities in the KG to provide a human-readable explanation for model prediction. KURIT-Net receives entity and relation trajectory embedding (RTE) and fully reflects potential interests of each user by summarizing all reasoning paths in a KG. Besides, we conduct extensive experiments on six public datasets, our KURIT-Net significantly outperforms state-of-the-art approaches and shows its interpretability in recommendation.

QDN: A Quadruplet Distributor Network for Temporal Knowledge Graph Completion.

Temporal knowledge graph completion (TKGC) is an extension of the traditional static knowledge graph completion (SKGC) by introducing the timestamp. The existing TKGC methods generally translate the original quadruplet to the form of the triplet by integrating the timestamp into the entity/relation, and then use SKGC methods to infer the missing item. However, such an integrating operation largely limits the expressive ability of temporal information and ignores the semantic loss problem due to the fact that entities, relations, and timestamps are located in different spaces. In this article, we propose a novel TKGC method called the quadruplet distributor network (QDN), which independently models the embeddings of entities, relations, and timestamps in their specific spaces to fully capture the semantics and builds the QD to facilitate the information aggregation and distribution among them. Furthermore, the interaction among entities, relations, and timestamps is integrated using a novel quadruplet-specific decoder, which stretches the third-order tensor to the fourth-order to satisfy the TKGC criterion. Equally important, we design a novel temporal regularization that imposes a smoothness constraint on temporal embeddings. Experimental results show that the proposed method outperforms the existing state-of-the-art TKGC methods. The source codes of this article are available at https://github.com/QDN for Temporal Knowledge Graph Completion.git.

A multi-entity reinforced main path analysis: Heterogeneous network embedding considering knowledge proximity

Main path analysis (MPA) is an important approach in detecting the trajectory of knowledge diffusion in a specific research domain. Previous studies always focus on citation-based relationships, overlooking other structural forms in citation network. This study introduces a multi-entity reinforced MPA model by constructing a knowledge graph from paper metadata, including citations, authors, journals, and keywords. We construct heterogeneous network to reveal relationships among various entities. Different knowledge graph embedding models are employed to train the network, thereby obtaining entity and relation embeddings. The cosine similarity algorithm is adopted to measure the knowledge proximity between these embeddings. We take the Internet of Thing domain as an example to verify the performance of the multi-entity reinforced MPA through both quantitative and qualitative analysis. Our findings indicate that the adjusted MPA exhibits stronger topic relevance, demonstrating the effectiveness of the method in capturing complex knowledge relationships.

A link prediction method for Chinese financial event knowledge graph based on graph attention networks and convolutional neural networks

Finance is a knowledge-intensive domain in nature, with its data containing a significant amount of interconnected information. Constructing a financial knowledge graph is an important application for transforming financial text/web content into machine-readable data. However, the complexity of Chinese financial knowledge and the dynamic and evolving nature of Chinese financial data often lead to incomplete knowledge graphs. To address this challenge, we propose a novel link prediction method for Chinese financial event knowledge graph based on Graph Attention Networks and Convolutional Neural Networks. Our method begins with the construction of the foundational Chinese financial event knowledge graph using a relational triple extraction module integrated with a large language model framework, along with a Prompting with Iterative Verification (PiVe) module for validation. To enhance the completeness of the knowledge graph, we introduce an encoder-decoder framework, where a graph attention network with joint embeddings of financial event entities and relations acts as the encoder, while a Convolutional Knowledge Base embedding model (ConvKB) serves as the decoder. This framework effectively aggregates crucial neighbor information and captures global relationships among entity and relation embeddings. Extensive comparative experiments demonstrate the utility and accuracy of this method, ultimately enabling the effective completion of Chinese financial event knowledge graphs.

Knowledge-reinforced explainable next basket recommendation

The next basket recommendation task aims to predict the items in the user’s next basket by modeling the user’s basket sequence. Existing next basket recommendations focus on improving recommendation performance, and most of these methods are black-box models, ignoring the importance of providing explanations to improve user satisfaction. Furthermore, most next basket recommendation methods are designed for consumer users, and few methods are proposed for business user characteristics. To address the above problems, we propose a Knowledge Reinforced Explainable Next Basket Recommendation (KRE-NBR). Specifically, we construct a basket-based knowledge graph and obtain pretrained embeddings of entities that contain rich information of the knowledge graph. To obtain high-quality user predictive vectors, we fuse user pretrained embeddings, user basket sequence level embeddings, and user repurchase embeddings. One highlight of the user repurchase embeddings is that they are able to model business user repurchase behavior. To make the results of next basket recommendations explainable, we use reinforcement learning for path reasoning to find the items recommended in the next basket and generate recommendation explanations at the same time. To the best of our knowledge, this is the first work to provide recommendation explanations for next basket recommendations. Extensive experiments on real datasets show that the recommendation performance of our proposed approach outperforms several state-of-the-art baselines.

On Entity Embeddings for Ordinal Features as Representation Learning in Recurrence Prediction of Urothelial Bladder Cancer.

Urothelial Bladder Cancer (UBC) is a common cancer with a high risk of recurrence, which is influenced by the TNM classification, grading, age, and other factors. Recent studies demonstrate reliable and accurate recurrence prediction using Machine Learning (ML) algorithms and even outperform traditional approaches. However, most ML algorithms cannot process categorical input features, which must first be encoded into numerical values. Choosing the appropriate encoding strategy has a significant impact on the prediction quality. We investigate the impact of encoding strategies for ordinal features in the prediction quality of ML algorithms. We compare three different encoding strategies namely one-hot, ordinal, and entity embedding in predicting the 2-year recurrence in UBC patients using an artificial neural network. We use ordered categorical and numerical data of UBC patients provided by the Cancer Registry Rhineland-Palatinate. We show superior prediction quality using entity embedding encoding with 84.6% precision, an overall accuracy of 73.8%, and 68.9% AUC on testing data over 100 epochs after 30 runs compared to one-hot and ordinal encoding. We confirm the superiority of entity embedding encoding as it could provide a more detailed and accurate representation of ordinal features in numerical scales. This can lead to enhanced generalizability, resulting in significantly improved prediction quality.

Anomalous behavior detection based on optimized graph embedding representation in social networks

Anomalous behaviors in social networks can lead to privacy leaks and the spread of false information. In this paper, we propose an anomalous behavior detection method based on optimized graph embedding representation. Specifically, the user behavior logs are first extracted into a social network user behavior temporal knowledge graph, based on which the graph embedding representation method is used to transform the network topology and temporal information in the user behavior knowledge graph into structural embedding vectors and temporal information embedding vectors, and the hybrid attention mechanism is used to merge the two types of vectors to obtain the final entity embedding to complete the prediction and complementation of the temporal knowledge graph of user behavior. We use graph neural networks, which use the temporal information of user behaviors as a time constraint and capture both user behavioral and semantic information. It converts the two parts of information into vectors for concatenation and linear transformation to obtain a comprehensive representation vector of the whole subgraph, as well as joint deep learning models to evaluate abnormal behavior. Finally, we perform experiments on the Yelp dataset to validate that our method achieves a 9.56% improvement in the F1-score.

GFedKG: GNN-based federated embedding model for knowledge graph completion

Knowledge Graph (KG) completion through reasoning over KGs is one of the most concerned issue in research around knowledge graphs. Current work focuses more on studying individual KG, while a series of related multi-source KGs are more common in real applications. In order to effectively exert their complementary roles in completing each KG under the premise of ensuring data privacy, federated learning becomes the focus of attention. However, existing methods ignore the explicit complementarity among KGs in terms of structural information. And they generally align embeddings of overlapping instances across clients, which constrains flexibility of each client model. Based on this, we propose the first GNN-based federated embedding framework for multiple knowledge graph completion, GFedKG. It focuses on the complementary role among KGs in topology structures and adaptively generates different aggregated neighboring features for entities in distinct clients. Such treatment achieves some consistency among clients while retaining the flexibility of each client model to characterize respective KG. Moreover, entity embeddings that represent the features of entities are better privacy-preserved since neighboring embeddings are transmitted instead of them. And with the aim of realizing more refined and accurate aggregation of neighboring features from multiple clients, we innovatively propose a context-aware graph neural network, along with the context-enhanced evaluation function. Homophily neighbors are also taken into account to cope with the low-homophily aspect of KGs. Extensive experiments demonstrate the advantages of GFedKG.

TS-align: A temporal similarity-aware entity alignment model for temporal knowledge graphs

Entity Alignment (EA) is a crucial step in knowledge graph fusion, aiming to match equivalent entity pairs across different knowledge graphs (KGs). In recent years, Temporal Knowledge Graphs (TKGs) have extended static KGs by introducing timestamps, providing a new perspective for EA. However, while temporal information is critical in TKGs, the EA approach to existing TKG does not fully utilize this critical resource. To solve this problem, we propose a temporal similarity-aware EA model for TKGs (TS-Align). The model innovatively uses the overlap degree of multiple temporal points and temporal intervals of entities to calculate the temporal similarity matrix, and deeply digs and integrates the temporal and relational information in TKGs through a well-designed encoder-decoder architecture. To verify the effectiveness of TS-Align, we conducted experiments on four standard datasets and compared it to other advanced methods. The experimental results show that the accuracy of TS-Align on EA tasks is significantly improved compared with other methods. Specifically, TS-Align is about 0.4 –1.8 % higher on the Hit@1 metric compared to the most advanced time-aware models. This is about 7.4 –17.6 % higher than the most advanced GCN-based models. This is about 20.8 –42.3 % higher than the most advanced translation-based models. In addition, through ablation experiments, we verify that the proposed temporal similarity matrix, relational embedding, and entity embedding contribute to the significant improvement in performance.

Multi-Source Information Graph Embedding with Ensemble Learning for Link Prediction

Link prediction is a key technique for connecting entities and relationships in a graph reasoning field. It leverages known information about the graph structure data to predict missing factual information. Previous studies have either focused on the semantic representation of a single triplet or on the graph structure data built on triples. The former ignores the association between different triples, and the latter ignores the true meaning of the node itself. Furthermore, common graph-structured datasets inherently face challenges, such as missing information and incompleteness. In light of this challenge, we present a novel model called Multi-source Information Graph Embedding with Ensemble Learning for Link Prediction (EMGE), which can effectively improve the reasoning of link prediction. Ensemble learning is systematically applied throughout the model training process. At the data level, this approach enhances entity embeddings by integrating structured graph information and unstructured textual data as multi-source information inputs. The fusion of these inputs is effectively addressed by introducing an attention mechanism. During the training phase, the principle of ensemble learning is employed to extract semantic features from multiple neural network models, facilitating the interaction of enriched information. To ensure effective model learning, a novel loss function based on contrastive learning is devised, effectively minimizing the discrepancy between predicted values and the ground truth. Moreover, to enhance the semantic representation of graph nodes in link prediction, two rules are introduced during the aggregation of graph structure information. These rules incorporate the concept of spreading activation, enabling a more comprehensive understanding of the relationships between nodes and edges in the graph. During the testing phase, the EMGE model is validated on three datasets, including WN18RR, FB15k-237, and a private Chinese financial dataset. The experimental results demonstrate a reduction in the mean rank (MR) by 0.2 times, an improvement in the mean reciprocal rank (MRR) by 5.9%, and an increase in the Hit@1 by 12.9% compared to the baseline model.

C-KGE: Curriculum learning-based Knowledge Graph Embedding

Knowledge graph embedding (KGE) aims to embed entities and relations in knowledge graphs (KGs) into a continuous, low-dimensional vector space. It has been shown as an effective tool for integrating knowledge graphs to improve various intelligent applications, such as question answering and information extraction. However, previous KGE models ignore the hidden natural order of knowledge learning on learning the embeddings of entities and relations, leaving room for improvement in their performance. Inspired by the easy-to-hard pattern used in human knowledge learning, this paper proposes a Curriculum learning-based KGE (C-KGE) model, which learns the embeddings of entities and relations from “basic knowledge” to “domain knowledge”. Specifically, a seed set representing the basic knowledge and several knowledge subsets are identified from KG. Then, entity overlap is employed to score the learning difficulty of each subset. Finally, C-KGE trains the entities and relations in each subset according to the learning difficulty score of each subset. C-KGE leverages trained embeddings of the seed set as prior knowledge and learns knowledge subsets iteratively to transfer knowledge between the seed set and subsets, smoothing the learning process of knowledge facts. Experimental results on real-world datasets demonstrate that the proposed model achieves improved embedding performances as well as reducing training time. Our codes and data will be released later.

CatEmbed: A Machine-Learned Representation Obtained via Categorical Entity Embedding for Predicting Adsorption and Reaction Energies on Bimetallic Alloy Surfaces.

Machine-learning models for predicting adsorption energies on metallic surfaces often rely on basic elemental properties and electronic and geometric descriptors. Here, we apply categorical entity embedding, a featurization method inspired by natural language processing techniques, to predict adsorption energies on bimetallic alloy surfaces using categorical descriptors. Using this method, we develop a machine-learned representation from categorical descriptors (e.g., surface composition, adsorbate type, and site type) of the slab/adsorbate complex. By combining this representation with numerical features (e.g., slab metal stoichiometric ratios), we create the CatEmbed representation. Remarkably, decision tree models trained using CatEmbed, which includes no explicit geometric information, achieve a Mean Absolute Error (MAE) of 0.12 eV. Additionally, we extend this technique to predict reaction energies on bimetallic surfaces, creating the CatEmbed-React representation, which achieves an MAE of 0.08 eV. These findings highlight the effectiveness of categorical entity embedding for predicting adsorption and reaction energies on bimetallic alloy surfaces.

TODEAR: Promoting explainable TKG reasoning through temporal offset enhanced dynamic embedding and adaptive reinforcement learning

Recently, Reinforcement Learning (RL) is utilized in Temporal Knowledge Graph (TKG) reasoning to generate analyzable reasoning paths, which achieves explainable reasoning on queries. The process of generating high-quality reasoning paths is facing two major challenges. The first is to construct an efficient entity embedding model from the complex temporal dependencies among entities. The second is to design a fine-grained reward mechanism for the reasoner based on the deep semantics of exploration paths. Motivated by the two challenges, a TKG reasoning framework based on Temporal Offset Enhanced Dynamic Embedding and Adaptive Reinforcement Learning (TODEAR) is proposed in this paper. Firstly, a temporal offset enhanced dynamic embedding model with a distance scoring function is designed to fully exploit the complex temporal dependencies among entities. In order to capture the evolutionary patterns of historical facts, it encodes both the relational structures of the TKG and the temporal offsets between events and queries. Then, a fine-grained adaptive reward mechanism is designed to optimize the reasoner. It generates real-time rewards by analyzing the logic and semantics of exploration paths to mitigate the adverse effects of sparse rewards. Extensive experiments on four benchmark datasets show that TODEAR significantly outperforms the state-of-the-art methods.

GEUKE: A geographic entities uniformly explicit knowledge embedding model

AbstractKnowledge embedding for geographic knowledge graphs can effectively improve computational efficiency and provide support for knowledge reasoning, knowledge answering and other applications of knowledge graphs. To maintain a more comprehensive understanding of spatial features through knowledge embedding, it is crucial to integrate the representation and computation of various entity types, encompassing points, lines, and polygons. This article proposes a geographic entities uniformly explicit knowledge embedding model (GEUKE). In GEUKE, spatial data of point, line, and polygon‐type geographic entities are expressed in the form of subgraphs, and space embedding is generated using a SubGNN‐based uniform spatial feature encoder. GEUKE improves the energy function in TransE to train spatial feature‐based embedding and structural‐based embedding of geographic entities into a unified vector space. Experimental results show that GEUKE has higher performance than TransE, TransH, TransD, and TransE‐GDR on link prediction and triple classification task. Within the spatial feature embedding process, GEUKE effectively preserves the inherent features of entities, encompassing location, neighborhood, and structural attributes, while simultaneously ensuring a coherent spatial data representation across all three entity types: points, lines, and polygons. By maintaining the spatial features of geographic entities and their interrelations, this capability unleashes the full potential of applications such as knowledge reasoning and geospatial question answering in a manner that is conducive to diverse geospatial scenarios.

Integration of Semantic and Topological Structural Similarity Comparison for Entity Alignment without Pre-Training

Entity alignment (EA) is a critical task in integrating diverse knowledge graph (KG) data and plays a central role in data-driven AI applications. Traditional EA approaches rely on entity embeddings, but their effectiveness is limited by scarce KG input data and representation learning techniques. Large language models have shown promise, but face challenges such as high hardware requirements, large model sizes and computational inefficiency, which limit their applicability. To overcome these limitations, we propose an entity-alignment model that compares the similarity between entities by capturing both semantic and topological information to enable the alignment of entities with high similarity. First, we analyze descriptive information to quantify semantic similarity, including individual features such as types and attributes. Then, for topological analysis, we introduce four conditions based on graph connectivity and structural patterns to determine subgraph similarity within three hops of the entity’s neighborhood, thereby improving accuracy. Finally, we integrate semantic and topological similarity using a weighted approach that considers dataset features. Our model requires no pre-training and is designed to be compact and generalizable to different datasets. Experimental results on four standard EA datasets validate the effectiveness of our proposed model.

Knowledge Graph Completion Using a Pre-Trained Language Model Based on Categorical Information and Multi-Layer Residual Attention

Knowledge graph completion (KGC) utilizes known knowledge graph triples to infer and predict missing knowledge, making it one of the research hotspots in the field of knowledge graphs. There are still limitations in generating high-quality entity embeddings and fully understanding the contextual information of entities and relationships. To overcome these challenges, this paper introduces a novel pre-trained language model-based method for knowledge graph completion that significantly enhances the quality of entity embeddings by integrating entity categorical information with textual descriptions. Additionally, this method employs an innovative multi-layer residual attention network in combination with PLMs, deepening the understanding of the joint contextual information of entities and relationships. Experimental results on the FB15k-237 and WN18RR datasets demonstrate that our proposed model significantly outperforms existing baseline models in link prediction tasks.

Social Perception with Graph Attention Network for Recommendation

Recommendation systems are designed to uncover users’ potential preferences and make recommendations. However, they often face challenges such as data sparsity and the cold start problem. Although the introduction of knowledge graphs has partially addressed the issue of data sparsity, the challenge of cold start has not been effectively resolved. In this paper, a novel approach called Social Perception with Graph Attention Network (SPGAT) for Recommendation is proposed. In SPGAT, we aim to leverage social perception to solve the cold start effectively for more accurate recommendations. The approach utilizes a multi-layer graph attention network to aggregate user preference features from collaborative knowledge graphs and social perception graphs. By analyzing the social network of a new user, associated friend users can be identified. The interaction data of these friend users is then provided as side information to recommend to the new user. To handle one-to-many and many-to-many relations, we introduce the TransD graph embedding model, which maps different types of relations and entities to different spaces. To optimize the proposed SPGAT, self-adversarial negative sampling is utilized to implement entity and relation embedding and generate negative samples. Experimental results demonstrate that SPGAT has achieved superior performance compared to several advanced methods.

MPASL: multi-perspective learning knowledge graph attention network for synthetic lethality prediction in human cancer.

Synthetic lethality (SL) is widely used to discover the anti-cancer drug targets. However, the identification of SL interactions through wet experiments is costly and inefficient. Hence, the development of efficient and high-accuracy computational methods for SL interactions prediction is of great significance. In this study, we propose MPASL, a multi-perspective learning knowledge graph attention network to enhance synthetic lethality prediction. MPASL utilizes knowledge graph hierarchy propagation to explore multi-source neighbor nodes related to genes. The knowledge graph ripple propagation expands gene representations through existing gene SL preference sets. MPASL can learn the gene representations from both gene-entity perspective and entity-entity perspective. Specifically, based on the aggregation method, we learn to obtain gene-oriented entity embeddings. Then, the gene representations are refined by comparing the various layer-wise neighborhood features of entities using the discrepancy contrastive technique. Finally, the learned gene representation is applied in SL prediction. Experimental results demonstrated that MPASL outperforms several state-of-the-art methods. Additionally, case studies have validated the effectiveness of MPASL in identifying SL interactions between genes.

Knowledge Graph Reasoning via Dynamic Subgraph Attention with Low Resource Computation

Knowledge graphs (KGs) suffer from inherent incompleteness, which has spurred research into knowledge graph reasoning (KGR), i.e., ways to infer missing facts based on existing triples. The most prevalent approaches employ a static mechanism for entity representation across the entire graph, sharing entity embeddings for different query relations. Nevertheless, these static entity embeddings fail to capture specific semantics for various query scenarios, resulting in inaccurate entity representations and susceptibility to reasoning errors. Furthermore, the whole graph learning style requires substantial computational resources, especially since KGs are often of large-scale. Consequently, these methods are impractical in low-resource environments. To address these issues, we devise a framework called dynamic subgraph attention (DSA), which learns dynamic entity embeddings for different query relations across subgraphs. In our approach, by utilizing multi-hop path history information obtained through path-based learning, we guide a dynamic attention mechanism to generate dynamic entity embeddings for different query relations. To ensure the semantic information of entities, the embedding-based and path-based learning are jointly trained for KGR. Additionally, the proposed dynamic aggregation mechanism operates on subgraphs, resulting in a remarkable 6–7 times resource conservation compared to GPU computations. Empirical experiments further demonstrate that DSA outperforms the current methods significantly on three benchmark datasets.

M[formula omitted]ixKG: Mixing for harder negative samples in knowledge graph

Knowledge graph embedding (KGE) involves mapping entities and relations to low-dimensional dense embeddings, enabling a wide range of real-world applications. The mapping is achieved via distinguishing the positive and negative triplets in knowledge graphs. Therefore, how to design high-quality negative triplets is critical in the effectiveness of KEG models. Existing KGE models face challenges in generating high-quality negative triplets. Some models employ simple static distributions, i.e. uniform or Bernoulli distribution, and it is difficult for these methods to be trained distinguishably because of the sampled uninformative negative triplets. Furthermore, current methods are confined to constructing negative triplets from existing entities within the knowledge graph, limiting their ability to explore harder negatives. We introduce a novel mixing strategy in knowledge graphs called M2ixKG. M2ixKG adopts mixing operation in generating harder negative samples from two aspects: one is mixing among the heads and tails in triplets with the same relation to strengthen the robustness and generalization of the entity embeddings; the other is mixing the negatives with high scores to generate harder negatives. Our experiments, utilizing three datasets and four classical score functions, highlight the exceptional performance of M2ixKG in comparison to previous negative sampling algorithms.