Temporal Knowledge Graph Research Articles

TS-align: A temporal similarity-aware entity alignment model for temporal knowledge graphs

Entity Alignment (EA) is a crucial step in knowledge graph fusion, aiming to match equivalent entity pairs across different knowledge graphs (KGs). In recent years, Temporal Knowledge Graphs (TKGs) have extended static KGs by introducing timestamps, providing a new perspective for EA. However, while temporal information is critical in TKGs, the EA approach to existing TKG does not fully utilize this critical resource. To solve this problem, we propose a temporal similarity-aware EA model for TKGs (TS-Align). The model innovatively uses the overlap degree of multiple temporal points and temporal intervals of entities to calculate the temporal similarity matrix, and deeply digs and integrates the temporal and relational information in TKGs through a well-designed encoder-decoder architecture. To verify the effectiveness of TS-Align, we conducted experiments on four standard datasets and compared it to other advanced methods. The experimental results show that the accuracy of TS-Align on EA tasks is significantly improved compared with other methods. Specifically, TS-Align is about 0.4 –1.8 % higher on the Hit@1 metric compared to the most advanced time-aware models. This is about 7.4 –17.6 % higher than the most advanced GCN-based models. This is about 20.8 –42.3 % higher than the most advanced translation-based models. In addition, through ablation experiments, we verify that the proposed temporal similarity matrix, relational embedding, and entity embedding contribute to the significant improvement in performance.

PLEASING: Exploring the historical and potential events for temporal knowledge graph reasoning

Temporal Knowledge Graphs (TKGs) enable effective modeling of knowledge dynamics and event evolution, facilitating deeper insights and analysis into temporal information. Recently, extrapolation of TKG reasoning has attracted great significance due to its remarkable ability to capture historical correlations and predict future events. Existing studies of extrapolation aim mainly at encoding the structural and temporal semantics based on snapshot sequences, which contain graph aggregators for the association within snapshots and recurrent units for the evolution. However, these methods are limited to modeling long-distance history, as they primarily focus on capturing temporal correlations over shorter periods. Besides, a few approaches rely on compiling historical repetitive statistics of TKGs for predicting future facts. But they often overlook explicit interactions in the graph structure among concurrent events. To address these issues, we propose a PotentiaL concurrEnt Aggregation and contraStive learnING (PLEASING) method for TKG extrapolation. PLEASING is a two-step reasoning framework that effectively leverages the historical and potential features of TKGs. It includes two encoders for historical and global events with an adaptive gated mechanism, acquiring predictions with appropriate weight of the two aspects. Specifically, PLEASING constructs two auxiliary graphs to capture temporal interaction among timestamps and correlations among potential concurrent events, respectively, enabling a holistic investigation of temporal characteristics and future potential possibilities in TKGs. Furthermore, PLEASING incorporates contrastive learning to strengthen its capacity to identify whether queries are related to history. Extensive experiments on seven benchmark datasets demonstrate the state-of-the-art performances of PLEASING and its comprehensive ability to model TKG semantics.

Predicting Future Disorders via Temporal Knowledge Graphs and Medical Ontologies.

Despite the vast potential for insights and value present in Electronic Health Records (EHRs), it is challenging to fully leverage all the available information, particularly that contained in the free-text data written by clinicians describing the health status of patients. The utilization of Named Entity Recognition and Linking tools allows not only for the structuring of information contained within free-text data, but also for the integration with medical ontologies, which may prove highly beneficial for the analysis of patient medical histories with the aim of forecasting future medical outcomes, such as the diagnosis of a new disorder. In this paper, we propose MedTKG, a Temporal Knowledge Graph (TKG) framework that incorporates both the dynamic information of patient clinical histories and the static information of medical ontologies. The TKG is used to model a medical history as a series of snapshots at different points in time, effectively capturing the dynamic nature of the patient's health status, while a static graph is used to model the hierarchies of concepts extracted from domain ontologies. The proposed method aims to predict future disorders by identifying missing objects in the quadruple 〈s, r, ?, t 〉, where s and r denote the patient and the disorder relation type, respectively, and t is the timestamp of the query. The method is evaluated on clinical notes extracted from MIMIC-III and demonstrates the effectiveness of the TKG framework in predicting future disorders and of medical ontologies in improving its performance.

SFTe: Temporal knowledge graphs embedding for future interaction prediction

Interaction prediction is a crucial task in the Social Internet of Things (SIoT), serving diverse applications including social network analysis and recommendation systems. However, the dynamic nature of items, users, and their interactions over time poses challenges in effectively capturing and analyzing these changes. Existing interaction prediction models often overlook the temporal aspect and lack the ability to model multi-relational user-item interactions over time. To address these limitations, in this paper, we propose a Structure, Facticity, and Temporal information preservation embedding model (SFTe) to predict future interaction. Our model leverages the advantages of Temporal Knowledge Graphs (TKGs) that can capture both the multi-relations and evolution. We begin by modeling user-item interactions over time by constructing a Temporal Interaction Knowledge Graph (TIKG). We then employ Structure Embedding (SE), Facticity Embedding (FE), and Temporal Embedding (TE) to capture topological structure, facticity consistency, and temporal dependence, respectively. In SE, we focus on preserving the first-order relationships to capture the topological structure of TIKG. In the FE component, given the distinct nature of SIoT, we introduce an attention mechanism to capture the effect of entities with the same additional information for generating subgraph embeddings. Lastly, TE utilizes recurrent neural networks to model the temporal dependencies among subgraphs and capture the evolving dynamics of the interactions over time. Experimental results on standard future interaction prediction demonstrate the superiority of the SFTe model compared with the state-of-the-art methods. Our model effectively addresses the challenges of time-aware interaction prediction, showcasing the potential of TKGs to enhance prediction performance.

Geometry Interaction Embeddings for Interpolation Temporal Knowledge Graph Completion

Knowledge graphs (KGs) have become a cornerstone for structuring vast amounts of information, enabling sophisticated AI applications across domains. The progression to temporal knowledge graphs (TKGs) introduces time as an essential dimension, allowing for a dynamic representation of entity relationships. Despite their potential, TKGs often suffer from incompleteness, necessitating the development of temporal knowledge graph completion (TKGC) techniques. These methods, particularly focusing on interpolation within the known timeframe, aim to infer missing temporal facts and enhance the predictive capabilities of TKGs. The prevalent reliance on Euclidean space modeling in TKGC methods presents challenges in capturing the complex, hierarchical, and time-varying nature of TKGs. To overcome these limitations, we introduced the attention-based geometry interaction embedding (ATGIE) method, a novel approach that leverages the strengths of multiple geometric spaces, i.e., Euclidean, hyperbolic, and hypersphere, to model the intricacies of TKGs more effectively. ATGIE employs an attention mechanism to dynamically weigh the contributions of different geometric spaces, allowing it to adaptively form reliable spatial structures based on interactive geometric information. This multi-space modeling not only captures the diverse relationships within TKGs but also facilitates a nuanced understanding of how entities and their relationships evolve over time. Through extensive experiments, we demonstrate ATGIE’s superiority in TKGC tasks, showcasing its improvement over existing methods, robustness to noise, and sensitivity to temporal dynamics. The results highlight ATGIE’s potential to advance the state-of-the-art in TKGC, offering a promising direction for research and application in the field.

Learning multi-graph structure for Temporal Knowledge Graph reasoning

Temporal Knowledge Graph (TKG) reasoning aiming at forecasting future events based on historical snapshots distributed over timestamps. Referred to as extrapolation in numerous studies, this aspect of research in KG has garnered significant attention in recent years. Owing to its extreme versatility and variation in spatial and temporal correlations, TKG reasoning presents a challenging task, demanding efficient capture of concurrent structures and evolutional interactions among facts. While existing methods have made strides in this direction, they still fall short of harnessing the diverse forms of intrinsic expressive semantics of TKGs, which encompass entity correlations across multiple timestamps and periodicity of temporal information. This limitation constrains their ability to thoroughly reflect historical dependencies and future trends. In response to these drawbacks, this paper proposes an innovative reasoning approach that focuses on Learning Multi-graph Structure (LMS). Concretely, it comprises three distinct modules concentrating on multiple aspects of graph structure knowledge within TKGs, including concurrent and evolutional patterns along timestamps, query-specific correlations across timestamps, and semantic dependencies of timestamps, which capture TKG features from various perspectives. Besides, LMS incorporates an adaptive gate for merging entity representations both along and across timestamps effectively. Moreover, it integrates timestamp semantics into graph attention calculations and time-aware decoders, in order to impose temporal constraints on events and narrow down prediction scopes with historical statistics. Extensive experimental results on five event-based benchmark datasets demonstrate that LMS outperforms state-of-the-art extrapolation models, indicating the superiority of modeling a multi-graph perspective for TKG reasoning.

HierarT: Multi-hop temporal knowledge graph forecasting with hierarchical reinforcement learning

Temporal knowledge graph (TKG) multi-hop reasoning is a dominant approach that infers the target entity by walking along the path connecting entities and relations. However, in most TKGs, there are multiple relations related to an identical entity and multiple tail entities for an identical pair of head entity and relation. This characteristic leads to a significant amount of redundant data in the binary action space which is a collection of binary tuples composed of relations and entities, thereby impeding model inference, i.e., action space explosion. We propose a method to address this issue that dismantles the reasoning process into a relation level for relation reasoning and an entity level for entity reasoning. In addition, hybrid time encoding is proposed to enhance the utilization of the timestamp information in the reasoning process. Moreover, K-means-based reward shaping is proposed to alleviate the issue of sparse reward matrices by comparing the pre-clustered labels of both the predicted entity and the target entity. A text transformer module is used to deal with the limitation of a single information modality by integrating the text information of TKG into model reasoning. Our method is evaluated using four benchmark datasets, and the results verify its superiority over state-of-the-art baselines.

TODEAR: Promoting explainable TKG reasoning through temporal offset enhanced dynamic embedding and adaptive reinforcement learning

Recently, Reinforcement Learning (RL) is utilized in Temporal Knowledge Graph (TKG) reasoning to generate analyzable reasoning paths, which achieves explainable reasoning on queries. The process of generating high-quality reasoning paths is facing two major challenges. The first is to construct an efficient entity embedding model from the complex temporal dependencies among entities. The second is to design a fine-grained reward mechanism for the reasoner based on the deep semantics of exploration paths. Motivated by the two challenges, a TKG reasoning framework based on Temporal Offset Enhanced Dynamic Embedding and Adaptive Reinforcement Learning (TODEAR) is proposed in this paper. Firstly, a temporal offset enhanced dynamic embedding model with a distance scoring function is designed to fully exploit the complex temporal dependencies among entities. In order to capture the evolutionary patterns of historical facts, it encodes both the relational structures of the TKG and the temporal offsets between events and queries. Then, a fine-grained adaptive reward mechanism is designed to optimize the reasoner. It generates real-time rewards by analyzing the logic and semantics of exploration paths to mitigate the adverse effects of sparse rewards. Extensive experiments on four benchmark datasets show that TODEAR significantly outperforms the state-of-the-art methods.

Temporal Knowledge Graph Reasoning Based on Entity Relationship Similarity Perception

Temporal knowledge graphs (TKGs) are used for dynamically modeling facts in the temporal dimension, and are widely used in various fields. However, existing reasoning models often fail to consider the similarity features between entity relationships and static attributes, making it difficult for them to effectively handle these temporal attributes. Therefore, these models have limitations in dealing with previously invisible entities that appear over time and the implicit associations of static attributes between entities. To address this issue, we propose a temporal knowledge graph reasoning model based on Entity Relationship Similarity Perception, known as ERSP. This model employs the similarity measurement method to capture the similarity features of entity relationships and static attributes, and then fuses these features to generate structural representations. Finally, we provide a decoder with entity relationship representation, static attribute representation, and structural representation information to form a quadruple. Experiments conducted on five common benchmark datasets show that ERSP surpasses the majority of TKG reasoning methods.

Improving static and temporal knowledge graph embedding using affine transformations of entities

To find a suitable embedding for a knowledge graph (KG) remains a big challenge nowadays. By measuring the distance or plausibility of triples and quadruples in static and temporal knowledge graphs, many reliable knowledge graph embedding (KGE) models are proposed. However, these classical models may not be able to represent and infer various relation patterns well, such as TransE cannot represent symmetric relations, DistMult cannot represent inverse relations, RotatE cannot represent multiple relations, etc.. In this paper, we improve the ability of these models to represent various relation patterns by introducing the affine transformation framework. Specifically, we first utilize a set of affine transformations related to each relation or timestamp to operate on entity vectors, and then these transformed vectors can be applied not only to static KGE models, but also to temporal KGE models. The main advantage of using affine transformations is their good geometry properties with interpretability. Our experimental results demonstrate that the proposed intuitive design with affine transformations provides a statistically significant increase in performance with adding a few extra processing steps and keeping the same number of embedding parameters. Taking TransE as an example, we employ the scale transformation (the special case of an affine transformation). Surprisingly, it even outperforms RotatE to some extent on various datasets. We also introduce affine transformations into RotatE, Distmult, ComplEx, TTransE and TComplEx respectively, and experiments demonstrate that affine transformations consistently and significantly improve the performance of state-of-the-art KGE models on both static and temporal knowledge graph benchmarks.

MetaTKG++: Learning evolving factor enhanced meta-knowledge for temporal knowledge graph reasoning

Reasoning over Temporal Knowledge Graphs (TKGs) aims to predict future facts based on the given history. One of the key challenges for prediction is to analyze the evolution process of facts. Most existing works focus on exploring evolutionary information in history to obtain effective temporal embeddings for entities and relations, but they ignore the variation in evolution patterns of facts caused by numerous diverse entities and latent evolving factors, which makes them struggle to adapt to future data with different evolution patterns. Moreover, new entities continue to emerge along with the evolution of facts over time. Since existing models highly rely on historical information to learn embeddings for entities, they perform poorly on such entities with little historical information. To tackle these issues, we propose a novel evolving factor enhanced temporal meta-learner framework for TKG reasoning, MetaTKG++ for brevity. Specifically, we first propose a temporal meta-learner which regards TKG reasoning as many temporal meta-tasks for training. From the training process of each meta-task, the obtained meta-knowledge can guide backbones to adapt to future data exhibiting various evolution patterns and to effectively learn entities with little historical information. Then, we design an Evolving Factor Learning module, which aims to assist backbones in learning evolution patterns by modeling latent evolving factors. Meanwhile, during the training process with the proposed meta-learner, the learnable evolving factor can enhance the meta-knowledge with providing more comprehensive information on learning evolution patterns. Extensive experiments on five widely-used datasets and four backbones demonstrate that our method can greatly improve the performance on TKG prediction.

Multi-hop temporal knowledge graph reasoning with multi-agent reinforcement learning

Knowledge graph (KG) is a key component of artificial intelligence. In recent years, many large-scale knowledge graphs have been produced and put into practical applications. At present, researchers have proposed many methods to reason facts that do not exist in knowledge graphs using the existing information. However, most traditional reasoning methods lack interpretability, and cannot get the reasoning paths. Therefore, the multi-hop path reasoning method of knowledge graph has gradually become a hot spot. This method finds the next relations through the reinforcement learning agent, gets the whole path, and scores the path. Although these multi-hop path reasoning methods make the reasoning method interpretable, these multi-hop path reasoning methods focus on the choice of relations, ignoring the importance of entities in the path reasoning process. Moreover, with the development of the Temporal Knowledge Graph (TKG), traditional multi-hop path reasoning methods cannot effectively process time information. To solve these problems, a multi-hop path reasoning method of temporal knowledge graph based on multi-agent reinforcement learning is proposed, which is named MA-TPath. MA-TPath uses two agents to perform relation selection and entity selection iteratively. Meanwhile, considering the diversity of temporal reasoning paths, we propose a new type of reward function. In MA-TPath, two agents employ the Long Short-Term Memory Networks (LSTM) to capture current results from the environment and output the corresponding action vectors to the environment through activation functions. Experimentally, MA-TPath outperforms all existing models in 4 out of 4 indicators on the YAGO dataset, 3 out of 4 indicators on the GDETL-5 dataset and the ICEWS-250 dataset, and 1 out of 4 indicators on the ICEWS05–15 dataset. Analysis of the temporal reasoning path indicates that MA-TPath not only surpasses other state-of-the-art reasoning methods over four public temporal knowledge graph datasets but provides rationality for results.

An embedding model for temporal knowledge graphs with long and irregular intervals

As the basis of various downstream applications such as automated question answering and knowledge reasoning, Temporal Knowledge Graph (TKG) embedding learning has attracted many interests. Recent works have explored several methods to learn dynamic embedding for TKGs and led to success in many applications, however, they are difficult to be optimized. In order to accurately catch semantic information propagation of entity-to-entity and entity-to-relation, as well as track dynamics of entities and relations in continuous time, we proposed a novel TKG embedding model ODETKGE. Instead of using recurrent neural network to model dynamics of semantic representations in continuous time, we leverage neural ordinary differential equations to model the dynamic evolution, which enable us to capture long-range dependencies between embeddings at different timestamps, track the dynamics of TKGs with irregular intervals and remain computationally efficient. Furthermore, to effectively model semantics of knowledge graph structure, we combine entity-relation collaboratively updated graph convolutional networks and neural ordinary differential equations. Additionally, to acquire important semantics such as creation and disappearance of relations, we utilize a graph transition layer. Comparing with RNN based models, the proposed ODETKGE can use fewer parameters to achieve deeper layers, and make the transformation of representations between two timestamps smoother. Extensive experiments have been executed on real-world datasets and the results verify that ODETKGE is superior than the state-of-the-art models in link forecasting, long-interval link forecasting, irregular-interval link forecasting, ablation study and training time.

Temporal Knowledge Graph for Food Risk Prediction

Temporal Knowledge Graph for Food Risk Prediction

DHyper: A Recurrent Dual Hypergraph Neural Network for Event Prediction in Temporal Knowledge Graphs

Event prediction is a vital and challenging task in temporal knowledge graphs (TKGs), which have played crucial roles in various applications. Recently, many graph neural networks based approaches are proposed to model the graph structure information in TKGs. However, these approaches only construct graphs based on quadruplets and model the pairwise correlation between entities, which fail to capture the high-order correlations among entities. To this end, we propose DHyper, a recurrent Dual Hypergraph neural network for event prediction in TKGs, which simultaneously models the influences of the high-order correlations among both entities and relations. Specifically, a dual hypergraph learning module is proposed to discover the high-order correlations among entities and among relations in a parameterized way. A dual hypergraph message passing network is introduced to perform the information aggregation and representation fusion on the entity hypergraph and the relation hypergraph. Extensive experiments on six real-world datasets demonstrate that DHyper achieves the state-of-the-art performances, outperforming the best baseline by an average of 13.09%, 4.26%, 17.60%, and 18.03% in MRR, Hits@1, Hits@3, and Hits@10, respectively.

Revolutionary Strategy for Depicting Knowledge Graphs with Temporal Attributes

In practical applications, the temporal completeness of knowledge graphs is of great importance. However, previous studies have mostly focused on static knowledge graphs, generally neglecting the dynamic evolutionary properties of facts. Moreover, the unpredictable and limited availability of temporal knowledge graphs, together with the complex temporal dependency patterns, make current models inadequate for effectively describing facts that experience temporal transitions. To better represent the evolution of things over time, we provide a learning technique that uses quaternion rotation to describe temporal knowledge graphs. This technique describes the evolution of entities as a temporal rotation change in quaternion space. Compared to the Ermitian inner product in complex number space, the Hamiltonian product in quaternion space is better at showing how things might be connected. This leads to a learning process that is both more effective and more articulate. Experimental results demonstrate that our learning method significantly outperforms existing methods in capturing the dynamic evolution of temporal knowledge graphs, with improved accuracy and robustness across a range of benchmark datasets.

Embedding-based entity alignment between multi-source temporal knowledge graphs

The goal of entity alignment is to identify entities in two multi-source knowledge graphs (KGs) that represent the same real-world object. Recent researches on multi-source entity alignment mainly concentrate on static KGs. In fact, temporal KGs have become valuable resources for numerous artificial intelligence applications, and entity alignment between multi-source temporal KGs is becoming more and more important. Current entity alignment models cannot support temporal tasks and fail to deal with the attributes with low literal similarity that share the same semantics through attribute embedding. Therefore, we propose a RDF (Resource Description Framework)-based model for representing temporal KGs, and an embedding-based entity alignment method for multi-source temporal KGs. This method computes for the similarity of temporal information and generates aligned attribute pairs in the predicate alignment module. We design an interactive module to make matched attributes and the matched entities help to find each other based on aligned attribute pairs. This module can calculate the similarity of attributes with low literal similarity. After getting the structure similarity of the structure embedding module, the final entity alignment result of temporal KGs is produced by the calculation of a binary linear regression function. Experimental results demonstrate that our proposed model outperforms existing approaches significantly.

On the Evaluation Framework of Comprehensive Trust for Data Interaction in Intermodal Transport

Due to the necessity to realize “building a strong transportation nation”, the construction of intermodal transportation is based on the information resource integration of diverse transport systems. To ensure the data security during the interaction between different transport modes, as well as the effect of data application, the status of entities and data flow in the network should be supervised throughout. Therefore, an evaluation framework of comprehensive trust is proposed in this paper. With feature analysis of transportation big data, a quality assessment is conducted by three-dimensional metric sets, which is considered as a significant factor of trust measurement. Furthermore, a hierarchical trust structure is put forward to assess the trust of entities in different levels, in terms of the static and dynamic evidence. Furthermore, the visualization of a dynamic global information security state is discussed, based on temporal knowledge graphs. As shown in practical application and simulation analysis, this framework can meet the requirements of data security supervision and lay the foundation of further intelligent management. This research is of great significance to improve the data security level in intermodal transport, and to promote the utilization and sharing of public information resources.

Intricate Spatiotemporal Dependency Learning for Temporal Knowledge Graph Reasoning

Knowledge Graph (KG) reasoning has been an interesting topic in recent decades. Most current researches focus on predicting the missing facts for incomplete KG. Nevertheless, Temporal KG (TKG) reasoning, which is to forecast future facts, still faces with a dilemma due to the complex interactions between entities over time. This article proposes a novel intricate Spatiotemporal Dependency learning Network (STDN) based on Graph Convolutional Network (GCN) to capture the underlying correlations of an entity at different timestamps. Specifically, we first learn an adaptive adjacency matrix to depict the direct dependencies from the temporally adjacent facts of an entity, obtaining its previous context embedding. Then, a Spatiotemporal feature Encoding GCN (STE-GCN) is proposed to capture the latent spatiotemporal dependencies of the entity, getting the spatiotemporal embedding. Finally, a time gate unit is used to integrate the previous context embedding and the spatiotemporal embedding at the current timestamp to update the entity evolutional embedding for predicting future facts. STDN could generate the more expressive embeddings for capturing the intricate spatiotemporal dependencies in TKG. Extensive experiments on WIKI, ICEWS14, and ICEWS18 datasets prove our STDN has the advantage over state-of-the-art baselines for the temporal reasoning task.

ConvTKG: A query-aware convolutional neural network-based embedding model for temporal knowledge graph completion

Temporal knowledge graphs (TKGs) are still far from completeness although they have benefited many artificial intelligence applications successfully. To alleviate the incompleteness of TKGs, temporal knowledge graph completion (TKGC) aims to perform link prediction and reason new time-sensitive facts from existing facts. Among the various TKGC methods, temporal knowledge graph embedding (TKGE) methods represent entities, relations, and timestamps in low-dimensional spaces, which are known for their high reasoning performance and robust scalability. Some previous researches on TKGE aim to endow the embedding quadruple with the translational characteristic by modeling the global relationship in the embedding quadruple. However, the weights of same dimensional entries of the embedding quadruple are fixed and unique, which limits the ability to capture diverse global relationships. Besides, most TKGE models view timestamps as a kind of general feature and overlook their interrelations and correlations with the queries. Moreover, the distinct characteristics of an entity as the subject entity and the object entity of facts have not been fully leveraged. In this paper, we propose a query-aware embedding model based on the convolutional neural network (CNN) named ConvTKG to perform TKGE, thus tackling the TKGC task. We design a new temporal information encoder based on the Gated Recurrent Unit (GRU) and attention mechanism to learn the query-aware representation of temporal information. More importantly, we utilize a CNN-based decoder where multiple one-dimensional convolution kernels are operated on the matrix composed of entity embedding, relation embedding, and temporal representation to capture global relationships among them. In addition, we assign two independent vectors for each entity and take advantage of the inverse relation to allow them to be learned dependently. Experiments show that ConvTKG achieves better link prediction performance than previous state-of-the-art baseline methods for TKGC on three benchmark datasets: ICEWS14, ICEWS05-15, and GDELT.