HierarT: Multi-hop temporal knowledge graph forecasting with hierarchical reinforcement learning

Abstract

Temporal knowledge graph (TKG) multi-hop reasoning is a dominant approach that infers the target entity by walking along the path connecting entities and relations. However, in most TKGs, there are multiple relations related to an identical entity and multiple tail entities for an identical pair of head entity and relation. This characteristic leads to a significant amount of redundant data in the binary action space which is a collection of binary tuples composed of relations and entities, thereby impeding model inference, i.e., action space explosion. We propose a method to address this issue that dismantles the reasoning process into a relation level for relation reasoning and an entity level for entity reasoning. In addition, hybrid time encoding is proposed to enhance the utilization of the timestamp information in the reasoning process. Moreover, K-means-based reward shaping is proposed to alleviate the issue of sparse reward matrices by comparing the pre-clustered labels of both the predicted entity and the target entity. A text transformer module is used to deal with the limitation of a single information modality by integrating the text information of TKG into model reasoning. Our method is evaluated using four benchmark datasets, and the results verify its superiority over state-of-the-art baselines.