Multi-hop temporal knowledge graph reasoning with multi-agent reinforcement learning

Abstract

Knowledge graph (KG) is a key component of artificial intelligence. In recent years, many large-scale knowledge graphs have been produced and put into practical applications. At present, researchers have proposed many methods to reason facts that do not exist in knowledge graphs using the existing information. However, most traditional reasoning methods lack interpretability, and cannot get the reasoning paths. Therefore, the multi-hop path reasoning method of knowledge graph has gradually become a hot spot. This method finds the next relations through the reinforcement learning agent, gets the whole path, and scores the path. Although these multi-hop path reasoning methods make the reasoning method interpretable, these multi-hop path reasoning methods focus on the choice of relations, ignoring the importance of entities in the path reasoning process. Moreover, with the development of the Temporal Knowledge Graph (TKG), traditional multi-hop path reasoning methods cannot effectively process time information. To solve these problems, a multi-hop path reasoning method of temporal knowledge graph based on multi-agent reinforcement learning is proposed, which is named MA-TPath. MA-TPath uses two agents to perform relation selection and entity selection iteratively. Meanwhile, considering the diversity of temporal reasoning paths, we propose a new type of reward function. In MA-TPath, two agents employ the Long Short-Term Memory Networks (LSTM) to capture current results from the environment and output the corresponding action vectors to the environment through activation functions. Experimentally, MA-TPath outperforms all existing models in 4 out of 4 indicators on the YAGO dataset, 3 out of 4 indicators on the GDETL-5 dataset and the ICEWS-250 dataset, and 1 out of 4 indicators on the ICEWS05–15 dataset. Analysis of the temporal reasoning path indicates that MA-TPath not only surpasses other state-of-the-art reasoning methods over four public temporal knowledge graph datasets but provides rationality for results.