Attention-based exploitation and exploration strategy for multi-hop knowledge graph reasoning

Abstract

Knowledge Graphs (KGs) typically suffer from incompleteness. A popular approach to solve this problem is multi-hop reasoning through Reinforcement Learning (RL) framework, which is an explainable and effective model to predict missing links in KGs. However, many previous RL-based models use the scoring function of the pre-trained Knowledge Graph Embedding (KGE) methods as the reward function, which will lead to the performance of the model be limited to the KGE methods. Moreover, the agent may reason a meaningless path if it cannot distinguish the different aspect of each entity in different triples. To solve both problems, we propose a multi-hop reasoning model named Ae2KGR, by applying two novel strategies: attention-based exploitation and attention-based exploration. The attention-based exploitation strategy incorporates historical and query information with the neighborhood of the current entity, and dynamically updates the current state during reasoning process to assign different semantic information to the entity to distinguish different triples. The attention-based exploration strategy designs a novel policy network and reward function to dynamically make decisions based on the constantly changing state. Extensive experiments on three standard datasets confirm the effectiveness of our innovations, and the performance of our proposed Ae2KGR is significantly improved compared to the state-of-the-art methods.