A new Knowledge Inference Approach Based on Multi-headed Attention Mechanism

Abstract

To address the problem of sparse relationships between entities in the current knowledge graph and the increase in complexity and computational effort in knowledge inference, this paper proposes a knowledge inference method based on a multi-headed attention mechanism - the MAtt-BiGRU model. For a given triad, the path information is encoded by a two-way gated recurrent neural network, and the confidence of the model is calculated by aggregating it with the candidate paths with the help of the multi-headed attention mechanism, while the inference ability of the model is enhanced by dynamic negative sample training. Numerical experiments are conducted on the MAtt-BiGRU model using large knowledge graph datasets NELL-995 and FB15k-237. The experimental results show that the proposed MAtt-BiGRU model has better results in Mean Average Precision (MAP) and Mean Reciprocal Rank(MRR) performance evaluation metrics when compared with existing commonly used path ranking algorithms such as Path Ranking Algorithm(PRA) and Path-RNN, and the proposed model can more accurately reason about the relations between entities.