Knowledge Graph Embedding Based on Adaptive Negative Sampling

Abstract

Knowledge graph embedding aims at embedding entities and relations in a knowledge graph into a continuous, dense, low-dimensional and real-valued vector space. Among various embedding models appeared in recent years, translation-based models such as TransE, TransH and TransR achieve state-of-the-art performance. However, in these models, negative triples used for training phase are generated by replacing each positive entity in positive triples with negative entities from the entity set with the same probability; as a result, a large number of invalid negative triples will be generated and used in the training process. In this paper, a method named adaptive negative sampling (ANS) is proposed to generate valid negative triples. In this method, it first divided all the entities into a number of groups which consist of similar entities by some clustering algorithms such as K-Means. Then, corresponding to each positive triple, the head entity was replaced by a negative entity from the cluster in which the head entity was located and the tail entity was replaced in a similar approach. As a result, it generated a set of high-quality negative triples which benefit for improving the effectiveness of embedding models. The ANS method was combined with the TransE model and the resulted model was named as TransE-ANS. Experimental results show that TransE-ANS achieves significant improvement in the link prediction task.