Siamese network text similarity calculation with multi-head self-attention mechanism

Abstract

Calculating text similarity is the key to natural language processing. Existing text similarity calculation methods can not fully extract deep semantic information. And the ability to calculate the similarity of long texts is limited. Aiming at the shortcoming of existing text similarity calculation methods, a Siamese network based on multi-head self-attention mechanism is proposed. This method uses the Siamese model based on bidirectional GRU to accurately extract the semantic information of the context. Then learning deep semantic information of long text by adding multi head self-attention mechanism. Experimental results show that Siamese network with multi-head self-attention mechanism can learn the deep semantic information of long text in the SICK dataset. Compared with other text similarity calculation methods, the correlation coefficient of the proposed method is significantly improved. The effect of processing long text is improved.