Bert-based Siamese Network for Semantic Similarity

Abstract

As a key technology in the field of natural language processing, text matching is widely used in many tasks such as question answering systems, search, recommendation, and advertising. Aiming at the insufficient feature extraction capabilities of traditional Siamese-based text matching methods, a fast matching method based on large-scale pre-training model BSSM (BERT based Siamese Semantic Model) is proposed. The proposed method uses a pre-training model to encode two texts separately, which interact the representation vectors of the two texts to obtain attention weights and generate new representation vectors, so that the new and old representation vectors can be pooled and aggregated, and finally two representation vectors are concatenated in some strategy and sent to the prediction layer for prediction. Experimental results on the AFQMC and LCQMC datasets show that the proposed method not only improves the accuracy rate, but also improves the computational efficiency.