Joint Extraction of Entities and Relations by Adversarial Training and Mixup Data Augmentation

Abstract

Joint extraction of entities and relations is an important task for building a knowledge graph and information extraction. However, small interference in the text can greatly change the semantics of words and sentences in natural language, thereby affecting the model’s prediction results. To solve the sensitivity of text to noise, in this paper, we present a method, called BERT of Adversarial Training and Sentence Mixup Data Augmentation (BERT-AT-SMDA), which use the BERT pre-training model to obtain the embedding vector representation of the text and then to incorporate the FGM adversarial training strategy into the fine-tuning of Bert to construct adversarial samples. Both the adversarial sample and the original sample into the encoder to training. Then, to prevent the over-fitting phenomenon and the additional interference caused by the instability of adversarial training, we introduce the mixup data augmentation method after obtaining the hidden layer of the encoder output, which performs a linear interpolation between the output hidden layer. In addition, we also study the effect of linear interpolation at different positions on the model’s joint extraction of entities and relations. We conduct experiments on a public dataset produced by the distant supervision method. The experimental results show that our method is effective for the joint extraction method based on tagging scheme. Not only improves the accuracy of entity and relation extraction, and ensures the robustness and generalization of the model.