Abstract
Recent relation extraction models’ architecture are evolved from the shallow neural networks to natural language model, such as convolutional neural networks or recurrent neural networks to Bert. However, these methods did not consider the semantic information in the sequence or the distance dependence problem, the internal semantic information may contain the useful knowledge which can help relation classification. Focus on these problems, this paper proposed a BERT-based relation classification method. Compare with the existing Bert-based architecture, the proposed model can obtain the internal semantic information between entity pair and solve the distance semantic dependence better. The pre-trained BERT model after fine tuning is used in this paper to abstract the semantic representation of sequence, then adopt the piecewise convolution to obtain semantic information which influence the extraction results. Compare with the existing methods, the proposed method can achieve a better accuracy on relational extraction task because of the internal semantic information extracted in the sequence. While, the generalization ability is still a problem that cannot be ignored, and the numbers of the relationships are difference between different categories. In this paper, the focal loss function is adopted to solve this problem by assigning a heavy weight to less number or hard classify categories. Finally, comparing with the existing methods, the F1 metric of the proposed method can reach a superior result 89.95% on the SemEval-2010 Task 8 dataset.
Highlights
With the advent of the intelligent era, information extraction technology is gradually used to process the massive text data on the Internet and digital platform
In this paper, we focus on the extract precision and data balance, and propose an extraction method based on pre-trained Bert with piecewise convolution and focal loss
The proposed relation extraction model is a sequential model, the sentences are encoded by BERT firstly, and the piecewise convolution extracts the features from the remaining segments of corpus
Summary
With the advent of the intelligent era, information extraction technology is gradually used to process the massive text data on the Internet and digital platform. In the processing of relation extraction, the existing deep learning methods are still not rational, which cannot extract essential semantic features. In this paper, we focus on the extract precision and data balance, and propose an extraction method based on pre-trained Bert with piecewise convolution and focal loss. The COTYPE model proposed by [14] and the residual network proposed by [15] both enhance the relation extraction effect These methods are based on supervised learning, which can solve the problem of wrong label and error propagation in traditional methods. The BERT model which has proven to be very effective for improving many natural language processing tasks [16], and focus on the feature extraction of remaining segments, the piecewise convolution has better performance in multi-classification task.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
