Deep purified feature mining model for joint named entity recognition and relation extraction

Abstract

Table filling based joint named entity recognition and relation extraction task aims to share representation of subtasks in a table to extract structured knowledge. However, most of existing studies need additional labels and dedicated deep neural networks to learn shared representation, imposing heavy burdens to decoders. More seriously, almost all these models suffer from feature confusion problem, failing to capture purified task-specific features from shared representation to perform subtasks. To address these challenging problems, in this paper we propose a novel and effective Deep puRified fEAture Mining (DREAM) model for joint named entity recognition and relation extraction task, which can automatically capture purified task-specific features to improve the classification performance of subtasks. Specifically, unlike introducing additional labels or dedicated network architectures, we design a new lightweight shared representation learning (LSRL) module by the plainest labels of joint task and thus encodes context by the hybrid convolutional neural networks. Afterwards, a task-aware information bottleneck (TIB) module is proposed to explore the relation between the mutual information of the joint distribution of each subtask and its task-specific features. With the above two modules well obtain shared representation and purified task-specific features, the satisfactory classification results of both subtasks can be guaranteed. Experiment results show that the proposed model is highly effective, obtaining the promising results on three different benchmarks: CoNNL04 (general text), ADE (biomedical text) and SciERC (scientific text). For example, DREAM respectively achieves F1-scores of 78.18%, 80.28% and 44.60% in performing the relation extraction subtask on the CoNNL04, ADE and SciERC datasets. The promising performance indicates that the proposed model can be applied to many practical applications such as biomedical information extraction. The source code is publicly available at https://github.com/SWT-AITeam/DREAM.