A New Hierarchical Reinforcement Learning Framework for Relation Extraction

Abstract

Relationships extraction from unstructured text is essential for building wholesale reusable information in social computing applications, such as knowledge base construction, intelligent understanding, and biomedical text mining. Most existing approaches only ascertain the relationship type after all entities are identified, so the interaction between the relationship type and entity extraction is not entirely modeled. Recently, HRL-RL is proposed to deal with relationship extraction by adopting joint extraction paradigms in the fralanguage of hierarchical reinforcement learning. Specifically, HRL-RL recognizes relationship indicators through a high-level reinforcement learning subtask and predicts the participating entities for the relationship through a low-level RL subtask. However, HRL-RL uses the Monte Carlo sampling method to learn low-level subtasks for entity extraction, which leads to inconsistencies between the learned entities and the relations detected in the high-level tasks because of high variance estimation. In this paper, we propose a new hierarchical reinforcement learning framework to reinforce interactions between entity extraction and relationship types. Specifically, in the low-level subtask, our algorithm uses an advantage function to learn low-level strategies and updates the value estimate of a current entity state through bootstrap operations, thereby reducing the variance of the model and speeding up learning.