RTRL: Relation-aware Transformer with Reinforcement Learning for Deep Question Generation

Abstract

Deep question generation is an important yet challenging NLP task where a multi-step reasoning chain connecting multiple documents is required to answer the questions. In this paper, we propose a Relation-aware Transformer with Reinforcement Learning (RTRL) to improve the performance of deep question generation. Specifically, RTRL first utilizes the relation-aware Transformer to capture both the explicit linguistic and implicit semantic relations for better text representation. Besides, RTRL utilizes the sentence-level reward in a reinforcement learning framework to deal with the metric inconsistency between training and evaluation, and thus improve evaluation metrics with reference questions. Experimental results on the HotpotQA dataset show that RTRL outperforms baselines, highlighting its effectiveness in generating deep questions.