Auto-regressive extractive summarization with replacement

Abstract

Auto-regressive extractive summarization approaches determine sentence extraction probability conditioning on previous decisions by maintaining a partial summary representation. Despite its popularity, the framework has two main drawbacks: 1) the partial summary representation is irresolutely denoted by a weighted summation of all the processed sentences without any filtering, resulting in a noisy representation and degrading the effectiveness of extracting subsequent sentences; 2) earlier sentences are biased towards a higher extraction probability due to the sequential nature of sequence tagging. To address these two problems, we propose the Auto-regressive Extractive Summarization with Replacement (AES-Rep), a novel auto-regressive extractive summarization model. In particular, the AES-Rep model consists of two main modules: the extraction decision module that determines whether a sentence should be extracted, and the replacement locater module that enables extracted deficient sentences to be replaced with latter sentences by comparing their expressiveness with respect to the main idea of the document. These modules update the partial summary with explicit actions using elaborated multidimensional guidance. We conduct extensive experiments on the benchmark CNN and DailyMail datasets. Experimental results show that AES-Rep can achieve better performance compared with various strong baselines in terms of multiple ROUGE metrics.