Improving Abstractive Document Summarization with Salient Information Modeling

Yongjian You,Tianyi Liu,Wenmian Yang,Weijia Jia

doi:10.18653/v1/p19-1205

Abstract

Comprehensive document encoding and salient information selection are two major difficulties for generating summaries with adequate salient information. To tackle the above difficulties, we propose a Transformer-based encoder-decoder framework with two novel extensions for abstractive document summarization. Specifically, (1) to encode the documents comprehensively, we design a focus-attention mechanism and incorporate it into the encoder. This mechanism models a Gaussian focal bias on attention scores to enhance the perception of local context, which contributes to producing salient and informative summaries. (2) To distinguish salient information precisely, we design an independent saliency-selection network which manages the information flow from encoder to decoder. This network effectively reduces the influences of secondary information on the generated summaries. Experimental results on the popular CNN/Daily Mail benchmark demonstrate that our model outperforms other state-of-the-art baselines on the ROUGE metrics.

Highlights

Document summarization is a fundamental task of natural language generation which condenses the given documents and generates fluent summaries with salient information automatically
We propose the Extended Transformer model for Abstractive Document Summarization (ETADS) to tackle the above issues
We design the focusattention mechanism to improve the capability of capturing the local context information and further encode the document comprehensively

Summary

Introduction

Document summarization is a fundamental task of natural language generation which condenses the given documents and generates fluent summaries with salient information automatically. Documents: a [duke student] has [admitted to hanging a noose made of rope] from a tree near a student union , [university officials] said thursday . The [student was identified during an investigation] by campus police and the office of student affairs and admitted to placing the noose on the tree early wednesday , the university said . The noose , made of rope , was discovered on campus about 2 a.m. ing improvements have been achieved recently (Li et al, 2018c; Kryscinski et al, 2018), there are still many problems are not studied well, such as the incompetence of salient information modeling. The most essential prerequisite for a practical document summarization model is that the generated summaries should contain adequate salient information of the original documents. Previous seq2seq models are still incapable of achieving convincing performance, which are restricted by the following two difficulties

Results

Discussion

Conclusion