MS-Pointer Network: Abstractive Text Summary Based on Multi-Head Self-Attention

Abstract

Abstractive text summarization plays an important role in the field of natural language processing. However, the abstractive text summary adopts deep learning research method to predict words often appears semantic inaccuracy and repetition and so on. at the present stage, in order to solve the problem that semantic inaccuracy, we propose an MS-Pointer Network that based on the multi-head self-attention mechanism, which a multi-head self-attention mechanism is introduced in the basic encoder-decoder model. Since multi-head self-attention can combine input words into the encoder-decoder arbitrarily, and given a higher weight of these words that combination of the semantics, thereby achieving the purpose of enhancing the semantic features of the text, so that the abstractive text summary is more semantically structured, And the multi-head self-attention mechanism add the position information of the input text, which can enhance the semantic representation of the text. At the same time, in order to solve the problem of out of vocabulary, a pointer network is introduced on the seqtoseq with a multi-head attention mechanism. The model is referred to as MS-Pointer Network. We used CNN/Daily Mail and Gigaword datasets to validate our model, and uses the ROUGE metric to measure model. Experiments have shown that abstractive text summaries generated using the multi-head self-attention mechanism outperforming current open state-of-the-art two points averagely.