Abstract
As a core task of natural language processing and information retrieval, automatic text summarization is widely applied in many fields. There are two existing methods for text summarization task at present: abstractive and extractive. On this basis we propose a novel hybrid model of extractive-abstractive to combine BERT (Bidirectional Encoder Representations from Transformers) word embedding with reinforcement learning. Firstly, we convert the human-written abstractive summaries to the ground truth labels. Secondly, we use BERT word embedding as text representation and pre-train two sub-models respectively. Finally, the extraction network and the abstraction network are bridged by reinforcement learning. To verify the performance of the model, we compare it with the current popular automatic text summary model on the CNN/Daily Mail dataset, and use the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metrics as the evaluation method. Extensive experimental results show that the accuracy of the model is improved obviously.
Highlights
Text summarization is a task of compressing long text into short one keeping up the central idea
400the tokens of thenetwork vanilla input different amountsamounts of data, of when article tokens inputthe of input the vanilla network is truncated, which causes loss of information, the input of the new network is the key is truncated, which causes loss of information, the input of the new network is the key sentences sentences from aforementioned model;the fourth, word embedding of twoare models are from aforementioned extractionextraction model; fourth, wordthe embedding of two models different, different, the word2vec is used for the vanilla pointer-generator network, the is used in our the word2vec is used for the vanilla pointer-generator network, the BERT is used in our abstractive abstractive addition, thetokenizer
BERT has achieved the most advanced performance in many natural language processing (NLP) tasks, but few works combine it with the extract model and abstract model for text summarization by the strategy gradient of reinforcement learning
Summary
Text summarization is a task of compressing long text into short one keeping up the central idea. When summarizing a very long text, the extractive approach is too simple and the readability is poor, and the abstract method of compressing a long input sequence with a single fixed-length vector may cause the information loss, neither of them could perform the long text summary better. Proposed a new model for the long text summary, which abstracts the summary by using a deep communication agent, first of all, dividing the long input text into multiple agents encoders, and generating the summary through a unified decoder These methods have achieved good results, due to the limitation of specific data sets and the small amount of data, their word embedding effect is not obvious and the semantic features of a text cannot be fully obtained.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
