Generation of Coherent Multi-Sentence Texts with a Coherence Mechanism

Abstract

Automatic generation of long texts containing multiple sentences has many applications in the field of Natural Language Processing (NLP) including question answering, machine translation, and paraphrase generation, etc. However, in terms of readability, the long texts generated by machines are not comparable to those organized by human beings. Through statistics, we observed that human-organized texts generally have a special property: one or more of the words (particularly nouns and pronouns) appeared in one sentence will reappear in the next one in the same or a different form. This repetition of words in consecutive sentences can greatly improve the readability. Based on this observation, we propose CMST, a deep neural network model for generating Coherent Multi-Sentence Texts. CMST explicitly incorporates a training strategy of coherence mechanism to evaluate the repetition of words in consecutive sentences. We evaluate the performance of the CMST on the CNN/Daily Mail dataset. The experimental results show that, compared with the baseline models, CMST not only improves the readability of the generated texts, but achieves higher METEOR and ROUGE values.