Modeling coherence by ordering paragraphs using pointer networks

Abstract

Coherence is a distinctive feature in well-written documents. One method to study coherence is to analyze how sentences are ordered in a document. In Multi-document Summarization, sentences from different sources need to be ordered. Cluster-based ordering algorithms aim to study various themes or topics that are present in a set of sentences. After the clusters of sentences have been identified, sentences are ordered within each cluster in isolation. One challenge that remains is to order these clusters or paragraphs to obtain a coherent ordering of information. Inspired by the success of deep neural networks in several NLP tasks, we propose an RNN-based encoder–decoder system to predict order for a given set of loose clusters or paragraphs. Universal Sentence Encoder (USE) is used to encode paragraphs into high dimensional embeddings, which are then fed into an LSTM encoder and consecutively passed to a pointer network, which finally outputs the paragraph order. Since Wikipedia is a source of well- structured articles, it is used to generate multiple datasets. Based on our experimental results, the proposed model satisfactorily outperforms the baseline model across multiple datasets. We observe a two-fold increase in Kendall’s tau values for the final paragraph orderings.