Toward Analyzing the Diversity of Extractive Summaries

Abstract

As the amount of text generated across the internet continues to increase, developing methods for processing that text to glean valuable insights is paramount. Automatic text summarization is one such method that aims to provide a concise and representative summary of input text, allowing users access to the most salient points from a large amount of textual data. However, in working with these summaries, especially those generated from social media data, questions arise about not only the quality of a summary, but also its ability to reflect the diversity of user perspectives. This work examines the quality of summaries with regards to dialect-diversity, as measured for human-written summaries as well as for those generated automatically. Specifically, in this work, we perform an extensive analysis on a dialect-diverse Twitter dataset, DivSumm. Our analysis suggests that humans typically write fairly diverse summaries. In addition, we also note that automatic clustering algorithms generate fairly well-representative clusters. Given these insights we propose a novel clustering-based approach for generating extractive summaries from dialect-diverse social media data. Our approach generates superior summaries than baseline methods when evaluated via ROUGE metrics.