Bayesian active summarization

Abstract

Bayesian Active Learning has had significant impact to various NLP problems, but nevertheless its application to text summarization has been explored very little. We introduce Bayesian Active Summarization (BAS), as a method of combining active learning methods with state-of-the-art summarization models. Our findings suggest that BAS achieves better and more robust performance, compared to random selection, particularly for small and very small data annotation budgets. More specifically, applying BAS with a summarization model like PEGASUS we managed to reach 95% of the performance of the fully trained model, using less than 150 training samples. Furthermore, we have reduced standard deviation by 18% compared to the conventional random selection strategy. Using BAS we showcase it is possible to leverage large summarization models to effectively solve real-world problems with very limited annotated data.