Learning From Summaries: Supporting e-Learning Activities by Means of Document Summarization

Abstract

e-Learning platforms allow users with different skills to explore large collections of electronic documents and annotate them with notes and highlights. Generating summaries of these document collections is potentially useful for gaining insights into teaching materials. However, most existing summarizers are general purpose. Thus, they do not consider neither annotations nor user skills during the document summarization process. This paper studies the application of a state-of-the-art summarization system, namely, the itemset-based summarizer (ItemSum), in an e-learning context. The summarizer produces an ordered sequence of key phrases extracted from a teaching document. The aim of this paper is threefold: 1) evaluate the usefulness of the generated summaries for supporting individual and collective learning activities in a real context; 2) understand to what extent document highlights, annotations, and user skill levels can be used to drive the summarization process; and 3) generate multiple summaries of the same document tailored to users with different skill levels. To accomplish task 1), a crowd-sourcing experience of evaluation of the generated summaries was conducted by involving the students of a B.S. course given by a technical university. The results show that the automatically generated summaries reflect, to a large extent, the students’ expectations. Hence, they can be useful for supporting learning activities in university-level computer science courses. To address task 2), three extended versions of the ItemSum summarizer, driven by highlights, annotations, and user skill levels, respectively, have been proposed, and their performance improvements with respect to the baseline version have been validated on benchmark documents. Finally, to accomplish task 3), multiple summaries of the same benchmark documents have been generated by considering only the annotations made by the users with a different skill level. The results confirm that the summary content reflects the level of expertise of the targeted users.