Single-document Summarization Research Articles

Comprehensive Survey of Abstractive Text Summarization Techniques

Text summarization using pre-trained encoders has become a crucial technique for efficiently managing large volumes of text data. The rise of automatic summarization systems addresses the need to process ever-increasing data while meeting user-specific requirements. Recent scientific research highlights significant advancements in abstractive summarization, with a particular focus on neural network-based methods. A detailed review of various neural network models for abstractive summarization identifies five key components essential to their design: encoder-decoder architecture, mechanisms, training strategies and optimization algorithms, dataset selection, and evaluation metrics. Each of these elements is pivotal in enhancing the summarization process. This study aims to provide a thorough understanding of the latest developments in neural network-based abstractive summarization models, offering insights into the evolving field and underscoring the associated challenges. Qualitative analysis using a concept matrix reveals common design trends in contemporary neural abstractive summarization systems. Notably, BERT-based encoder-decoder models have emerged as leading innovations, representing the most recent progress in the field. Based on the insights from this review, the study recommends integrating pre-trained language models with neural network techniques to achieve optimal performance in abstractive summarization tasks. As the volume of online information continues to surge, the field of automatic text summarization has garnered significant attention within the Natural Language Processing (NLP) community. Spanning over five decades, researchers have approached this problem from diverse angles, exploring various domains and employing a multitude of paradigms. This survey aims to delve into some of the most pertinent methodologies, focusing on both single-document and multiple-document summarization techniques, with a particular emphasis on empirical methods and extractive approaches. Additionally, the survey explores promising strategies that target specific intricacies of the summarization task. Notably, considerable attention is dedicated to the automatic evaluation of summarization systems, recognizing its pivotal role in guiding future research endeavors.

Features in extractive supervised single-document summarization: case of Persian news

AbstractText summarization has been one of the most challenging areas of research in NLP. Much effort has been made to overcome this challenge by using either abstractive or extractive methods. Extractive methods are preferable due to their simplicity compared with the more elaborate abstractive methods. In extractive supervised single-document approaches, the system will not generate sentences. Instead, via supervised learning, it learns how to score sentences within the document based on some textual features and subsequently selects those with the highest rank. Therefore, the core objective is ranking, which enormously depends on the document structure and context. These dependencies have been unnoticed by many state-of-the-art solutions. In this work, document-related features such as topic and relative length are integrated into the vectors of every sentence to enhance the quality of summaries. Our experiment results show that the system takes contextual and structural patterns into account, which will increase the precision of the learned model. Consequently, our method will produce more comprehensive and concise summaries.

Supervised weight learning-based PSO framework for single document extractive summarization

The need for automatic text summarization is natural: there is a huge volume of information available online, which prompts for a widespread interest in extracting relevant information in a concise and understandable manner. Here, automated text summarization has been treated as an extractive single-document summarization problem in the proposed system. To solve this problem, a particle swarm optimisation (PSO) algorithm-based approach is suggested, with the goal of producing good summaries in terms of content coverage, informativeness, and readability.This paper introduces XSumm-PSO: a new approach based on PSO optimization technique in a supervised manner for extractive summarization. Further, this paper also contributes a new feature “incorrect word” that captures misspelled words in the candidate sentences. This feature is combined with nine existing features used by proposed model to generate error free summaries. As a result, the proposed XSumm-PSO framework produces superior performance achieving improvements of +2.7%, +0.8%, and +0.8% for ROUGE-1, ROUGE-2, and ROUGE-L scores, respectively, on DUC 2002 dataset, over state-of-the-art techniques. The corresponding improvements on the CNN/DailyMail dataset are +0.97%, +0.25%, and +0.49%.We also performed sample t-test, showing the proposed approach is statistically consistent across various runs.

Optimizing Multidocument Summarization by Blending Reinforcement Learning Policies

We consider extractive summarization within a cluster of related texts (multi-document summarization). Unlike single-document summarization, redundancy is particularly important because sentences across related documents might convey overlapping information. Thus, sentence extraction in such a setting is difficult because one will need to determine which pieces of information are relevant while avoiding unnecessary repetitiveness. To solve this difficult problem, we propose a novel reinforcement learning based method <inline-formula><tex-math notation="LaTeX">$\mathbf{PoBRL}$</tex-math></inline-formula> (<b>Po</b>licy <b>B</b>lending with maximal marginal relevance and <b>R</b>einforcement <b>L</b>earning) for solving multi-document summarization. PoBRL jointly optimizes over the following objectives necessary for a high-quality summary: importance, relevance, and length. Our strategy decouples this multi-objective optimization into different sub-problems that can be solved individually by reinforcement learning. Utilizing PoBRL, we then blend each learned policies to produce a summary that is a concise and a complete representation of the original input. Our empirical analysis shows high performance on several multi-document datasets. Human evaluation also shows that our method produces high-quality output.

Graph-based extractive text summarization method for Hausa text.

Automatic text summarization is one of the most promising solutions to the ever-growing challenges of textual data as it produces a shorter version of the original document with fewer bytes, but the same information as the original document. Despite the advancements in automatic text summarization research, research involving the development of automatic text summarization methods for documents written in Hausa, a Chadic language widely spoken in West Africa by approximately 150,000,000 people as either their first or second language, is still in early stages of development. This study proposes a novel graph-based extractive single-document summarization method for Hausa text by modifying the existing PageRank algorithm using the normalized common bigrams count between adjacent sentences as the initial vertex score. The proposed method is evaluated using a primarily collected Hausa summarization evaluation dataset comprising of 113 Hausa news articles on ROUGE evaluation toolkits. The proposed approach outperformed the standard methods using the same datasets. It outperformed the TextRank method by 2.1%, LexRank by 12.3%, centroid-based method by 19.5%, and BM25 method by 17.4%.

CovSumm: an unsupervised transformer-cum-graph-based hybrid document summarization model for CORD-19.

The number of research articles published on COVID-19 has dramatically increased since the outbreak of the pandemic in November 2019. This absurd rate of productivity in research articles leads to information overload. It has increasingly become urgent for researchers and medical associations to stay up to date on the latest COVID-19 studies. To address information overload in COVID-19 scientific literature, the study presents a novel hybrid model named CovSumm, an unsupervised graph-based hybrid approach for single-document summarization, that is evaluated on the CORD-19 dataset. We have tested the proposed methodology on the scientific papers in the database dated from January 1, 2021 to December 31, 2021, consisting of 840 documents in total. The proposed text summarization is a hybrid of two distinctive extractive approaches (1) GenCompareSum (transformer-based approach) and (2) TextRank (graph-based approach). The sum of scores generated by both methods is used to rank the sentences for generating the summary. On the CORD-19, the recall-oriented understudy for gisting evaluation (ROUGE) score metric is used to compare the performance of the CovSumm model with various state-of-the-art techniques. The proposed method achieved the highest scores of ROUGE-1: 40.14%, ROUGE-2: 13.25%, and ROUGE-L: 36.32%. The proposed hybrid approach shows improved performance on the CORD-19 dataset when compared to existing unsupervised text summarization methods.

Special issue on International Conference on Computing and Communication Networks

Special issue on International Conference on Computing and Communication Networks

Single document text summarization addressed with a cat swarm optimization approach.

The availability of a tremendous amount of online information bringing about a broad interest in extracting relevant information in a compact and meaningful way, prompted the need for automatic text summarization. Hence, in the proposed system, the automated text summarization has been considered as an extractive single-document summarization problem, and a Cat Swarm Optimization (CSO) algorithm-based approach is proposed to solve it, whose objective is to generate good summaries in terms of content coverage, informative, anti-redundancy, and readability. In this work, input documents are pre-processed first. Then the cat population is initialized, where each individual (cat) in a binary vector is randomly initialized in the search space, considering the constraint. The objective function is then formulated considering different sentence quality measures. The Best Cat Memory Pool (BCMP) is initialized based on the objective function score. After that, individuals are randomly distributed for position updating to perform seeking/tracing mode operations based on the mixture ratio in each iteration. BCMP is also updated accordingly. Finally, an optimal individual is chosen to generate the summary after the last iteration. DUC-2001 and DUC-2002 data sets and ROUGE measures are used for system evaluation, and the obtained results are compared with the various state-of-the-art methods. We have achieved approximately 25% and 5% improvement on ROUGE-1 and ROUGE-2 scores on the datasets over the best existing method mentioned in this paper, revealing the proposed method’s superiority. The proposed system is also evaluated considering the generational distance, CPU processing time, cohesion, and readability factor, reflecting that the system-generated summaries are readable, concise, relevant, and fast. We have also conducted a two-sample t-test, and one-way ANOVA test showing the proposed approach is statistically significant.

Extractive single-document summarization using adaptive binary constrained multi-objective differential evaluation

Extractive single-document summarization using adaptive binary constrained multi-objective differential evaluation

Single Document Summarization Using BertSum and Pointer Generator Network

Single Document Summarization Using BertSum and Pointer Generator Network

A Quantum-Inspired Genetic Algorithm for Extractive Text Summarization

Automatic text summarization has recently become a key instrument for reducing the huge quantity of textual data. In this paper, the authors propose a quantum-inspired genetic algorithm (QGA) for extractive single-document summarization. The QGA is used inside a totally automated system as an optimizer to search for the best combination of sentences to be put in the final summary. The presented approach is compared with 11 reference methods including supervised and unsupervised summarization techniques. They have evaluated the performances of the proposed approach on the DUC 2001 and DUC 2002 datasets using the ROUGE-1 and ROUGE-2 evaluation metrics. The obtained results show that the proposal can compete with other state-of-the-art methods. It is ranked first out of 12, outperforming all other algorithms.

Investigation of Pre-Trained Bidirectional Encoder Representations from Transformers Checkpoints for Indonesian Abstractive Text Summarization

Text summarization aims to reduce text by removing less useful information to obtain information quickly and precisely. In Indonesian abstractive text summarization, the research mostly focuses on multi-document summarization which methods will not work optimally in single-document summarization. As the public summarization datasets and works in English are focusing on single-document summarization, this study emphasized on Indonesian single-document summarization. Abstractive text summarization studies in English frequently use Bidirectional Encoder Representations from Transformers (BERT), and since Indonesian BERT checkpoint is available, it was employed in this study. This study investigated the use of Indonesian BERT in abstractive text summarization on the IndoSum dataset using the BERTSum model. The investigation proceeded by using various combinations of model encoders, model embedding sizes, and model decoders. Evaluation results showed that models with more embedding size and used Generative Pre-Training (GPT)-like decoder could improve the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score and BERTScore of the model results.

Single Document Summarization Based on Grey Wolf Optimization

The amazing growth of online services has caused an information explosion issue. Text summarisation is condensing the text into a small version and preserving its overall concept. Text summarisation is an important way to extract significant information from documents and offer that information to the user in an abbreviated form while preserving its major content. For human beings, it is very difficult to summarise large documents. To do this, this paper uses some sentence features and word features. These features assign scores to all the sentences. In this paper, we combine these features by Grey Wolf Optimiser (GWO). Optimisation of features gives better results than using individual features. This is the first attempt to show the performance of GWO for Persian text summarisation. The proposed method is compared with the genetic algorithm and the evolutionary strategy. The results show that our model will be useful in this research area. Keywords: Text summarisation, genetic algorithm, sentence, score function, evolutionary strategy.

Automatic Arabic Text Summarization Using Analogical Proportions

Automatic text summarization is the process of generating or extracting a brief representation of an input text. There are several algorithms for extractive summarization in the literature tested by using English and other languages datasets; however, only few extractive Arabic summarizers exist due to the lack of large collection in Arabic language. This paper proposes and assesses new extractive single-document summarization approaches based on analogical proportions which are statements of the form “a is to b as c is to d”. The goal is to study the capability of analogical proportions to represent the relationship between documents and their corresponding summaries. For this purpose, we suggest two algorithms to quantify the relevance/irrelevance of an extracted keyword from the input text, to build its summary. In the first algorithm, the analogical proportion representing this relationship is limited to check the existence/non-existence of the keyword in any document or summary in a binary way without considering keyword frequency in the text, whereas the analogical proportion of the second algorithm considers this frequency. We have assessed and compared these two algorithms with some language-independent summarizers (LexRank, TextRank, Luhn and LSA (Latent Semantic Analysis)) using our large corpus ANT (Arabic News Texts) and a small test collection EASC (Essex Arabic Summaries Corpus) by computing ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and BLEU (BiLingual Evaluation Understudy) metrics. The best-achieved results are ROUGE-1 = 0.96 and BLEU-1 = 0.65 corresponding to educational documents from EASC collection which outperform the best LexRank algorithm. The proposed algorithms are also compared with three other Arabic extractive summarizers, using EASC collection, and show better results in terms of ROUGE-1 = 0.75 and BLEU-1 = 0.47 for the first algorithm, and ROUGE-1 = 0.74 and BLEU-1 = 0.49 for the second one. Experimental results show the interest of analogical proportions for text summarization. In particular, analogical summarizers significantly outperform three among four language-independent summarizers in the case of BLEU-1 for ANT collection and they are not significantly outperformed by any other summarizer in the case of EASC collection.

Persian Automatic Text Summarization Based on Named Entity Recognition

In this paper, we propose an unsupervised method for summarizing Farsi texts based on our neural named entity recognition (NER) system. This method consists of three phases: training a supervised NER model, recognizing named entities of the text, and generating a summary. The proposed method is an unsupervised extractive single-document summarization method. Although the proposed method is language independent, we focus on Farsi text summarization in this work. Firstly, we produce a word embedding based on Hamshahri2 corpus. Secondly, we train a neural network on Arman NER corpus. Then, the proposed algorithm ranks the sentences of the text based on the named entities in each sentence and produces the summary. Finally, the proposed method is evaluated on Pasokh single-document data set using the ROUGE evaluation measure. Without using any handcrafted features, our proposed method achieves state-of-the-art results. We compared our unsupervised method with the best supervised Farsi methods, and we achieved an overall improvement of ROUGE-2 recall score of 10.2%.

Event-based summarization of news articles

In recent years, with the increase of available digital information on the Web, the time needed to find relevant information is also increased. Therefore, to reduce the time spent on searching, research on automatic text summarization has gained importance. The proposed summarization process is based on event extraction methods and is called an event-based extractive single-document summarization. In this method, the important features of event extraction and summarization methods are analyzed and combined together to extract the summaries from single-source news documents. Among the tested features, six features are found to be the most effective in constructing good summaries. The constructed summaries are tested on benchmark Document Understanding Conferences 2001 and 2002 datasets, and the results outperformed most of the other well-known summarization methods.

Marathi Text Analysis using Unsupervised Learning and Word Cloud

Managing a large number of textual documents is a critical and significant task and supports many applications ranging from information retrieval to clustering search engine results. Marathi is one of the oldest of the regional languages in the Indo-Aryan language family, dating from about AD 1000. Abundance of Marathi literature has generated a big corpus and need of summarization of information. The objective of this study is to overcome the scalability problem while managing the documents and summarize the Marathi corpus by extracting tokens. The work is better in terms of scalability and supports the consistent quality of cluster for incremental data set. Most of the past and contemporary research works have targeted English corpus document management. Marathi corpus has been mostly exploited by the researchers for exploring stemming, single-document summarization and classifier design on Marathi corpus. Implementing unsupervised learning on the Marathi corpus for summarization of multiple documents through Word Cloud is still an untouched area. Technically speaking, the current work is an application of TF-IDF, cosine-based document similarity measures and cluster dendrograms, in addition to various other Natural Language Processing (NLP) activities. Entropy and precision are used to evaluate the experiments carried on different datasets and results prove the robustness of the proposed approach for Marathi Corpus.

A novel extractive text summarization system with self-organizing map clustering and entity recognition

Extractive text summarization yields the sensitive parts of the document by neglecting the irrelevant and redundant information. In this paper, we propose a new strategy for extractive single-document summarization in Malayalam. Initially, entity recognition is done, followed by relevance analysis is made based on some context-aware features. The scored sentences are then clustered using self-organizing maps (SOM) and from these clusters, relevant sentences are extracted out based on the proposed algorithm. Both theoretical and practical evaluations are done to analyze the implemented system. In theoretical evaluation, gradient calculations of relevance equations are used to know that which of these sentence scoring features are contributing more. The relevance equation is optimized with the help of Lagrange’s multiplier. The complexity analysis of the proposed algorithms is also performed. In practical evaluation, the system compared with online and offline summarizers upon metrics like precision, recall, and F-measure. The system is tested through a non-clustering approach also in order to analyze the impact of clustering used in our work. Some existing strategies like question game evaluation, sentence rank evaluation, and keyword association are also done to evaluate the different parameters like the relevance of sentences, important entity words, etc.

What is this Article about? Extreme Summarization with Topic-aware Convolutional Neural Networks

We introduce "extreme summarization," a new single-document summarization task which aims at creating a short, one-sentence news summary answering the question "What is the article about?". We argue that extreme summarization, by nature, is not amenable to extractive strategies and requires an abstractive modeling approach. In the hope of driving research on this task further: (a) we collect a real-world, large scale dataset by harvesting online articles from the British Broadcasting Corporation (BBC); and (b) propose a novel abstractive model which is conditioned on the article's topics and based entirely on convolutional neural networks. We demonstrate experimentally that this architecture captures long-range dependencies in a document and recognizes pertinent content, outperforming an oracle extractive system and state-of-the-art abstractive approaches when evaluated automatically and by humans on the extreme summarization dataset.

An unsupervised constrained optimization approach to compressive summarization

Automatic summarization is typically aimed at selecting as much information as possible from text documents using a predefined number of words. Extracting complete sentences into a summary is not an optimal way to solve this problem due to redundant information that is contained in some sentences. Removing the redundant information and compiling a summary from compressed sentences should provide a much more accurate result. Major challenges of compressive approaches include the cost of creating large summarization corpora for training the supervised methods, the linguistic quality of compressed sentences, the coverage of the relevant content, and the time complexity of the compression procedure. In this work, we attempt to address these challenges by proposing an unsupervised polynomial-time compressive summarization algorithm. The proposed algorithm iteratively removes redundant parts from original sentences. It uses constituency-based parse trees and hand-crafted rules for generating elementary discourse units (EDUs) from their subtrees (standing for phrases) and selects ones with a sufficient tree gain. We define a parse tree gain as a weighted function of its node weights, which can be computed by any extractive summarization model capable of assigning importance weights to terms. The results of automatic evaluations on a single-document summarization task confirm that the proposed sentence compression procedure helps to avoid redundant information in the generated summaries. Furthermore, the results of human evaluations confirm that the linguistic quality—in terms of readability and coherency—is preserved in the compressed summaries while improving their coverage. However, the same evaluations show that compression in general harms the grammatical correctness of compressed sentences though, in most cases, this effect is not significant for the proposed compression procedure.