Multi-document Summarization Task Research Articles

TOMDS (Topic-Oriented Multi-Document Summarization): Enabling Personalized Customization of Multi-Document Summaries

In a multi-document summarization task, if the user can decide on the summary topic, the generated summary can better align with the reader’s specific needs and preferences. This paper addresses the issue of overly general content generation by common multi-document summarization models and proposes a topic-oriented multi-document summarization (TOMDS) approach. The method is divided into two stages: extraction and abstraction. During the extractive stage, it primarily identifies and retrieves paragraphs relevant to the designated topic, subsequently sorting them based on their relevance to the topic and forming an initial subset of documents. In the abstractive stage, building upon the transformer architecture, the process includes two parts: encoding and decoding. In the encoding part, we integrated an external discourse parsing module that focuses on both micro-level within-paragraph semantic relationships and macro-level inter-paragraph connections, effectively combining these with the implicit relationships in the source document to produce more enriched semantic features. In the decoding part, we incorporated a topic-aware attention mechanism that dynamically zeroes in on information pertinent to the chosen topic, thus guiding the summary generation process more effectively. The proposed model was primarily evaluated using the standard text summary dataset WikiSum. The experimental results show that our model significantly enhanced the thematic relevance and flexibility of the summaries and improved the accuracy of grammatical and semantic comprehension in the generated summaries.

Chain-of-event prompting for multi-document summarization by large language models

PurposeIn the era of big data, various industries are generating large amounts of text data every day. Simplifying and summarizing these data can effectively serve users and improve efficiency. Recently, zero-shot prompting in large language models (LLMs) has demonstrated remarkable performance on various language tasks. However, generating a very “concise” multi-document summary is a difficult task for it. When conciseness is specified in the zero-shot prompting, the generated multi-document summary still contains some unimportant information, even with the few-shot prompting. This paper aims to propose a LLMs prompting for multi-document summarization task.Design/methodology/approachTo overcome this challenge, the authors propose chain-of-event (CoE) prompting for multi-document summarization (MDS) task. In this prompting, the authors take events as the center and propose a four-step summary reasoning process: specific event extraction; event abstraction and generalization; common event statistics; and summary generation. To further improve the performance of LLMs, the authors extend CoE prompting with the example of summary reasoning.FindingsSummaries generated by CoE prompting are more abstractive, concise and accurate. The authors evaluate the authors’ proposed prompting on two data sets. The experimental results over ChatGLM2-6b show that the authors’ proposed CoE prompting consistently outperforms other typical promptings across all data sets.Originality/valueThis paper proposes CoE prompting to solve MDS tasks by the LLMs. CoE prompting can not only identify the key events but also ensure the conciseness of the summary. By this method, users can access the most relevant and important information quickly, improving their decision-making processes.

Tiered sentence based topic model for multi-document summarization

In this work, a probabilistic two level topic model named Tiered Sentence based Topic Model is proposed which models the document at sentence and word levels and infer hierarchical latent topics for sentences. The proposed model uses two latent variables for the generation of words- a super topic and a subtopic for each sentence of the document, to model word groupings at sentence level. Popular super topics identify general theme of the documents and are used for selecting summary sentences. The model parameters are used for ranking sentences considering sentence importance and topic coverage. Collapsed Gibbs sampling is used for inference and parameter estimation. The proposed model is used to compare with two sentence based topic models- SenLDA and LDCC on query focused multi-document summarization task, over standard DUC2005 dataset using ROUGE-1 and ROUGE-2 precision and recall scores. The proposed model performs better than Latent Dirichlet Allocation and SenLDA but has been outperformed by LDCC.

VoteSumm: A Multi-Document Summarization Scheme Using Influential Nodes of Multilayer Weighted Sentence Network

This work proposes a sentence network-based approach for performing the task of multi-document text summarization. The sentences of the input set of documents are represented by the nodes of the network. Weighted edges are added between the nodes to represent the semantic similarity between the corresponding sentences. The network has a multilayer structure, where each layer corresponds to an individual input document. This helps in effective differentiation between the inter-document and intra-document edges. A hyperparameter, namely layering factor, has been used to alter the strength of inter-document connections through reinforcement or weakening. It is hypothesized that the summary sentence nodes must act as effective information spreaders in the sentence network. Summary generation is performed by identifying the influential nodes of the network using VoteRank scheme. A comparative study with different network measures, such as Weighted Degree, PageRank, Betweenness centrality, and Closeness centrality reveals the efficacy of the proposed VoteSumm technique for multi-document text summarization. Improved performance is observed when an additional pre-processing step of syntactic simplification is applied on the raw text. Performance is further improved when keyword information is included in the simplified texts.

VLSP 2022 Abmusu Task Dataset: A Resource for Vietnamese Abstractive Multi-Document Summarization

The performance of automatic summarization systems has improved significantly with the development of supervised approaches. However, in the Vietnamese abstractive multi-document summarization task, the available datasets are insufficient for training the model. With this motivation, we contribute a new gold standard Vietnamese abstractive multi-document summarization dataset, named Abmusu. Following the collecting and clustering of articles, we have built a hierarchical annotation process to generate summaries, with three roles: annotator, supervisor, and curator. As a result, the dataset contains 600 news clusters formed from 1839 articles and the corresponding human-generated summaries. To the best of our knowledge, Abmusu dataset is the biggest dataset for Vietnamese abstractive multi-document summarization that is freely available for research. Moreover, summaries are more concise, making it challenging to train the summarization models. We also used various summarization baselines to benchmark the Abmusu dataset.

Improvements in Multi-Document Abstractive Summarization using Multi Sentence Compression with Word Graph and Node Alignment

The present work proposes a scheme for multi-document abstractive text summarization using node-aligned Word Graph based representation of clustered sentences. In the first step, the proposed scheme uses SBERT embedding for representing the sentences as fixed-size vectors. The sentences belonging to the same cluster are then represented using Word Graph, in which words of different sentences are aligned based on their semantic and syntactic similarities. The advantage of the above representation is that it utilizes alignment information of words between pairs of similar sentences to merge nodes in the Word Graph, and thereby facilitating the generation of sentences with multiple chunks of information. A sentence scoring function assisted by an intensification function is used to measure the grammaticality and informativeness of the generated sentences. Integer Linear Programming has been used to make the final selection of the scored sentences for generating the abstract. Experiments conducted for the task of sentence fusion and multi-document summarization demonstrate superior performance in comparison with the state-of-the-art techniques available in the literature.

A Highly Scalable Method for Extractive Text Summarization Using Convex Optimization

The paper describes a convex optimization formulation of the extractive text summarization problem and a simple and scalable algorithm to solve it. The optimization program is constructed as a convex relaxation of an intuitive but computationally hard integer programming problem. The objective function is highly symmetric, being invariant under unitary transformations of the text representations. Another key idea is to replace the constraint on the number of sentences in the summary with a convex surrogate. For solving the program we have designed a specific projected gradient descent algorithm and analyzed its performance in terms of execution time and quality of the approximation. Using the datasets DUC 2005 and Cornell Newsroom Summarization Dataset, we have shown empirically that the algorithm can provide competitive results for single document summarization and multi-document query-based summarization. On the Cornell Newsroom Summarization Dataset, it ranked second among the unsupervised methods tested. For the more challenging task of multi-document query-based summarization, the method was tested on the DUC 2005 Dataset. Our algorithm surpassed the other reported methods with respect to the ROUGE-SU4 metric, and it was at less than 0.01 from the top performing algorithms with respect to ROUGE-1 and ROUGE-2 metrics.

Summarizing Learning Materials Using Graph Based Multi-Document Summarization

The learners and teachers of the teaching-learning process highly depend on online learning systems such as E-learning, which contains huge volumes of electronic contents related to a course. The multi-document summarization (MDS) is useful for summarizing such electronic contents. This article applies the task of MDS in an E-learning context. The objective of this article is threefold: 1) design a generic graph based multi-document summarizer DSGA (Dynamic Summary Generation Algorithm) to produce a variable length (dynamic) summary of academic text based learning materials based on a learner's request; 2) analyze the summary generation process; 3) perform content-based and task-based evaluations on the generated summary. The experimental results show that the DSGA summarizer performs better than the graph-based summarizers LexRank (LR) and Aggregate Similarity (AS). From the task-based evaluation, it is observed that the generated summary helps the learners to understand and comprehend the materials easily.

Efficient Relational Sentence Ordering Network

In this paper, we propose a novel deep Efficient Relational Sentence Ordering Network (referred to as ERSON) by leveraging pre-trained language model in both encoder and decoder architectures to strengthen the coherence modeling of the entire model. Specifically, we first introduce a divide-and-fuse BERT (referred to as DF-BERT), a new refactor of BERT network, where lower layers in the improved model encode each sentence in the paragraph independently, which are shared by different sentence pairs, and the higher layers learn the cross-attention between sentence pairs jointly. It enables us to capture the semantic concepts and contextual information between the sentences of the paragraph, while significantly reducing the runtime and memory consumption without sacrificing the model performance. Besides, a Relational Pointer Decoder (referred to as RPD) is developed, which utilizes the pre-trained Next Sentence Prediction (NSP) task of BERT to capture the useful relative ordering information between sentences to enhance the order predictions. In addition, a variety of knowledge distillation based losses are added as auxiliary supervision to further improve the ordering performance. The extensive evaluations on Sentence Ordering, Order Discrimination, and Multi-Document Summarization tasks show the superiority of ERSON to the state-of-the-art ordering methods.

Nutri-bullets: Summarizing Health Studies by Composing Segments

We introduce Nutri-bullets, a multi-document summarization task for health and nutrition. First, we present two datasets of food and health summaries from multiple scientific studies. Furthermore, we propose a novel extract-compose model to solve the problem in the regime of limited parallel data. We explicitly select key spans from several abstracts using a policy network, followed by composing the selected spans to present a summary via a task specific language model. Compared to state-of-the-art methods, our approach leads to more faithful, relevant and diverse summarization -- properties imperative to this application. For instance, on the BreastCancer dataset our approach gets a more than 50% improvement on relevance and faithfulness.

A Novel Framework for Multi-Document Temporal Summarization (MDTS)

Internet or Web consists of a massive amount of information, handling which is a tedious task. Summarization plays a crucial role in extracting or abstracting key content from multiple sources with its meaning contained, thereby reducing the complexity in handling the information. Multi-document summarization gives the gist of the content collected from multiple documents. Temporal summarization concentrates on temporally related events. This paper proposes a Multi-Document Temporal Summarization (MDTS) technique that generates the summary based on temporally related events extracted from multiple documents. This technique extracts the events with the time stamp. TIMEML standards tags are used in extracting events and times. These event-times are stored in a structured database form for easier operations. Sentence ranking methods are build based on the frequency of events occurrences in the sentence. Sentence similarity measures are computed to eliminate the redundant sentences in an extracted summary. Depending on the required summary length, top-ranked sentences are selected to form the summary. Experiments are conducted on DUC 2006 and DUC 2007 data set that was released for multi-document summarization task. The extracted summaries are evaluated using ROUGE to determine precision, recall and F measure of generated summaries. The performance of the proposed method is compared with particle swarm optimization-based algorithm (PSOS), Cat swarm optimization-based summarization (CSOS), Cuckoo Search based multi-document summarization (MDSCSA). It is found that the performance of MDTS is better when compared with other methods. Doi: 10.28991/esj-2021-01268 Full Text: PDF

Knowledge Base Driven Automatic Text Summarization using Multi-objective Optimization

Automatic Text summarization aims to automati-cally generate condensed summary from a large set of documents on the same topic. We formulate text summarization task as a multi-objective optimization problem by defining information coverage and diversity as two conflicting objective functions. With this formulation, we propose a novel technique to improve the performance using a knowledge base. The main rationale of the approach is to extract important text features of the original text by detecting important entities in a knowledge base. Next, an improvement on the multi-objective optimization algorithm is also proposed for the automatic text summarization problem. The focus is on improving efficiency of the each steps in the evolutionary multi-objective optimization process which is applicable to all tasks with the same problem formulation. The result summary of the suggested method ensure the maximum coverage of the original documents and the diversity of the sentences in the summary among each other. The experiments on DUC2002 and DUC2004 multi-document summarization task dataset shows that the proposed model is effective compared to other methods.

Extractive Multi-Document Summarization Model Based On Different Integrations of Double Similarity Measures

Currently, the prominence of automatic multi document summarization task belongs to the information rapid increasing on the Internet. Automatic document summarization technology is progressing and may offer a solution to the problem of information overload. 
 Automatic text summarization system has the challenge of producing a high quality summary. In this study, the design of generic text summarization model based on sentence extraction has been redirected into a more semantic measure reflecting individually the two significant objectives: content coverage and diversity when generating summaries from multiple documents as an explicit optimization model. The proposed two models have been then coupled and defined as a single-objective optimization problem. Also, for improving the performance of the proposed model, different integrations concerning two similarity measures have been introduced and applied to the proposed model along with the single similarity measures that are based on using Cosine, Dice and similarity measures for measuring text similarity. For solving the proposed model, Genetic Algorithm (GA) has been used. Document sets supplied by Document Understanding Conference 2002 ( ) have been used for the proposed system as an evaluation dataset. Also, as an evaluation metric, Recall-Oriented Understudy for Gisting Evaluation ( ) toolkit has been used for performance evaluation of the proposed method. Experimental results have illustrated the positive impact of measuring text similarity using double integration of similarity measures against single similarity measure when applied to the proposed model wherein the best performance in terms of and has been recorded for the integration of Cosine similarity and similarity.

Multiobjective-Based Approach for Microblog Summarization

In recent years, social networking sites such as Twitter have become the primary sources for real-time information of ongoing events such as political rallies, natural disasters, and so on. At the time of occurrence of natural disasters, it has been seen that relevant information collected from tweets can help in different ways. Therefore, there is a need to develop an automated microblog/tweet summarization system to automatically select relevant tweets. In this article, we employ the concept of multiobjective optimization in microblog summarization to produce good quality summaries. Different statistical quality measures namely, length, tf-idf score of the tweets, antiredundancy, measuring different aspects of summary, are optimized simultaneously using the search capability of a multiobjective differential evolution technique. Different types of genetic operators including recently developed self-organizing map (a type of neural network) based operator, are explored in the proposed framework. To measure the similarity between tweets, word mover distance is utilized which is capable of capturing the semantic similarity between tweets. For evaluation, four benchmark data sets related to disaster events are used, and the results obtained are compared with various state-of-the-art techniques using ROUGE measures. It has been found that our algorithm improves by 62.37% and 5.65% in terms of ROUGE–2 and ROUGE–L scores, respectively, over the state-of-the-art techniques. Results are also validated using statistical significance t-test. At the end of the article, extension of the proposed approach to solve the multidocument summarization task is also illustrated.

Central Embeddings for Extractive Summarization Based on Similarity

In this work we propose using word embeddings combined with unsupervised methods suchas clustering for the multi-document summarization task of DUC (Document Understanding Conference) 2002. We aim to find evidence that semantic information is kept in word embeddings and this representation is subject to be grouped based on their similarity, so that main ideas can be identified in sets of documents. We experiment with different clustering methods to extract candidates for the multi-document summarization task. Our experiments show that our method is able to find the prevalent ideas. ROUGE measures of our experiments are similar to the state of the art, despite the fact that not all the main ideas are found; as our method does not require annotated resources, it provides a domain and language independent way to create a summary.

A Game Theory Approach for Multi-document Summarization

In today’s era, information has been growing exponentially on the web, due to which extraction of relevant and concise information has become a challenging task. To overcome the above problem, a fundamental tool known as summarization techniques has been used for understanding and organizing such large datasets. Recently, researchers have been devoting a lot of effort to develop semantics-based models, so as to improve summarization performance. In this paper, a versatile and principled game theory-based multi-document summarization framework integrated with Wikipedia ontology is proposed. The framework exploits the submodularity hidden in underlying ontology and is optimized using the proposed improved algorithm, to enhance the summarization performance. Results of the proposed approach were evaluated with the ROUGE evaluation metric for different multi-document summarization tasks against human-generated summaries and it outperformed DUC, TAC competitors, and other competitive methods.

Calculating the Upper Bounds for Multi-Document Summarization using Genetic Algorithms

Over the last years, several Multi-Document Summarization (MDS) methods have been presented in Document Understanding Conference (DUC) workshops. Since DUC01, several methods have been presented in approximately 268 publications of the state-of-the-art, that have allowed the continuous improvement of MDS, however in most works the upper bounds were unknowns. Recently, some works have been focused to calculate the best sentence combinations of a set of documents and in previous works we have been calculated the significance for single-document summarization task in DUC01 and DUC02 datasets. However, for MDS task has not performed an analysis of significance to rank the best multi-document summarization methods. In this paper, we propose a method based on Genetic Algorithms for calculating the best sentence combinations of DUC01 and DUC02 datasets in MDS through a meta-document representation. Moreover, we have calculated three heuristics mentioned in several works of state-of-the-art to rank the most recent MDS methods, through the calculus of upper bounds and lower bounds.

Extractive multi-document summarization using multilayer networks

Huge volumes of textual information has been produced every single day. In order to organize and understand such large datasets, in recent years, summarization techniques have become popular. These techniques aims at finding relevant, concise and non-redundant content from such a big data. While network methods have been adopted to model texts in some scenarios, a systematic evaluation of multilayer network models in the multi-document summarization task has been limited to a few studies. Here, we evaluate the performance of a multilayer-based method to select the most relevant sentences in the context of an extractive multi document summarization (MDS) task. In the adopted model, nodes represent sentences and edges are created based on the number of shared words between sentences. Differently from previous studies in multi-document summarization, we make a distinction between edges linking sentences from different documents (inter-layer) and those connecting sentences from the same document (intra-layer). As a proof of principle, our results reveal that such a discrimination between intra- and inter-layer in a multilayered representation is able to improve the quality of the generated summaries. This piece of information could be used to improve current statistical methods and related textual models.

Extractive Summarization: Limits, Compression, Generalized Model and Heuristics

Due to its promise to alleviate information overload, text summarization has attracted the attention of many researchers. However, it has remained a serious challenge. Here, we first prove empirical limits on the recall (and F1-scores) of extractive summarizers on the DUC datasets under ROUGE evaluation for both the single-document and multi-document summarization tasks. Next we define the concept of compressibility of a document and present a new model of summarization, which generalizes existing models in the literature and integrates several dimensions of the summarization problem, viz., abstractive versus extractive, single versus multi-document, and syntactic versus semantic. Finally, we examine some new and some existing single-document summarization algorithms in a single framework and compare with state of the art summarizers on DUC data.

Weighted hierarchical archetypal analysis for multi-document summarization

Multi-document summarization (MDS) is becoming a crucial task in natural language processing. MDS targets to condense the most important information from a set of documents to produce a brief summary. Most existing extractive multi-document summarization methods employ different sentence selection approaches to obtain the summary as a subset of sentences from the given document set. The ability of the weighted hierarchical archetypal analysis to select “the best of the best” summary sentences motivates us to use this method in our solution to multi-document summarization tasks. In this paper, we propose a new framework for various multi-document summarization tasks based on weighted hierarchical archetypal analysis. The paper demonstrates how four variant summarization tasks, including general, query-focused, update, and comparative summarization, can be modeled as different versions acquired from the proposed framework. Experiments on summarization data sets (DUC04-07, TAC08) are conducted to demonstrate the efficiency and effectiveness of our framework for all four kinds of the multi-document summarization tasks.