Method For Multi-document Summarization Research Articles

SGCSumm: An extractive multi-document summarization method based on pre-trained language model, submodularity, and graph convolutional neural networks

The increase in online text generation by humans and machines needs automatic text summarization systems. Recent research studies commonly use deep learning, besides sentence embedding and feature learning mechanisms, to find a solution for text summarization. But they ignore the fact that while finding the optimal solution for extractive text summarization is NP-hard, how do they ensure the quality of their solution? In our previous work, an extractive summarizer, called DSNSum, was proposed based on deep submodular network (DSN) that uses handcrafted features. It leverages submodularity to guarantee a minimum bound for performance. In this paper, submodular graph convolutional summarizer (SGCSumm), an extractive multi-document summarization method, is presented where DSN is used to guarantee a minimum performance bound. It addresses DSNSum’s shortcomings and makes enhancements that yield a summarizer supporting sentence embedding and graph structure-aware feature learning. It takes a formal approach to show how the enhancements are possible without losing the guarantee of a minimum bound for performance. Finally, to evaluate the performance of the proposed summarization method, the DUC 2004 and DailyMail/CNN datasets are used. The experimental results show that the performance of SGCSumm is comparable to that of state-of-the-art summarization methods in terms of ROUGE scores.

A multi-objective memetic algorithm for query-oriented text summarization: Medicine texts as a case study

Automatic text summarization is a topic of great interest in many fields of knowledge. Particularly, query-oriented extractive multi-document text summarization methods have increased their importance recently, since they can automatically generate a summary according to a query given by the user. One way to address this problem is by multi-objective optimization approaches. In this paper, a memetic algorithm, specifically a Multi-Objective Shuffled Frog-Leaping Algorithm (MOSFLA) has been developed, implemented, and applied to solve the query-oriented extractive multi-document text summarization problem. Experiments have been conducted with datasets from Text Analysis Conference (TAC), and the obtained results have been evaluated with Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. The results have shown that the proposed approach has achieved important improvements with respect to the works of scientific literature. Specifically, 25.41%, 7.13%, and 30.22% of percentage improvements in ROUGE-1, ROUGE-2, and ROUGE-SU4 scores have been respectively reached. In addition, MOSFLA has been applied to medicine texts from the Topically Diverse Query Focus Summarization (TD-QFS) dataset as a case study.

A Survey on Multi-document Summarization and Domain-Oriented Approaches

Before the advent of the World Wide Web, lack of information was a problem. But with the advent of the web today, we are faced with an explosive amount of information in every area of search. This extra information is troublesome and prevents a quick and correct decision. This is the problem of information overload. Multi-document summarization is an important solution for this problem by producing a brief summary containing the most important information from a set of documents in a short time. This summary should preserve the main concepts of the documents. When the input documents are related to a specific domain, for example, medicine or law, summarization faces more challenges. Domain-oriented summarization methods use special characteristics related to that domain to generate summaries. This paper introduces the purpose of multi-document summarization systems and discusses domain-oriented approaches. Various methods have been proposed by researchers for multi-document summarization. This survey reviews the categorizations that authors have made on multi-document summarization methods. We also categorize the multi-document summarization methods into six categories: machine learning, clustering, graph, Latent Dirichlet Allocation (LDA), optimization, and deep learning. We review the different methods presented in each of these groups. We also compare the advantages and disadvantages of these groups. We have discussed the standard datasets used in this field, evaluation measures, challenges and recommendations.

Unsupervised Graph-Based Tibetan Multi-Document Summarization

Text summarization creates subset that represents the most important or relevant information in the original content, which effectively reduce information redundancy. Recently neural network method has achieved good results in the task of text summarization both in Chinese and English, but the research of text summarization in low-resource languages is still in the exploratory stage, especially in Tibetan. What’s more, there is no large-scale annotated corpus for text summarization. The lack of dataset severely limits the development of low-resource text summarization. In this case, unsupervised learning approaches are more appealing in low-resource languages as they do not require labeled data. In this paper, we propose an unsupervised graph-based Tibetan multi-document summarization method, which divides a large number of Tibetan news documents into topics and extracts the summarization of each topic. Summarization obtained by using traditional graph-based methods have high redundancy and the division of documents topics are not detailed enough. In terms of topic division, we adopt two level clustering methods converting original document into document-level and sentence-level graph, next we take both linguistic and deep representation into account and integrate external corpus into graph to obtain the sentence semantic clustering. Improve the shortcomings of the traditional K-Means clustering method and perform more detailed clustering of documents. Then model sentence clusters into graphs, finally remeasure sentence nodes based on the topic semantic information and the impact of topic features on sentences, higher topic relevance summary is extracted. In order to promote the development of Tibetan text summarization, and to meet the needs of relevant researchers for high-quality Tibetan text summarization datasets, this paper manually constructs a Tibetan summarization dataset and carries out relevant experiments. The experiment results show that our method can effectively improve the quality of summarization and our method is competitive to previous unsupervised methods.

Hybrid multi-document summarization using pre-trained language models

Abstractive multi-document summarization is a type of automatic text summarization. It obtains information from multiple documents and generates a human-like summary from them. In this paper, we propose an abstractive multi-document summarization method called HMSumm. The proposed method is a combination of extractive and abstractive summarization approaches. First, it constructs an extractive summary from multiple input documents, and then uses it to generate the abstractive summary. Redundant information, which is a global problem in multi-document summarization, is managed in the first step. Specifically, the determinantal point process (DPP) is used to deal with redundancy. This step also controls the length of the input sequence for the abstractive summarization process. This step has two effects. The first is to reduce the computational time. The second is to preserve the important parts of the input documents for an abstractive summarizer. We employ a deep submodular network (DSN) to determine the quality of the sentences in the extractive summary, and use BERT-based similarities to compute the redundancy. The obtained extractive summary is fed into BART and T5 pre-trained models to generate two abstractive summaries. We use the diversity of sentences in each summary to select one of them as the final abstractive summary. To evaluate the performance of HMSumm, we use both human evaluations and ROUGE-based assessments, and compare it with several state-of-the-art methods. We use DUC 2002, DUC 2004, Multi-News, and CNN/DailyMail datasets to evaluate the algorithms. The experimental results show that HMSumm outperforms the related state-of-the-art algorithms.

Unsupervised extractive multi-document summarization method based on transfer learning from BERT multi-task fine-tuning

Text representation is a fundamental cornerstone that impacts the effectiveness of several text summarization methods. Transfer learning using pre-trained word embedding models has shown promising results. However, most of these representations do not consider the order and the semantic relationships between words in a sentence, and thus they do not carry the meaning of a full sentence. To overcome this issue, the current study proposes an unsupervised method for extractive multi-document summarization based on transfer learning from BERT sentence embedding model. Moreover, to improve sentence representation learning, we fine-tune BERT model on supervised intermediate tasks from GLUE benchmark datasets using single-task and multi-task fine-tuning methods. Experiments are performed on the standard DUC’2002–2004 datasets. The obtained results show that our method has significantly outperformed several baseline methods and achieves a comparable and sometimes better performance than the recent state-of-the-art deep learning–based methods. Furthermore, the results show that fine-tuning BERT using multi-task learning has considerably improved the performance.

Extractive multi document summarization using harmony search algorithm

The exponential growth of information on the internet makes it troublesome for users to get valuable information. Text summarization is the process to overcome such a problem. An adequate summary must have wide coverage, high diversity, and high readability. In this article, a new method for multi-document summarization has been supposed based on a harmony search algorithm that optimizes the coverage, diversity, and readability. Concerning the benchmark dataset Text Analysis Conference (TAC-2011), the ROUGE package used to measure the effectiveness of the proposed model. The calculated results support the effectiveness of the proposed approach.

The impact of term-weighting schemes and similarity measures on extractive multi-document text summarization

Automatic text summarization is currently a topic of great interest in many knowledge fields. Extractive multi-document text summarization methods aim to reduce the textual information from a document collection by covering the main content and reducing the redundant information. In the scientific literature, there are different approaches related to term-weighting schemes and similarity measures, which are necessary for implementing an automatic summary system. However, to the best of the authors’ knowledge, there are no studies to analyze the performance of the different schemes and measures. In this paper, all possible combinations of the most common term-weighting schemes and similarity measures used in the extractive multi-document text summarization field have been implemented, compared, and analyzed. Experiments have been performed with Document Understanding Conferences (DUC) datasets, and the model performance has been assessed with eight Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics and the execution time. Results show that the best term-weighting scheme is the term-frequency inverse-sentence-frequency scheme, and the best similarity measure is the cosine similarity. Even more, the combination formed by both of them has obtained the best average results in 87.5% of ROUGE scores compared to the other combinations.

A deep neural architecture based meta-review generation and final decision prediction of a scholarly article

Peer reviews form an essential part of scientific communications. Research papers and proposals are reviewed by several peers before they are finally accepted or rejected for publication and funding, respectively. With the steady increase in the number of research domains, scholarly venues (journal and/or conference), researchers, and papers, managing the peer review process is becoming a daunting task. Application of recommender systems to assist peer reviewing is, therefore, being explored and becoming an emerging research area. In this paper, we present a deep learning network based Meta-Review Generation considering peer review prediction of the scholarly article (MRGen). MRGen is able to provide solutions for: (i) Peer review prediction (Task 1) and (ii) Meta-review generation (Task 2). First, the system takes the peer reviews as input and produces a draft meta-review. Then it employs an integrated framework of convolution layer, long short-term memory (LSTM) model, Bi-LSTM model, and attention mechanism to predict the final decision (accept/reject) of the scholarly article. Based on the final decision, the proposed model MRGen incorporates Pointer Generator Network-based abstractive summarization to generate the final meta-review. The focus of our approach is to give a concise meta-review that maximizes information coverage, coherence, readability and also reduces redundancy. Extensive experiments conducted on the PeerRead dataset demonstrate good consistency between the recommended decisions and original decisions. We also compare the performance of MRGen with some of the existing state-of-the-art multi-document summarization methods. The system also outperforms a few existing models based on accuracy, Rouge scores, readability, non-redundancy, and cohesion.

An unsupervised method for extractive multi-document summarization based on centroid approach and sentence embeddings

Extractive multi-document summarization (MDS) is the process of automatically summarizing a collection of documents by ranking sentences according to their importance and informativeness. Text representation is a fundamental process that affects the effectiveness of many text summarization methods. Word embedding representations have shown to be effective for several Natural Language Processing (NLP) tasks including Automatic Text Summarization (ATS). However, most of these representations do not consider the order and the semantic relationships between words in a sentence. This does not fully allow grasping the sentence semantics and the syntactic relationships between sentences constituents. In this paper, to overcome this problem, we propose an unsupervised method for generic extractive multi-document summarization based on the sentence embedding representations and the centroid approach. The proposed method selects relevant sentences according to the final score obtained by combining three scores: sentence content relevance, sentence novelty, and sentence position scores. The sentence content relevance score is computed as the cosine similarity between the centroid embedding vector of the cluster of documents and the sentence embedding vectors. The sentence novelty metric is explicitly adopted to deal with redundancy. The sentence position metric assumes that the first sentences of a document are more relevant to the summary, and it assigns high scores to these sentences. Moreover, this paper provides a comparative analysis of nine sentence embedding models used to represent sentences as dense vectors in a low dimensional vector space in the context of extractive multi-document summarization. Experiments are performed on the standard DUC’2002–2004 benchmark datasets and the Multi-News dataset. The overall obtained results have shown that our method outperforms several state-of-the-art methods and achieves promising results compared to the best performing methods including supervised deep learning based methods.

ISSE: a new iterative sentence scoring and extraction scheme for automatic text summarization

An ever-increasing volume of contextual data appearing on various types of media overwhelms users sharing information, as well as bringing up storage concerns. Automatic text summarization mitigates these challenges and helps users cope more efficiently by generating more accurate and representative summaries. In this paper, we propose a new extractive, multi-document summarization method based on a new iterative sentence scoring and extraction scheme which uses the lexical chain and clustering concepts. The evaluation is done by a well-known standard evaluator in terms of conventional measures of performance, Recall, Precision, and F-score. The results show definite improvement at reasonable compression rates of around 10%, compared to other major benchmark summarizers in the literature.

Sparse two level topic model for extraction of general summary words

Extractive multi-document summarization methods based on topic models find relevant general concepts or topics that are most representative of the documents. These topics are used for sentence ranking and selection. In this paper, a two level topic model using spike and slab prior is proposed that identify better general topics for summarization. Spike and slab prior is used earlier for finding aspect specific topics. Proposed two level model uses spike and slab prior to achieve better general topics at high level of topic hierarchy. Experiments conducted on DUC2007 dataset show that proposed model is able to identify more summary oriented general words and improve ROUGE score.

Semantic Similarity Based Automatic Document Summarization Method

Document summarization is the process of generating the summary of the documents gathered from the web sources. It reduces the burden of web readers by reducing the necessity of reading the entire document contents by generating the short summary. In our previous research work this is performed by introducing the method namely Noun weight based Automated Multi-Document Summarization method (NW-AMDSM). However the previous research work doesn’t concentrate on the semantic similarity which might reduce the accuracy of the summarization outcome. This is resolved in the proposed research method by introducing the method namely Semantic Similarity based Automatic Document Summarization Method (SS-ADSM). In this research work, multi document grouping is done is based on semantic similarity computation, thus the document with similar contents can be grouped more accurately. Here the semantic similarity computation is performed with the help of word net analyzer. The document grouping is done by introducing the modified FCM clustering algorithm. Finally hybrid neuro fuzzy genetic algorithm is introduced to perform the automatic summarization. The numerical analysis of the proposed research method is conducted in the matlab simulation environment and compared with other research methods in terms various performance metrics. The simulation analysis proved proposed method tends to have better performance in terms of increased accuracy of document summarization outcome.

An Indicator-based Multi-Objective Optimization Approach Applied to Extractive Multi-Document Text Summarization

The massive amount of textual information on the Internet makes that automatic text summarization methods are becoming very important nowadays. Particularly, the purpose of extractive multi-document text summarization methods is to generate summaries from a document collection by, simultaneously, covering the main content and reducing the redundant information. In the scientific literature, these summarization methods have been addressed through optimization techniques, being almost all of them single-objective optimization approaches. Nevertheless, multi-objective approaches have gained importance because their results have improved the single-objective ones.On the other hand, in the multi-objective optimization field, indicator-based approaches have obtained good results in other applications. For this reason, an Indicator-based Multi-Objective Artificial Bee Colony (IMOABC) algorithm has been developed and applied to the extractive multi-document text summarization problem. Experiments have been carried out based on Document Understanding Conferences (DUC) datasets, and the obtained results have been evaluated and compared with Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. The results have improved to the ones in the scientific literature between 7.37% and 40.76% and 2.59% and 11.24% for ROUGE-2 and ROUGE-L, respectively.

MCRMR: Maximum coverage and relevancy with minimal redundancy based multi-document summarization

In this paper, we propose a novel extraction based method for multi-document summarization that covers three important features of a good summary: coverage, non-redundancy, and relevancy. The coverage and non-redundancy features are modeled to generate a single document from the multiple documents. These features are explored by the weighted combination of word embedding and Google based similarity methods. To accommodate the relevancy feature in the system generated summaries, the text summarization task is modeled as an optimization problem, where various text features with their optimized weights are used to score the sentences to find the relevant sentences. For features’ weight optimization, we use the meta-heuristic approach, Shark Smell Optimization (SSO). The experiments are performed on six benchmark datasets (DUC04, DUC06, DUC07, TAC08, TAC11, and MultiLing13) with the co-selection and content based performance parameters. The experimental results show that the proposed approach is viable and effective for multi-document summarization.

Extractive multi-document summarization based on textual entailment and sentence compression via knapsack problem

AbstractBy increasing the amount of data in computer networks, searching and finding suitable information will be harder for users. One of the most widespread forms of information on such networks are textual documents. So exploring these documents to get information about their content is difficult and sometimes impossible. Multi-document text summarization systems are an aid to producing a summary with a fixed and predefined length, while covering the maximum content of the input documents. This paper presents a novel method for multi-document extractive summarization based on textual entailment relations and sentence compression via formulating the problem as a knapsack problem. In this approach, sentences of documents are ranked according to the extended Tf-Idf method, then entailment scores of selected sentences are computed. Through these scores, the final score of each sentence is calculated. Finally, by decreasing the lengths of sentences via sentence compression, the problem has been solved by greedy and dynamic Programming approaches to the knapsack problem. Experiments on standard summarization datasets and evaluating the results based on the Rouge system show that the suggested method, according to the best of our knowledge, has increased F-measure of query-based summarization systems by two per cent and F-measure of general summarization systems by five per cent.

Calculating the Upper Bounds for Multi-Document Summarization using Genetic Algorithms

Over the last years, several Multi-Document Summarization (MDS) methods have been presented in Document Understanding Conference (DUC) workshops. Since DUC01, several methods have been presented in approximately 268 publications of the state-of-the-art, that have allowed the continuous improvement of MDS, however in most works the upper bounds were unknowns. Recently, some works have been focused to calculate the best sentence combinations of a set of documents and in previous works we have been calculated the significance for single-document summarization task in DUC01 and DUC02 datasets. However, for MDS task has not performed an analysis of significance to rank the best multi-document summarization methods. In this paper, we propose a method based on Genetic Algorithms for calculating the best sentence combinations of DUC01 and DUC02 datasets in MDS through a meta-document representation. Moreover, we have calculated three heuristics mentioned in several works of state-of-the-art to rank the most recent MDS methods, through the calculus of upper bounds and lower bounds.

Event-based Multi-document Summarization

Daily amount of news reporting real-world events is growing exponentially. At the same time, Organizations are looking for information about current and past events that affects them, such as mergers and acquisitions of companies. The Organizations need to obtain event information in a fast and summarized form to make decisions. Event-based retrieval and summarization systems offer an efficient solution to this problem. Most summarization research work uses news stories. Although this type of documents is characterized by conveying information about events, almost all work concentrates on approaches that do not take into account this aspect. The proposed multi-document summarization methods are based on the hierarchical combination of single-document summaries. We improved our multi-document summarization methods using event information. Our approach is based on a two-stage single-document\ method that extracts a collection of key phrases, which are then used in a centrality-asrelevance passage retrieval model. To adapt centrality-as-relevance single-document summarization for multi-document summarization that is able to use event information, we needed a good and adaptable baseline system. Because the key phrase extraction play a significant role in the summarization, we improved a state-of-the-art key phrase extraction toolkit using four additional sets of semantic features. The event detection method is based on Fuzzy Fingerprint, which is a supervised method trained on documents with annotated event tags. We explored three different ways to integrate event information, achieving state-of-the-art results in both single and multi-document summarization using filtering and event-based features. To cope with the possible usage of different terms to describe the same event, we explored distributed representations of text in the form of word embeddings, which contributed to improve the multi-document summarization results. The automatic evaluation and user study performed show that these methods improve upon current state-of-the-art multi-document summarization systems on two mainstream evaluation datasets, DUC 2007 and TAC 2009. We show a relative improvement in ROUGE- 1 scores of 16% for TAC 2009 and of 17% for DUC 2007. We have also obtained improvements in ROUGE-1 upon current state-of-the-art single-document summarization systems of between 32% in clean data and 19% in noisy data. These improvements derived from the inclusion of key phrases and event information. The extraction of key phrases was also refined with additional pre-processing steps and features, which lead to a relative improvement in NDCG scores of 9%. The introduction of Fuzzy Fingerprints for event detection enabled the detection of all event types, while the best competitor, an SVM with enhanced features, only detects roughly 85% of the different types of events. This lead to a large increase in the G-Mean and variants results when using the Fuzzy Fingerprints method. This doctoral work was carried under the Dual Ph.D. program in Languages and Information Technologies of Carnegie Mellon Portugal Program. The program took place at both the Language Technologies Institute at Carnegie Mellon University (CMU) and at the Department of Computer of Science and Engineering at Instituto Superior Técnico (IST). The doctoral work was performed under the supervision of University Professor Jaime Carbonell (CMU), Distinguished Career Professor Anatole Gershman (CMU), Assistant Professor David Martins de Matos (IST), and Assistant Professor Joo P. Neto (IST). Full Professor Ricardo Baeza-Yates (Yahoo!Research/Univ. Pompeu Fabra/Univ. de Chile), Research Professor Eduard Hovy (CMU), Associate Professor Ana Paiva (IST), and Full Professor Isabel Trancoso served as dissertation committee members.

Weighted hierarchical archetypal analysis for multi-document summarization

Multi-document summarization (MDS) is becoming a crucial task in natural language processing. MDS targets to condense the most important information from a set of documents to produce a brief summary. Most existing extractive multi-document summarization methods employ different sentence selection approaches to obtain the summary as a subset of sentences from the given document set. The ability of the weighted hierarchical archetypal analysis to select “the best of the best” summary sentences motivates us to use this method in our solution to multi-document summarization tasks. In this paper, we propose a new framework for various multi-document summarization tasks based on weighted hierarchical archetypal analysis. The paper demonstrates how four variant summarization tasks, including general, query-focused, update, and comparative summarization, can be modeled as different versions acquired from the proposed framework. Experiments on summarization data sets (DUC04-07, TAC08) are conducted to demonstrate the efficiency and effectiveness of our framework for all four kinds of the multi-document summarization tasks.

Exploring events and distributed representations of text in multi-document summarization

In this article, we explore an event detection framework to improve multi-document summarization. Our approach is based on a two-stage single-document method that extracts a collection of key phrases, which are then used in a centrality-as-relevance passage retrieval model. We explore how to adapt this single-document method for multi-document summarization methods that are able to use event information. The event detection method is based on Fuzzy Fingerprint, which is a supervised method trained on documents with annotated event tags. To cope with the possible usage of different terms to describe the same event, we explore distributed representations of text in the form of word embeddings, which contributed to improve the summarization results. The proposed summarization methods are based on the hierarchical combination of single-document summaries. The automatic evaluation and human study performed show that these methods improve upon current state-of-the-art multi-document summarization systems on two mainstream evaluation datasets, DUC 2007 and TAC 2009. We show a relative improvement in ROUGE-1 scores of 16% for TAC 2009 and of 17% for DUC 2007.