Query-focused Multi-document Summarization Research Articles

An indicator-based multi-objective variable neighborhood search approach for query-focused summarization

Currently, automatic multi-document summarization is an interesting subject in numerous fields of study. As a part of it, query-focused summarization is becoming increasingly important in recent times. These methods can automatically produce a summary based on a query given by the user, including the most relevant information from the query at the same time as the redundancy among sentences is reduced. This can be achieved by developing and applying a multi-objective optimization approach. In this paper, an Indicator-based Multi-Objective Variable Neighborhood Search (IMOVNS) algorithm has been designed, implemented, and tested for the query-focused extractive multi-document summarization problem. Experiments have been carried out with datasets from Text Analysis Conference (TAC). The results were evaluated using the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. IMOVNS has greatly improved the results presented in the scientific literature, providing improvement percentages in ROUGE metric reaching up to 69.24% in ROUGE-1, up to 57.70% in ROUGE-2, and up to 77.37% in ROUGE-SU4 scores. Hence, the proposed IMOVNS offers a promising solution to the query-focused summarization problem, thus highlighting its efficacy and potential for enhancing automatic summarization techniques.

Review on Query-focused Multi-document Summarization (QMDS) with Comparative Analysis

The problem of query-focused multi-document summarization (QMDS) is to generate a summary from multiple source documents on identical/similar topics based on the query submitted by the users. This article provides a systematic review of the literature of QMDS. The research works are classified into six major categories based on the summarization methodologies used. Different techniques used for finding query-relevant summaries for different algorithms under each of the six major groups are reported. Further, 17 evaluation metrics used for evaluating algorithms for text summaries against the human-curated summaries are compiled here in this article. Extensive experiments are performed on eight different datasets. Comparative results of nine methodologies, each representing one of the six different groups, are presented. Seven different evaluation metrics are used in the comparative study. It is observed that DL- and ML-based QMDS methods perform. better in comparison to the other methods.

Can Anaphora Resolution Improve Extractive Query-Focused Multi-Document Summarization?

Can Anaphora Resolution Improve Extractive Query-Focused Multi-Document Summarization?

Query-focused multi-document text summarization using fuzzy inference

The present paper proposes a fuzzy inference system for query-focused multi-document text summarization (MTS). The overall scheme is based on Mamdani Inferencing scheme which helps in designing Fuzzy Rule base for inferencing about the decision variable from a set of antecedent variables. The antecedent variables chosen for the task are from linguistic and positional heuristics, and similarity of the documents with the user-defined query. The decision variable is the rank of the sentences as decided by the rules. The final summary is generated by solving an Integer Linear Programming problem. For abstraction coreference resolution is applied on the input sentences in the pre-processing step. Although designed on the basis of a small set of antecedent variables the results are very promising.

Sentiment-oriented query-focused text summarization addressed with a multi-objective optimization approach

Nowadays, the automatic text summarization is a highly relevant task in many contexts. In particular, query-focused summarization consists of generating a summary from one or multiple documents according to a query given by the user. Additionally, sentiment analysis and opinion mining analyze the polarity of the opinions contained in texts. These two issues are integrated in an approach to produce an opinionated summary according to the user’s query. Thereby, the query-focused sentiment-oriented extractive multi-document text summarization problem entails the optimization of different criteria, specifically, query relevance, redundancy reduction, and sentiment relevance. An adaptation of the metaheuristic population-based crow search algorithm has been designed, implemented, and tested to solve this multi-objective problem. Experiments have been carried out by using datasets from the Text Analysis Conference (TAC) datasets. Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics and the Pearson correlation coefficient have been used for the performance assessment. The results have reported that the proposed approach outperforms the existing methods in the scientific literature, with a percentage improvement of 75.5% for ROUGE-1 score and 441.3% for ROUGE-2 score. It also has been obtained a Pearson correlation coefficient of +0.841, reporting a strong linear positive correlation between the sentiment scores of the generated summaries and the sentiment scores of the queries of the topics.

Data Augmentation for Abstractive Query-Focused Multi-Document Summarization

The progress in Query-focused Multi-Document Summarization (QMDS) has been limited by the lack of sufficient largescale high-quality training datasets. We present two QMDS training datasets, which we construct using two data augmentation methods: (1) transferring the commonly used single-document CNN/Daily Mail summarization dataset to create the QMDSCNN dataset, and (2) mining search-query logs to create the QMDSIR dataset. These two datasets have complementary properties, i.e., QMDSCNN has real summaries but queries are simulated, while QMDSIR has real queries but simulated summaries. To cover both these real summary and query aspects, we build abstractive end-to-end neural network models on the combined datasets that yield new state-of-the-art transfer results on DUC datasets. We also introduce new hierarchical encoders that enable a more efficient encoding of the query together with multiple documents. Empirical results demonstrate that our data augmentation and encoding methods outperform baseline models on automatic metrics, as well as on human evaluations along multiple attributes.

Unsupervised query-focused multi-document summarization based on transfer learning from sentence embedding models, BM25 model, and maximal marginal relevance criterion

Extractive query-focused multi-document summarization (QF-MDS) is the process of automatically generating an informative summary from a collection of documents that answers a pre-given query. Sentence and query representation is a fundamental cornerstone that affects the effectiveness of several QF-MDS methods. Transfer learning using pre-trained word embedding models has shown promising performance in many applications. However, most of these representations do not consider the order and the semantic relationships between words in a sentence, and thus they do not carry the meaning of a full sentence. In this paper, to deal with this issue, we propose to leverage transfer learning from pre-trained sentence embedding models to represent documents’ sentences and users’ queries using embedding vectors that capture the semantic and the syntactic relationships between their constituents (words, phrases). Furthermore, BM25 and semantic similarity function are linearly combined to retrieve a subset of sentences based on their relevance to the query. Finally, the maximal marginal relevance criterion is applied to re-rank the selected sentences by maintaining query relevance and minimizing redundancy. The proposed method is unsupervised, simple, efficient, and requires no labeled text summarization training data. Experiments are conducted using three standard datasets from the DUC evaluation campaign (DUC’2005–2007). The overall obtained results show that our method outperforms several state-of-the-art systems and achieves comparable results to the best performing systems, including supervised deep learning-based methods.

Regression model focused on query for multi documents summarization based on significance of the sentence position

Document summarization is needed to get the information effectively and efficiently. One method used to obtain the document summarization by applying machine learning techniques. This paper proposes the application of regression models to query-focused multi-document summarization based on the significance of the sentence position. The method used is the Support Vector Regression (SVR) which estimates the weight of the sentence on a set of documents to be made as a summary based on sentence feature which has been defined previously. A series of evaluations performed on a data set of DUC 2005. From the test results obtained summary which has an average precision and recall values of 0.0580 and 0.0590 for measurements using ROUGE-2, ROUGE 0.0997 and 0.1019 for measurements using the proposed regression-SU4. Model can perform measurements of the significance of the position of the sentence in the document well.

Sentence Relations for Extractive Summarization with Deep Neural Networks

Sentence regression is a type of extractive summarization that achieves state-of-the-art performance and is commonly used in practical systems. The most challenging task within the sentence regression framework is to identify discriminative features to represent each sentence. In this article, we study the use of sentence relations, e.g., Contextual Sentence Relations (CSR), Title Sentence Relations (TSR), and Query Sentence Relations (QSR), so as to improve the performance of sentence regression. CSR, TSR, and QSR refer to the relations between a main body sentence and its local context, its document title, and a given query, respectively. We propose a deep neural network model, Sentence Relation-based Summarization (SRSum), that consists of five sub-models, PriorSum, CSRSum, TSRSum, QSRSum, and SFSum. PriorSum encodes the latent semantic meaning of a sentence using a bi-gram convolutional neural network. SFSum encodes the surface information of a sentence, e.g., sentence length, sentence position, and so on. CSRSum, TSRSum, and QSRSum are three sentence relation sub-models corresponding to CSR, TSR, and QSR, respectively. CSRSum evaluates the ability of each sentence to summarize its local contexts. Specifically, CSRSum applies a CSR-based word-level and sentence-level attention mechanism to simulate the context-aware reading of a human reader, where words and sentences that have anaphoric relations or local summarization abilities are easily remembered and paid attention to. TSRSum evaluates the semantic closeness of each sentence with respect to its title, which usually reflects the main ideas of a document. TSRSum applies a TSR-based attention mechanism to simulate people’s reading ability with the main idea (title) in mind. QSRSum evaluates the relevance of each sentence with given queries for the query-focused summarization. QSRSum applies a QSR-based attention mechanism to simulate the attentive reading of a human reader with some queries in mind. The mechanism can recognize which parts of the given queries are more likely answered by a sentence under consideration. Finally as a whole, SRSum automatically learns useful latent features by jointly learning representations of query sentences, content sentences, and title sentences as well as their relations. We conduct extensive experiments on six benchmark datasets, including generic multi-document summarization and query-focused multi-document summarization. On both tasks, SRSum achieves comparable or superior performance compared with state-of-the-art approaches in terms of multiple ROUGE metrics.

Query-focused multi-document summarization using hypergraph-based ranking

General graph random walk has been successfully applied in multi-document summarization, but it has some limitations to process documents by this way. In this paper, we propose a novel hypergraph based vertex-reinforced random walk framework for multi-document summarization. The framework first exploits the Hierarchical Dirichlet Process (HDP) topic model to learn a word-topic probability distribution in sentences. Then the hypergraph is used to capture both cluster relationship based on the word-topic probability distribution and pairwise similarity among sentences. Finally, a time-variant random walk algorithm for hypergraphs is developed to rank sentences which ensures sentence diversity by vertex-reinforcement in summaries. Experimental results on the public available dataset demonstrate the effectiveness of our framework.

Exploring actor–object relationships for query-focused multi-document summarization

Most research on multi-document summarization explores methods that generate summaries based on queries regardless of the users' preferences. We note that, different users can generate somewhat different summaries on the basis of the same source data and query. This paper presents our study on how to exploit the information regards how users summarized their texts. Models of different users can be used either separately, or in an ensemble-like fashion. Machine learning methods are explored in the construction of the individual models. However, we explore yet another hypothesis. We believe that the sentences selected into the summary should be coherent and supplement each other in their meaning. One method to model this relationship between sentences is by detecting actor---object relationship (AOR). The sentences that satisfy this relationship have their importance value enhanced. This paper combines ensemble summarizing system and AOR to generate summaries. We have evaluated this method on DUC 2006 and DUC 2007 using ROUGE measure. Experimental results show the supervised method that exploits the ensemble summarizing system combined with AOR outperforms previous models when considering performance in query-based multi-document summarization tasks.

A Graph Based Query Focused Multi-Document Summarization

A user's information need, normally represented as a search query, can be satisfied by creating a query focused coherent and readable summary, by fusing the relevant parts of information from multiple documents. While aggregating the information from multiple documents, the quality of the summary is improved by eliminating redundant information from the document set. In this paper, we focus on removing such redundant information and identifying the essential components from multiple documents (represented as a single global semantic graph), with respect to the given query (represented as a query graph). While the redundancy elimination is carried out using various levels of graph matching which are then indicated through canonical labeling of graphs, the selection of essential components for a query focused summary is performed, through the modified spreading activation theory, where the query graph is also integrated during the spreading activation over the global graph. The proposed system shows significant improvements in generating summaries when compared to other existing summarization systems.

A Novel Feature-based Bayesian Model for Query Focused Multi-document Summarization

Supervised learning methods and LDA based topic model have been successfully applied in the field of multi-document summarization. In this paper, we propose a novel supervised approach that can incorporate rich sentence features into Bayesian topic models in a principled way, thus taking advantages of both topic model and feature based supervised learning methods. Experimental results on DUC2007, TAC2008 and TAC2009 demonstrate the effectiveness of our approach.

A Topic-driven Summarization using K-mean Clustering and Tf-Isf Sentence Ranking

Enormous online information is available due to the World Wide Web. This needed efficient and accurate summarization systems to extract significant information. Text summarization system automatically generates a summary of a given document and helps people to make effective decisions in less time. In this paper two methods have been proposed for query-focused multi-document summarization that uses k-mean clustering, term-frequency and inversesentence-frequency method for sentence weighting to rank the sentences of the documents with respect to a given query. The proposed method finds the proximity of documents and query, and later uses this proximity to rank sentences of each document. It is assumed that the document which is nearer to a query might contain more meaning full sentences with respect to the information need expressed by user’s query further if a sentence contains rare query term than it is more informative than the sentences that contains frequent query term. Both methods first gives weights to documents according to their proximity and use these document weights to rank each of their sentences with tf-idf ranking function. A relative study for proposed methods has been done and experimental results shows that both methods are comparable because of a slight difference in performance. DUC 2007 test dataset and ROUGH-1.5.5 summarization evaluation package is used for evaluation purpose.

Weighted archetypal analysis of the multi-element graph for query-focused multi-document summarization

Most existing research on applying the matrix factorization approaches to query-focused multi-document summarization (Q-MDS) explores either soft/hard clustering or low rank approximation methods. We employ a different kind of matrix factorization method, namely weighted archetypal analysis (wAA) to Q-MDS. In query-focused summarization, given a graph representation of a set of sentences weighted by similarity to the given query, positively and/or negatively salient sentences are values on the weighted data set boundary. We choose to use wAA to compute these extreme values, archetypes, and hence to estimate the importance of sentences in target documents set. We investigate the impact of using the multi-element graph model for query focused summarization via wAA. We conducted experiments on the data of document understanding conference (DUC) 2005 and 2006. Experimental results evidence the improvement of the proposed approach over other closely related methods and many of state-of-the-art systems.

Exploiting relevance, coverage, and novelty for query-focused multi-document summarization

Summarization plays an increasingly important role with the exponential document growth on the Web. Specifically, for query-focused summarization, there exist three challenges: (1) how to retrieve query relevant sentences; (2) how to concisely cover the main aspects (i.e., topics) in the document; and (3) how to balance these two requests. Specially for the issue relevance, many traditional summarization techniques assume that there is independent relevance between sentences, which may not hold in reality. In this paper, we go beyond this assumption and propose a novel Probabilistic-modeling Relevance, Coverage, and Novelty (PRCN) framework, which exploits a reference topic model incorporating user query for dependent relevance measurement. Along this line, topic coverage is also modeled under our framework. To further address the issues above, various sentence features regarding relevance and novelty are constructed as features, while moderate topic coverage are maintained through a greedy algorithm for topic balance. Finally, experiments on DUC2005 and DUC2006 datasets validate the effectiveness of the proposed method.

Mutually Reinforced Manifold-Ranking Based Relevance Propagation Model for Query-Focused Multi-Document Summarization

Manifold-ranking has been recently exploited for query-focused summarization. It propagates query relevance from the given query to the document sentences by making use of both the relationships among the sentences and the relationships between the given query and the sentences. The sentences in a document set can be grouped into several topic themes with each theme represented by a cluster of highly related sentences. However, it is a well-recognized fact that a document set often covers a number of such topic themes. In this paper, we present a novel model to enhance manifold-ranking based relevance propagation via mutual reinforcement between sentences and theme clusters. Based on the proposed model, we develop two new sentence ranking algorithms, namely the reinforcement after relevance propagation (RARP) algorithm and the reinforcement during relevance propagation (RDRP) algorithm. The convergence issues of the two algorithms are examined. When evaluated on the DUC2005-2007 datasets and TAC2008 dataset, the performance of the two proposed algorithms is comparable with that of the top three systems. The results also demonstrate that the RDRP algorithm is more effective than the RARP algorithm.

Query-focused multi-document summarization: automatic data annotations and supervised learning approaches

AbstractIn this paper, we apply different supervised learning techniques to build query-focused multi-document summarization systems, where the task is to produce automatic summaries in response to a given query or specific information request stated by the user. A huge amount of labeled data is a prerequisite for supervised training. It is expensive and time-consuming when humans perform the labeling task manually. Automatic labeling can be a good remedy to this problem. We employ five different automatic annotation techniques to build extracts from human abstracts using ROUGE, Basic Element overlap, syntactic similarity measure, semantic similarity measure, and Extended String Subsequence Kernel. The supervised methods we use are Support Vector Machines, Conditional Random Fields, Hidden Markov Models, Maximum Entropy, and two ensemble-based approaches. During different experiments, we analyze the impact of automatic labeling methods on the performance of the applied supervised methods. To our knowledge, no other study has deeply investigated and compared the effects of using different automatic annotation techniques on different supervised learning approaches in the domain of query-focused multi-document summarization.

Applying regression models to query-focused multi-document summarization

Most existing research on applying machine learning techniques to document summarization explores either classification models or learning-to-rank models. This paper presents our recent study on how to apply a different kind of learning models, namely regression models, to query-focused multi-document summarization. We choose to use Support Vector Regression (SVR) to estimate the importance of a sentence in a document set to be summarized through a set of pre-defined features. In order to learn the regression models, we propose several methods to construct the “pseudo” training data by assigning each sentence with a “nearly true” importance score calculated with the human summaries that have been provided for the corresponding document set. A series of evaluations on the DUC data sets are conducted to examine the efficiency and the robustness of the proposed approaches. When compared with classification models and ranking models, regression models are consistently preferable.

Using query expansion in graph-based approach for query-focused multi-document summarization

This paper presents a novel query expansion method, which is combined in the graph-based algorithm for query-focused multi-document summarization, so as to resolve the problem of information limit in the original query. Our approach makes use of both the sentence-to-sentence relations and the sentence-to-word relations to select the query biased informative words from the document set and use them as query expansions to improve the sentence ranking result. Compared to previous query expansion approaches, our approach can capture more relevant information with less noise. We performed experiments on the data of document understanding conference (DUC) 2005 and DUC 2006, and the evaluation results show that the proposed query expansion method can significantly improve the system performance and make our system comparable to the state-of-the-art systems.