Recall-Oriented Understudy For Gisting Evaluation Research Articles

Evaluation of Content Compaction in Assamese Language

Abstract Text summarization is the task of condensing the input text documents into a shorter version by retaining the overall meaning and information of the original document. Though plenty of text compaction research works in English and other languages have been done till date, but text compaction in Assamese language is still lagging behind due to the limitations of resources for text compaction. This paper proposes a content compaction approach for Assamese Text to generate a summary by incorporating statistical and linguistic features and makes an attempt to extract the relevant points in the summary from the input Assamese document. The statistical features applied in this approach incorporate a new modified tf-idf, sentence positioning, length of sentences and identification of numericals. Whereas the linguistic features employed include identifying the nouns, cue words and cue phrases. To contrast the summaries that are generated both automatically and by humans, the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) framework is used. Furthermore, our proposed approach is also compared with text summarization approaches by using traditional tf-idf. It is observed that this approach has yielded impressive results while summarizing Assamese documents with average F-measure of 0.633 at the compression ratio of 50%.

Karcı summarization: A simple and effective approach for automatic text summarization using Karcı entropy

Increases in the amount of text resources available via the Internet has amplified the need for automated document summarizing tools. However, further efforts are needed in order to improve the quality of the existing summarization tools currently available. The current study proposes Karcı Summarization, a novel methodology for extractive, generic summarization of text documents. Karcı Entropy was used for the first time in a document summarization method within a unique approach. An important feature of the proposed system is that it does not require any kind of information source or training data. At the stage of presenting the input text, a tool for text processing was introduced; known as KUSH (named after its authors; Karcı, Uçkan, Seyyarer, and Hark), and is used to protect semantic consistency between sentences. The Karcı Entropy-based solution chooses the most effective, generic and most informational sentences within a paragraph or unit of text. Experimentation with the Karcı Summarization approach was tested using open-access document text (Document Understanding Conference; DUC-2002, DUC-2004) datasets. Performance achievement of the Karcı Summarization approach was calculated using metrics known as Recall-Oriented Understudy for Gisting Evaluation (ROUGE). The experimental results showed that the proposed summarizer outperformed all current state-of-the-art methods in terms of 200-word summaries in the metrics of ROUGE-1, ROUGE-2, ROUGE-L, and ROUGE-W-1.2. In addition, the proposed summarizer outperformed the nearest competitive summarizers by a factor of 6.4% for ROUGE-1 Recall on the DUC-2002 dataset. These results demonstrate that Karcı Summarization is a promising technique and it is therefore expected to attract interest from researchers in the field. Our approach was shown to have a high potential for adoptability. Moreover, the method was assessed as quite insensitive to disorderly and missing texts due to its KUSH text processing module.

A neural graph embedding approach for selecting review sentences

Abstract Product reviews written by the crowd on e-commerce shopping websites have become a critical information source for making purchasing decisions. An important challenge, however, is that the vast majority of products (e.g., 90 % of products on amazon.com) do not receive enough attention and lack sufficient reviews by the users; hence, they constitute the so-called cold products. One solution to address cold products, which has already been studied in the literature, is to generate reviews for these products by sampling review sentences from closely related warm products. Our method proposed in this paper is specifically focused on such a solution. While a majority of the works in the literature rely on product specification similarity to identify relevant reviews that can be used for review sentence selection, our work differs in that it not only employs product specification similarity but also employs product-review, product-user, and user-review interactions when determining the suitability of a review sentence to be selected. More specifically, the contributions of our work can be enumerated as follows: (1) We propose that the selection of review sentences from other products should not only consider product-product similarity but also consider product-review, user-review, and user-user relationships. As such, we show how neural graph embeddings can be used to encode product, user, and review information into an attributed heterogeneous graph representation based on which similarities can be calculated. (2) We further propose how review relevance and importance can be considered using graph traversal to select appropriate review sentences for a given cold product. (3) Finally, we systematically compare the performance of our work with those of several state-of-the-art baselines on five datasets collected from CNET.com and rottentomatoes.com with different characteristics from both quantitative (e.g., the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics) and qualitative aspects and show how our proposed approach was able to provide statistically significantly improved performance over various strong baselines.

A Text Abstraction Summary Model Based on BERT Word Embedding and Reinforcement Learning

As a core task of natural language processing and information retrieval, automatic text summarization is widely applied in many fields. There are two existing methods for text summarization task at present: abstractive and extractive. On this basis we propose a novel hybrid model of extractive-abstractive to combine BERT (Bidirectional Encoder Representations from Transformers) word embedding with reinforcement learning. Firstly, we convert the human-written abstractive summaries to the ground truth labels. Secondly, we use BERT word embedding as text representation and pre-train two sub-models respectively. Finally, the extraction network and the abstraction network are bridged by reinforcement learning. To verify the performance of the model, we compare it with the current popular automatic text summary model on the CNN/Daily Mail dataset, and use the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metrics as the evaluation method. Extensive experimental results show that the accuracy of the model is improved obviously.

Experimental analysis of multiple criteria for extractive multi-document text summarization

Automatic text summarization methods are increasingly needed in different fields of knowledge. In the scientific literature, generic extractive multi-document text summarization can be formulated as an optimization problem which involves several criteria. Only two criteria have been considered simultaneously, i.e., content coverage and redundancy reduction, whereas the other ones, relevance and coherence have been considered separately. Therefore, there is a lack of studies comparing the performance of different criteria. For this reason, a comparative study of the different criteria suitable for generic extractive multi-document text summarization is performed here. All possible combinations of two, three, and four criteria have been considered within a multi-objective optimization context. Experiments have been carried out based on datasets from Document Understanding Conferences (DUC), and the combinations of objective functions have been compared and evaluated with Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. Redundancy reduction has been demonstrated as an indispensable criterion, being the coherence the least significant and efficient criterion. The combination that includes content coverage, redundancy reduction, and relevance obtains the most balanced results in terms of average ROUGE and execution time.

An Indicator-based Multi-Objective Optimization Approach Applied to Extractive Multi-Document Text Summarization

The massive amount of textual information on the Internet makes that automatic text summarization methods are becoming very important nowadays. Particularly, the purpose of extractive multi-document text summarization methods is to generate summaries from a document collection by, simultaneously, covering the main content and reducing the redundant information. In the scientific literature, these summarization methods have been addressed through optimization techniques, being almost all of them single-objective optimization approaches. Nevertheless, multi-objective approaches have gained importance because their results have improved the single-objective ones.On the other hand, in the multi-objective optimization field, indicator-based approaches have obtained good results in other applications. For this reason, an Indicator-based Multi-Objective Artificial Bee Colony (IMOABC) algorithm has been developed and applied to the extractive multi-document text summarization problem. Experiments have been carried out based on Document Understanding Conferences (DUC) datasets, and the obtained results have been evaluated and compared with Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. The results have improved to the ones in the scientific literature between 7.37% and 40.76% and 2.59% and 11.24% for ROUGE-2 and ROUGE-L, respectively.

Multi-document extractive text summarization: A comparative assessment on features

Text summarization is the process of generating a brief version of a text that preserves the salient information of the text. For information retrieval, it is a good dimension reduction solution. In addition, it reduces the required reading time. This study focused on extracting informative summaries from multiple documents using commonly used hand-crafted features from the literature. The first investigation focused on the generation of a feature vector. The features were the number of sentences, term frequency, similarity with the title, term frequency-inverse sentence frequency, sentence position, sentence length, sentence–sentence similarity, bushy-path results, phrases of the sentence, proper nouns, n-gram co-occurrence, and length of the document. Secondly, several combinations of these features were examined and a shallow multi-layer perceptron and two differently modeled fuzzy inference systems were used to extract salient sentences from texts in the Document Understanding Conference (DUC) dataset. The summarization performances of these models were evaluated using original classification performance metrics, and recall-oriented understudy for gisting evaluation (ROUGE)-n. This study recommended the use of fuzzy systems based on a feature vector and a fuzzy rule set for extractive text summarization. The extraction methods were evaluated against a changing compression ratio. Results of experiments showed that the implemented neural model tended to incorrectly infer sentences that were not considered salient by human annotators. However, for distinguishing between summary-worthy and summary-unworthy sentences, the fuzzy inference systems performed better than the utilized neural network, as well as better than the existing fuzzy inference-based text summarization approaches in the literature.

Fuzzy AHP approach for legal judgement summarization

Legal documents are generally big and complex documents because of specific vocabulary, semantics and structure. One of the major challenges in legal processing systems is to generate summary of legal judgements. Till date, in most of the legal systems, the summary of judgements is produced manually by legal experts which are then used by Lawyers, Judges and other legal professionals. The manual process of summarization is very inefficient and time-consuming. Automatic text summarization (ATS) is the process of reducing the content of a textual document, while retaining the core description of text through the use of appropriate tool. The present work proposes a novel Fuzzy Analytical Hierarchical process (FAHP) based feature weighting scheme which helps in producing an efficient and effective summary of legal judgement. Model is applied on a number of legal judgements taken from Indian IT Act. Validation of the model is done using ROUGE (Recall-Oriented Understudy for Gisting Evaluation) tool with recall, precision, and f-measure as performance measures. The generated summaries are further assessed by legal experts and are found to be more promising than the summaries generated by traditional approaches.

Extractive document summarization using an adaptive, knowledge based cognitive model

Document summarization involves identifying the salient text in a document and creating a representative summary. The event-index cognitive model describes the human cognitive processes involved in generating situational models from a given text to comprehend that text. This paper proposes an adaptive, knowledge based event-index cognitive model that creates an extractive summary for a given document. The proposed cognitive model uses the hierarchical human memory model and emotion to create the extractive summary. The performance of the proposed cognitive model was compared with the existing state-of-art methods using the DUC (Document Understanding Conference) 2001 dataset and the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) summary evaluation tool. Under ROUGE-N, ROUGE-L, ROUGE-S and ROUGE-SU evaluations, the proposed cognitive model for document summarization achieved an improvement of 25%, 12%, 24% and 18% in mean F-measure values. The results are statistically verified as significantly better than existing state-of-art methods using one way ANOVA with post-hoc Tukey-HSD test. In addition, this paper discusses the possible future research directions in document summarization based on the proposed cognitive model.

Comparison of automatic methods for reducing the Pareto front to a single solution applied to multi-document text summarization

Nowadays, automatic text summarization methods are needed in many different contexts. Extractive multi-document summarization approaches are intended to, simultaneously, synthesize the main content of a document collection and reduce the redundant information. Multi-objective optimization seems to be the natural way to address this problem. The result of applying a multi-objective optimization approach is a set containing many non-dominated solutions or Pareto set. However, only one relevant summary is needed, so a post-Pareto analysis must be performed to reduce this set to a single solution. Several methods have been considered to address this task, including those related to the largest hypervolume, the consensus solution, the shortest distance to the ideal point, and the shortest distance to all points. The approaches have been tested and compared through different experiments, which have been performed using datasets from Document Understanding Conferences (DUC). The methods have been compared with Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. The results show that consensus is the method providing the best average values. The results provided by the consensus method improve the ones obtained with the other methods between 10.68% and 27.32% in ROUGE scores.

CIBS: A biomedical text summarizer using topic-based sentence clustering.

Automatic text summarizers can reduce the time required to read lengthy text documents by extracting the most important parts. Multi-document summarizers should produce a summary that covers the main topics of multiple related input texts to diminish the extent of redundant information. In this paper, we propose a novel summarization method named Clustering and Itemset mining based Biomedical Summarizer (CIBS). The summarizer extracts biomedical concepts from the input documents and employs an itemset mining algorithm to discover main topics. Then, it applies a clustering algorithm to put the sentences into clusters such that those in the same cluster share similar topics. Selecting sentences from all the clusters, the summarizer can produce a summary that covers a wide range of topics of the input text. Using the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) toolkit, we evaluate the performance of the CIBS method against four summarizers including a state-of-the-art method. The results show that the CIBS method can improve the performance of single- and multi-document biomedical text summarization. It is shown that the topic-based sentence clustering approach can be effectively used to increase the informative content of summaries, as well as to decrease the redundant information.

Graph-based biomedical text summarization: An itemset mining and sentence clustering approach

ObjectiveAutomatic text summarization offers an efficient solution to access the ever-growing amounts of both scientific and clinical literature in the biomedical domain by summarizing the source documents while maintaining their most informative contents. In this paper, we propose a novel graph-based summarization method that takes advantage of the domain-specific knowledge and a well-established data mining technique called frequent itemset mining. MethodsOur summarizer exploits the Unified Medical Language System (UMLS) to construct a concept-based model of the source document and mapping the document to the concepts. Then, it discovers frequent itemsets to take the correlations among multiple concepts into account. The method uses these correlations to propose a similarity function based on which a represented graph is constructed. The summarizer then employs a minimum spanning tree based clustering algorithm to discover various subthemes of the document. Eventually, it generates the final summary by selecting the most informative and relative sentences from all subthemes within the text. ResultsWe perform an automatic evaluation over a large number of summaries using the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. The results demonstrate that the proposed summarization system outperforms various baselines and benchmark approaches. ConclusionThe carried out research suggests that the incorporation of domain-specific knowledge and frequent itemset mining equips the summarization system in a better way to address the informativeness measurement of the sentences. Moreover, clustering the graph nodes (sentences) can enable the summarizer to target different main subthemes of a source document efficiently. The evaluation results show that the proposed approach can significantly improve the performance of the summarization systems in the biomedical domain.

A Hybrid Approach for Arabic Text Summarization Using Domain Knowledge and Genetic Algorithms

Text summarization is the process of producing a shorter version of a specific text. Automatic summarization techniques have been applied to various domains such as medical, political, news, and legal domains proving that adapting domain-relevant features could improve the summarization performance. Despite the existence of plenty of research work in the domain-based summarization in English and other languages, there is a lack of such work in Arabic due to the shortage of existing knowledge bases. In this paper, a hybrid, single-document text summarization approach (abbreviated as (ASDKGA)) is presented. The approach incorporates domain knowledge, statistical features, and genetic algorithms to extract important points of Arabic political documents. The ASDKGA approach is tested on two corpora KALIMAT corpus and Essex Arabic Summaries Corpus (EASC). The Recall-Oriented Understudy for Gisting Evaluation (ROUGE) framework was used to compare the automatically generated summaries by the ASDKGA approach with summaries generated by humans. Also, the approach is compared against three other Arabic text summarization approaches. The (ASDKGA) approach demonstrated promising results when summarizing Arabic political documents with average F-measure of 0.605 at the compression ratio of 40%.

Natural Language Description of Video Streams Using Task-Specific Feature Encoding

In recent years, deep learning approaches have gained great attention due to their superior performance and the availability of high speed computing resources. These approaches are also extended towards the real time processing of multimedia content exploiting its spatial and temporal structure. In this paper, we propose a deep learning-based video description framework which first extracts visual features from video frames using deep convolutional neural networks (CNN) and then pass the derived representations into a long-short term memory-based language model. In order to capture accurate information for human presence, a fine-tuned multi-task CNN is presented. The proposed pipeline is end-to-end, trainable, and capable of learning dense visual features along with an accurate framework for the generation of natural language descriptions of video streams. The evaluation is done by calculating Metric for Evaluation of Translation with Explicit ORdering and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scores between system generated and human annotated video descriptions for a carefully designed data set. The video descriptions generated by the traditional feature learning and proposed deep learning frameworks are also compared through the ROUGE scores.

Different approaches for identifying important concepts in probabilistic biomedical text summarization.

Automatic text summarization tools help users in the biomedical domain to acquire their intended information from various textual resources more efficiently. Some of biomedical text summarization systems put the basis of their sentence selection approach on the frequency of concepts extracted from the input text. However, it seems that exploring other measures rather than the raw frequency for identifying valuable contents within an input document, or considering correlations existing between concepts, may be more useful for this type of summarization. In this paper, we describe a Bayesian summarization method for biomedical text documents. The Bayesian summarizer initially maps the input text to the Unified Medical Language System (UMLS) concepts; then it selects the important ones to be used as classification features. We introduce six different feature selection approaches to identify the most important concepts of the text and select the most informative contents according to the distribution of these concepts. We show that with the use of an appropriate feature selection approach, the Bayesian summarizer can improve the performance of biomedical summarization. Using the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) toolkit, we perform extensive evaluations on a corpus of scientific papers in the biomedical domain. The results show that when the Bayesian summarizer utilizes the feature selection methods that do not use the raw frequency, it can outperform the biomedical summarizers that rely on the frequency of concepts, domain-independent and baseline methods.

Extractive multi-document text summarization using a multi-objective artificial bee colony optimization approach

Automatic text summarization methods are increasingly needed nowadays. Extractive multi-document summarization approaches aim to obtain the main content of a document collection at the same time that the redundant information is reduced. This can be addressed from an optimization point of view. There is a lack of multi-objective approaches applied in this context. In this paper, a Multi-Objective Artificial Bee Colony (MOABC) algorithm has been designed and implemented for this task. Experiments have been performed based on datasets from Document Understanding Conference (DUC) and model performances have been evaluated with Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics, as is usual in this knowledge field. The results of the proposed approach show important improvements, i.e., in average, 31.09% (8.43%) and 18.63% (6.09%) of improvement in ROUGE-2 (ROUGE-L) have been obtained with respect to the best single-objective and multi-objective results in the scientific literature. Even more, the proposed approach has been proven to produce more concentrated ROUGE values when the algorithm execution is repeated (between 620.63% and 1333.95% of reduction in the relative dispersion, that is, between 6 and 13 times better), leading to more robust results.

Quantifying the informativeness for biomedical literature summarization: An itemset mining method

ObjectiveAutomatic text summarization tools can help users in the biomedical domain to access information efficiently from a large volume of scientific literature and other sources of text documents. In this paper, we propose a summarization method that combines itemset mining and domain knowledge to construct a concept-based model and to extract the main subtopics from an input document. Our summarizer quantifies the informativeness of each sentence using the support values of itemsets appearing in the sentence. MethodsTo address the concept-level analysis of text, our method initially maps the original document to biomedical concepts using the Unified Medical Language System (UMLS). Then, it discovers the essential subtopics of the text using a data mining technique, namely itemset mining, and constructs the summarization model. The employed itemset mining algorithm extracts a set of frequent itemsets containing correlated and recurrent concepts of the input document. The summarizer selects the most related and informative sentences and generates the final summary. ResultsWe evaluate the performance of our itemset-based summarizer using the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics, performing a set of experiments. We compare the proposed method with GraphSum, TexLexAn, SweSum, SUMMA, AutoSummarize, the term-based version of the itemset-based summarizer, and two baselines. The results show that the itemset-based summarizer performs better than the compared methods. The itemset-based summarizer achieves the best scores for all the assessed ROUGE metrics (R-1: 0.7583, R-2: 0.3381, R-W-1.2: 0.0934, and R-SU4: 0.3889). We also perform a set of preliminary experiments to specify the best value for the minimum support threshold used in the itemset mining algorithm. The results demonstrate that the value of this threshold directly affects the accuracy of the summarization model, such that a significant decrease can be observed in the performance of summarization due to assigning extreme thresholds. ConclusionCompared to the statistical, similarity, and word frequency methods, the proposed method demonstrates that the summarization model obtained from the concept extraction and itemset mining provides the summarizer with an effective metric for measuring the informative content of sentences. This can lead to an improvement in the performance of biomedical literature summarization.

An Ontology-based Summarization System for Arabic Documents (OSSAD)

With the problem of increased web resources and the huge amount of information available, the necessity of having automatic summarization systems appeared. Since summarization is needed the most in the process of searching for information on the web, where the user aims at a certain domain of interest according to his query, domain-based summaries would serve the best. Despite the existence of plenty of research work in the domain-based summarization in English, there is lack of them in Arabic due to the shortage of existing knowledge bases. In this paper an Ontology-based Summarization System for Arabic Documents, OSSAD, is introduced. Domain knowledge is extracted from an Arabic corpus and represented by topic related concepts/keywords and the lexical relations among them. The user’s query is first expanded by using the Arabic WordNet and then by adding the domain-specific knowledge base to the expansion. For summarization, decision tree algorithm (C4.5) is used, which was trained by a set of features extracted from the original documents. For the testing dataset, Essex Arabic Summaries Corpus (EASC) was used. Recall Oriented Understudy for Gisting Evaluation (ROUGE) was used to compare OSSAD summaries with the human summaries along with other automatic summarization systems, showing that the proposed approach demonstrated promising results.

A semantic graph-based approach to biomedical summarisation

Access to the vast body of research literature that is available in biomedicine and related fields may be improved by automatic summarisation. This paper presents a method for summarising biomedical scientific literature that takes into consideration the characteristics of the domain and the type of documents. To address the problem of identifying salient sentences in biomedical texts, concepts and relations derived from the Unified Medical Language System (UMLS) are arranged to construct a semantic graph that represents the document. A degree-based clustering algorithm is then used to identify different themes or topics within the text. Different heuristics for sentence selection, intended to generate different types of summaries, are tested. A real document case is drawn up to illustrate how the method works. A large-scale evaluation is performed using the recall-oriented understudy for gisting-evaluation (ROUGE) metrics. The results are compared with those achieved by three well-known summarisers (two research prototypes and a commercial application) and two baselines. Our method significantly outperforms all summarisers and baselines. The best of our heuristics achieves an improvement in performance of almost 7.7 percentage units in the ROUGE-1 score over the LexRank summariser (0.7862 versus 0.7302). A qualitative analysis of the summaries also shows that our method succeeds in identifying sentences that cover the main topic of the document and also considers other secondary or "satellite" information that might be relevant to the user. The method proposed is proved to be an efficient approach to biomedical literature summarisation, which confirms that the use of concepts rather than terms can be very useful in automatic summarisation, especially when dealing with highly specialised domains.

Metodologia de acesso a dissertações de mestrado de tradução por estrangeiros, uma abordagem preliminar

O artigo aborda a primeira etapa de uma proposta metodológica para facilitar o acesso de estrangeiros a dissertações de mestrado escritas em português disponíveis em repositórios digitais. Essa etapa consiste na criação e formatação para tratamento computacional dos corpora de dissertações, na seleção de um sumarizador automatico da língua portuguesa, no processamento das sumarizações automáticas e na comparação dos sumários assim obtidos com os resumos das dissertações. São também apresentadas e analisadas as métricas obtidas através do programa ROUGE (Recall-Oriented Understudy for Gisting Evaluation). É mencionada igualmente a etapa final do trabalho (que fará uso de tecnologias da língua tais como tradução auxiliada por computador, alinhamento de textos), a partir do corpus ora construído, a ser usado para completar o desenvolvimento da metodologia proposta, a qual poderá ser testada em salas de referência de bibliotecas.