Automatic Summarization System Research Articles

Contextual embedded text summarizer system: A hybrid approach

AbstractSelecting crucial sentences from a document is a pivotal task in automatic text summarization systems. Abstractive summarization involves rephrasing key content through advanced natural language techniques, generating a concise, new text conveying critical information. Conversely, extractive summarization reproduces important material from the original text. In the proposed method, a hybrid ensemble approach combines BERTsum for extractive summarization and Longformer2Roberta for abstractive summarization for generating a contextual semantic rich summary for a huge collection of text. These proposed system‐generated summaries were evaluated against reference summaries using the ROUGE package at three rouge levels (Rouge‐1, Rouge‐2, and Rouge‐L). The Proposed contextual embedded hybrid text summarization model has shown significant performance improvement in multiple levels of Rouge score and word mover distance (WMD) of generated summary with a reference summary. The proposed hybrid model demonstrates superior performance over existing state‐of‐the‐art summarizing models on three distinct datasets CNN dataset, WikiSum, and Gigaword dataset. The proposed hybrid model as a text summarizer involves leveraging its capabilities to process longer sequences of text with domain‐specific contextual summaries. This transformers‐based text summarization model has great potential in developing expert systems in various research domains such as health decision support systems, the education sector, customer support chatbots, financial analysis investment recommendations, and financial assistance.

Comprehensive Survey of Abstractive Text Summarization Techniques

Text summarization using pre-trained encoders has become a crucial technique for efficiently managing large volumes of text data. The rise of automatic summarization systems addresses the need to process ever-increasing data while meeting user-specific requirements. Recent scientific research highlights significant advancements in abstractive summarization, with a particular focus on neural network-based methods. A detailed review of various neural network models for abstractive summarization identifies five key components essential to their design: encoder-decoder architecture, mechanisms, training strategies and optimization algorithms, dataset selection, and evaluation metrics. Each of these elements is pivotal in enhancing the summarization process. This study aims to provide a thorough understanding of the latest developments in neural network-based abstractive summarization models, offering insights into the evolving field and underscoring the associated challenges. Qualitative analysis using a concept matrix reveals common design trends in contemporary neural abstractive summarization systems. Notably, BERT-based encoder-decoder models have emerged as leading innovations, representing the most recent progress in the field. Based on the insights from this review, the study recommends integrating pre-trained language models with neural network techniques to achieve optimal performance in abstractive summarization tasks. As the volume of online information continues to surge, the field of automatic text summarization has garnered significant attention within the Natural Language Processing (NLP) community. Spanning over five decades, researchers have approached this problem from diverse angles, exploring various domains and employing a multitude of paradigms. This survey aims to delve into some of the most pertinent methodologies, focusing on both single-document and multiple-document summarization techniques, with a particular emphasis on empirical methods and extractive approaches. Additionally, the survey explores promising strategies that target specific intricacies of the summarization task. Notably, considerable attention is dedicated to the automatic evaluation of summarization systems, recognizing its pivotal role in guiding future research endeavors.

Automatic News Summarization using Web Scraping

In an era of information overload, accessing relevant news content efficiently is crucial. However, language barriers often hinder users from accessing news articles in languages they understand. To address this challenge, we propose an automatic news summarization and translation system. This system aggregates news articles from multiple URLs, summarizes them into concise summaries, and translates the summaries into the user's preferred language. By leveraging web scraping, natural language processing, and translation integration techniques, our system aims to provide users with access to relevant news content in their preferred language, thereby overcoming language barriers and fostering global connectivity. Keywords— automatic news summarization, translation integration, web scraping, natural language processing, language barriers, information accessibility, global connectivity, user preferences, multilingual content, web-based news aggregation

A Graph-Based Keyword Extraction Method for Academic Literature Knowledge Graph Construction

In this paper, we construct an academic literature knowledge graph based on the relationship between documents to facilitate the storage and research of academic literature data. Keywords are an important type of node in the knowledge graph. To solve the problem that there are no keywords in some documents for several reasons in the process of knowledge graph construction, an improved keyword extraction algorithm called TP-CoGlo-TextRank is proposed by using word frequency, position, word co-occurrence frequency, and a word embedding model. By combining the word frequency and position in the document, the importance of words is distinguished. By introducing the GloVe word-embedding model, which brings the external knowledge of documents into the TextRank algorithm, and combining the internal word co-occurrence frequency in the documents, the word-adjacency relationship is transferred non-uniformly. Finally, the words with the highest scores are combined into phrases if they are adjacent in the original text. The validity of the TP-CoGlo-TextRank algorithm is verified by experiments. On this basis, the Neo4j graph database is used to store and display the academic literature knowledge graph, to provide data support for research tasks such as text clustering, automatic summarization, and question-answering systems.

Implementasi Algoritma Graf dan Algoritma Genetika pada Peringkasan Single Document

In today's technological advancements, finding information is easier and faster. But not a little information that is not true or commonly referred to as hoaxes. Therefore, information must be obtained from several sources to ensure the accuracy of the information. Automatic Text Summarization System is a system used for text based document summarization. This system can help find the core of a news document, so it does not require much time to read. Researchers use Graph Algorithms and Genetic Algorithms in system development. From the test results obtained by the accuracy of the system produced by the system with manual numbers have a cosine similarity value of 71.21%. This can prove that the system built can be used by users because the results of tests carried out get high accuracy values.

Automatic Text Summarization System for Scientific Papers on the basis of T5 Model, on-the-fly Constructed Corpus, and Citation

Automatic Text Summarization System for Scientific Papers on the basis of T5 Model, on-the-fly Constructed Corpus, and Citation

Automated Abstractive Text Summarization Using Multidimensional Long Short-Term Memory

Text summarization is a technique for condensing and presenting the most crucial information from a larger text in a succinct manner. It is a challenging task for human beings to manually summarize extensive documents due to the time required and cognitive effort involved. The increasing amount of textual data generated in recent years has led to a greater demand for efficient and automated text summarization methods. This study explores the utilization of deep learning and neural network concepts to create an automatic abstractive text summarization system for product reviews using multi-dimensional long short-term memory. Since, reading product reviews can be a time-consuming task, especially given the abundance and diversity of reviews written by numerous individuals. The proposed model was evaluated by comparing the generated summaries to reference summaries using metrics such as Rouge. Further, the performance of the model is compared with the prevalent state-of-the-art techniques. The proposed Multi-Dimensional LSTM model was evaluated using the Amazon food review dataset and compared with four other text summarization methods. The proposed model outperformed the prevalent models in terms of ROUGE-1 and ROUGE-L scores but was slightly outperformed in terms of the ROUGE-2 score.

Automatic Update Summarization by a Multiobjective Number-One-Selection Genetic Approach.

Currently, the explosive growth of the information available on the Internet makes automatic text summarization systems increasingly important. A particularly relevant challenge is the update summarization task. Update summarization differs from traditional summarization in its dynamic nature. While traditional summarization is static, that is, the document collections about a specific topic remain unchanged, update summarization addresses dynamic document collections based on a specific topic. Therefore, update summarization consists of summarizing the new document collection under the assumption that the user has already read a previous summarization and only the new information is interesting. The multiobjective number-one-selection genetic algorithm (MONOGA) has been designed and implemented to address this problem. The proposed algorithm produces a summary that is relevant to the user's given query, and it also contains updates information. Experiments were conducted on Text Analysis Conference (TAC) datasets, and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics were considered to assess the model performance. The results obtained by the proposed approach outperform those from the existing approaches in the scientific literature, obtaining average percentage improvements between 12.74% and 55.03% in the ROUGE scores.

An Extractive Text Summarization Model for Generating Extended Abstracts of Medical Papers in Turkish

The rapid growth of technology has led to an increase in the amount of data available in the digital environment. This situation makes it difficult for users to find the information they are looking for within this vast dataset, making it time-consuming. To alleviate this difficulty, automatic text summarization systems have been developed as a more efficient way to access relevant information in texts compared to traditional summarization techniques. This study aims to extract extended summaries of Turkish medical papers written about COVID-19. Although scientific papers already have abstracts, more comprehensive summaries are still needed. To the best of our knowledge, automatic summarization of academic studies related to COVID-19 in the Turkish language has not been done before. A dataset was created by collecting 84 Turkish papers from DergiPark. Extended summaries of 2455 and 1708 characters were obtained using widely used extractive methods such as Term Frequency and LexRank algorithms, respectively. The performance of the text summarization model was evaluated based on Recall, Precision, and F-score criteria, and the algorithms were shown to be effective for Turkish. The results of the study showed similar accuracy rates to previous studies in the literature.

Automatic Summarization for Academic Articles using Deep Learning and Reinforcement Learning with Viewpoints

The purpose of this research is to develop a Viewpoint Refinement in Automatic Summarization (VPRAS) system for research articles. The system will reflect viewpoints of survey to support surveys stage for researchers and students. We collect academic articles using web scraping technology and construct training data by combining sections and sentences through analysis of the article's PDF structure. We use machine learning techniques to classify sentences in Japanese articles into viewpoints. In addition to supervised learning, we introduce reinforcement learning and Dynamic Programming (DP) to extract important sentences for each viewpoint. Finally, we implemented an agent to automatically extract summary sentences based on a reward function.

GA-SCS: Graph-Augmented Source Code Summarization

Automatic source code summarization system aims to generate a valuable natural language description for a program, which can facilitate software development and maintenance, code categorization, and retrieval. However, previous sequence-based research did not consider the long-distance dependence and highly structured characteristics of source code simultaneously. In this article, we present a Transformer-based Graph-Augmented Source Code Summarization (GA-SCS), which can effectively incorporate inherent structural and textual features of source code to generate an effective code description. Specifically, we develop a graph-based structure feature extraction scheme leveraging abstract syntax tree and graph attention networks to mine global syntactic information. And then, to take full advantage of the lexical and syntactic information of code snippets, we extend the original attention to a syntax-informed self-attention mechanism in our encoder. In the training process, we also adopt a reinforcement learning strategy to enhance the readability and informativity of generated code summaries. We utilize the Java dataset and Python dataset to evaluate the performance of different models. Experimental results demonstrate that our GA-SCS model outperforms all competitive methods on BLEU, METEOR, ROUGE, and human evaluations.

Automatic multi-documents text summarization by a large-scale sparse multi-objective optimization algorithm

AbstractDue to the exponential overflow of textual information in various fields of knowledge and on the internet, it is very challenging to extract important information or to generate a summary from some multi-document collection in a specific field. With such a gigantic amount of textual content, human text summarization becomes impractical since it is expensive and consumes a lot of time and effort. So, developing automatic text summarization (ATS) systems is becoming increasingly essential. ATS approaches are either extractive or abstractive. The extractive approach is simpler and faster than the abstractive approach. This work proposes an extractive ATS system that aims to extract a small subset of sentences from a large multi-document text. First, the whole text is preprocessed by applying some natural language processing techniques such as sentences segmentation, words tokenization, removal of stop-words, and stemming to provide a structured representation of the original document collection. Based on this structured representation, the ATS problem is formulated as a multi-objective optimization (MOO) problem that optimizes the extracted summary to maintain the coverage of the main text content while avoiding redundant information. Secondly, an evolutionary sparse multi-objective algorithm is developed to solve the formulated large-scale MOO. The output of this algorithm is a set of non-dominated summaries (Pareto front). A novel criterion is proposed to select the target summary from the Pareto front. The proposed ATS system has been examined using (DUC) datasets, and the output summaries have been evaluated using (ROUGE) metrics and compared with the literature.

An Empirical Survey on Long Document Summarization: Datasets, Models, and Metrics

Long documents such as academic articles and business reports have been the standard format to detail out important issues and complicated subjects that require extra attention. An automatic summarization system that can effectively condense long documents into short and concise texts to encapsulate the most important information would thus be significant in aiding the reader’s comprehension. Recently, with the advent of neural architectures, significant research efforts have been made to advance automatic text summarization systems, and numerous studies on the challenges of extending these systems to the long document domain have emerged. In this survey, we provide a comprehensive overview of the research on long document summarization and a systematic evaluation across the three principal components of its research setting: benchmark datasets, summarization models, and evaluation metrics. For each component, we organize the literature within the context of long document summarization and conduct an empirical analysis to broaden the perspective on current research progress. The empirical analysis includes a study on the intrinsic characteristics of benchmark datasets, a multi-dimensional analysis of summarization models, and a review of the summarization evaluation metrics. Based on the overall findings, we conclude by proposing possible directions for future exploration in this rapidly growing field.

Deep Transformer Language Models for Arabic Text Summarization: A Comparison Study

Large text documents are sometimes challenging to understand and time-consuming to extract vital information from. These issues are addressed by automatic text summarizing techniques, which condense lengthy texts while preserving their key information. Thus, the development of automatic summarization systems capable of fulfilling the ever-increasing demands of textual data becomes of utmost importance. It is even more vital with complex natural languages. This study explores five State-Of-The-Art (SOTA) Arabic deep Transformer-based Language Models (TLMs) in the task of text summarization by adapting various text summarization datasets dedicated to Arabic. A comparison against deep learning and machine learning-based baseline models has also been conducted. Experimental results reveal the superiority of TLMs, specifically the PEAGASUS family, against the baseline approaches, with an average F1-score of 90% on several benchmark datasets.

SGCSumm: An extractive multi-document summarization method based on pre-trained language model, submodularity, and graph convolutional neural networks

The increase in online text generation by humans and machines needs automatic text summarization systems. Recent research studies commonly use deep learning, besides sentence embedding and feature learning mechanisms, to find a solution for text summarization. But they ignore the fact that while finding the optimal solution for extractive text summarization is NP-hard, how do they ensure the quality of their solution? In our previous work, an extractive summarizer, called DSNSum, was proposed based on deep submodular network (DSN) that uses handcrafted features. It leverages submodularity to guarantee a minimum bound for performance. In this paper, submodular graph convolutional summarizer (SGCSumm), an extractive multi-document summarization method, is presented where DSN is used to guarantee a minimum performance bound. It addresses DSNSum’s shortcomings and makes enhancements that yield a summarizer supporting sentence embedding and graph structure-aware feature learning. It takes a formal approach to show how the enhancements are possible without losing the guarantee of a minimum bound for performance. Finally, to evaluate the performance of the proposed summarization method, the DUC 2004 and DailyMail/CNN datasets are used. The experimental results show that the performance of SGCSumm is comparable to that of state-of-the-art summarization methods in terms of ROUGE scores.

Comparative Analysis of Automated Text Summarization Techniques: The Case of Ethiopian Languages

Nowadays, there is an abundance of information available from both online and offline sources. For a single topic, we can get more than hundreds of sources containing a wealth of information. The ability to extract or generate a summary of popular content allows users to quickly search for content and obtain preliminary data in the shortest amount of time. Manually extracting useful information from them is a difficult task. Automatic text summarization (ATS) systems are being developed to address this issue. Text summarization is the process of extracting useful information from large documents and compressing it into a summary while retaining all the relevant contents. This review paper provides a broad overview of ATS research works in various Ethiopian languages such as Amharic, Afan Oromo, and Tigrinya using different text summarization approaches. The work has identified the novel and recommended state-of-the-art techniques and methods for future researchers in the area and provides knowledge and useful support to new researchers in this field by providing a concise overview of the various feature extraction methods and classification techniques required for different types of ATS approaches applied to the Ethiopian languages. Finally, different recommendations for future researchers have been forwarded.

VLSP 2022 Abmusu Task Dataset: A Resource for Vietnamese Abstractive Multi-Document Summarization

The performance of automatic summarization systems has improved significantly with the development of supervised approaches. However, in the Vietnamese abstractive multi-document summarization task, the available datasets are insufficient for training the model. With this motivation, we contribute a new gold standard Vietnamese abstractive multi-document summarization dataset, named Abmusu. Following the collecting and clustering of articles, we have built a hierarchical annotation process to generate summaries, with three roles: annotator, supervisor, and curator. As a result, the dataset contains 600 news clusters formed from 1839 articles and the corresponding human-generated summaries. To the best of our knowledge, Abmusu dataset is the biggest dataset for Vietnamese abstractive multi-document summarization that is freely available for research. Moreover, summaries are more concise, making it challenging to train the summarization models. We also used various summarization baselines to benchmark the Abmusu dataset.

Ensemble of Support Vector Machine and Ontological Structures to Generate Abstractive Text Summarization

Automatic summarization systems are much needed to lessen the information overload which is being faced by people due to exponential growth of data on World Wide Web. These systems choose the most significant part of the text from a single document or multiple documents and present the compressed surrogate form of the complete information which was intended to be conveyed. In this research paper, we propose an approach to generate summary from a given text first by extracting the most relevant sentences and then making further concise by creating ontological structures of these sentences and then generating the abstractive summary from these structures. Our proposed system is evaluated with DUC 2002 data set and it is found that the performance of this system as evaluated using ROUGE-1 is 58.175 which is better than other state of the art systems. The values reported in the experimental process of the research report the significant contribution of this innovative method.

Automatic Text Summarization of Biomedical Text Data: A Systematic Review

In recent years, the evolution of technology has led to an increase in text data obtained from many sources. In the biomedical domain, text information has also evidenced this accelerated growth, and automatic text summarization systems play an essential role in optimizing physicians’ time resources and identifying relevant information. In this paper, we present a systematic review in recent research of text summarization for biomedical textual data, focusing mainly on the methods employed, type of input data text, areas of application, and evaluation metrics used to assess systems. The survey was limited to the period between 1st January 2014 and 15th March 2022. The data collected was obtained from WoS, IEEE, and ACM digital libraries, while the search strategies were developed with the help of experts in NLP techniques and previous systematic reviews. The four phases of a systematic review by PRISMA methodology were conducted, and five summarization factors were determined to assess the studies included: Input, Purpose, Output, Method, and Evaluation metric. Results showed that 3.5% of 801 studies met the inclusion criteria. Moreover, Single-document, Biomedical Literature, Generic, and Extractive summarization proved to be the most common approaches employed, while techniques based on Machine Learning were performed in 16 studies and Rouge (Recall-Oriented Understudy for Gisting Evaluation) was reported as the evaluation metric in 26 studies. This review found that in recent years, more transformer-based methodologies for summarization purposes have been implemented compared to a previous survey. Additionally, there are still some challenges in text summarization in different domains, especially in the biomedical field in terms of demand for further research.

Extensible Model for Turkish Single Document Summarization

The development of Automatic Text Summarization systems with the help of a computer is one of the most challenge for two decades. The amount of information currently available and the time problem of those who need information reveal the importance of such summaries.
 Generally, successful summaries are produced by people. However, it is obvious that human resources will not be enough to summarize the daily flow of information. Today, work is focused on designing and improving systems that automatically extract the abstract from the text.
 In this study, a summarization system based on a parametric method has been proposed and implemented. It summarizes a single Turkish text. The compression ratio of the summary can be determined by the user. The system also offers title and keyword support to the user. The evaluation reveals an average model, however the hardness of Turkish language considered it is promising.