Extractive Text Summarization Research Articles

Hybrid model for extractive single document summarization: utilizing BERTopic and BERT model

Extractive text summarization has been a popular research area for many years. The goal of this task is to generate a compact and coherent summary of a given document, preserving the most important information. However, current extractive summarization methods still face several challenges such as semantic drift, repetition, redundancy, and lack of coherence. A novel approach is presented in this paper to improve the performance of an extractive summarization model based on bidirectional encoder representations from transformers (BERT) by incorporating topic modeling using the BERTopic model. Our method first utilizes BERTopic to identify the dominant topics in a document and then employs a BERT-based deep neural network to extract the most salient sentences related to those topics. Our experiments on the cable news network (CNN)/daily mail dataset demonstrate that our proposed method outperforms state-of-the-art BERT-based extractive summarization models in terms of recall-oriented understudy for gisting evaluation (ROUGE) scores, which resulted in an increase of 32.53% of ROUGE-1, 47.55% of ROUGE-2, and 16.63% of ROUGE-L when compared to baseline BERT-based extractive summarization models. This paper contributes to the field of extractive text summarization, highlights the potential of topic modeling in improving summarization results, and provides a new direction for future research.

A study of extractive summarization of long documents incorporating local topic and hierarchical information

In recent years, the transformer-based language models have achieved remarkable success in the field of extractive text summarization. However, there are still some limitations in this kind of research. First, the transformer language model usually regards the text as a linear sequence, ignoring the inherent hierarchical structure information of the text. Second, for long text data, traditional extractive models often focus on global topic information, which poses challenges in how they capturing and integrating local contextual information within topic segments. To address these issues, we propose a long text extractive summarization model that employs a local topic information extraction module and a text hierarchical extraction module to capture the local topic information and document's hierarchical structure information of the original text. Our approach enhances the ability to determine whether a sentence belongs to the summary. In this experiment, ROUGE score is used as the experimental evaluation index, and evaluates the model on three large public datasets. Through experimental validation, the model demonstrates superior performance in terms of ROUGE-1, ROUGE-2, and ROUGE-L scores compared to current mainstream summarization models, affirming the effectiveness of incorporating local topic information and document hierarchical structure into the model.

Pre-Training Clustering Models to Summarize Vietnamese Texts

Our investigation aims at pre-training clustering models to summarize Vietnamese texts. For this purpose, we create a large-scale dataset by collecting Vietnamese articles from newspaper websites and extracting the plain text to build the dataset, including 1,101,101 documents. We propose a new single-document extractive text summarization model based on clustering models. Our proposal clusters the documents with the hard clustering k-means algorithm and the soft clustering LDA (Latent Dirichlet Allocation) algorithm. Then, based on the pre-training clustering models, a summary model is used to select the salient sentence in the input text to construct the summary. The empirical results showed that our summary model achieved 51.22% ROUGE-1, 17.62% ROUGE-2 and 29.16% ROUGE-L on the testing set. Besides the traditional word representation such as BoW (Bag-of-Words), we also use the word meaning-based tools like FastText and BERT (Bidirectional Encoder Representations from Transformers) in our model. The additional benefit of our proposed extractive summary model is that the output summary is a long-text, readable document. Furthermore, the model’s architecture is straightforward, easy to understand and runs on cost-efficient resources like arm CPU and GPU too.

Enhancing Study Experience using Handwritten Character and Digit Recognition and Text Summarization

The integration of Handwritten characters and, Digit Recognition and Deep Learning in education heralds a transformative era in learning methodologies. This abstract delves into the multifaceted benefits derived from the amalgamation of these technologies, redefining the educational landscape. Handwritten characters and Digit Recognition technology facilitates the seamless digitization of handwritten content, transcending the limitations of manual note-taking. Its introduction into educational frameworks enhances accessibility, promotes organization, and augments the searchability of diverse educational materials. Deep Learning, acting in tandem with Handwriting Recognition, amplifies these advantages manifold. Deep learning powered study assistants offer personalized learning experiences tailored to individual needs, adapting to varied learning styles. Additionally, these assistants facilitate collaborative opportunities, providing real time feedback and evaluation tools that revolutionize the learning process. Key Words: Image recognition, CNN, RNN, LSTM, Neural Networks, SVM, Deep Learning, Convolutional layer, PreLU, ReLU, Text Summarization, Extractive text summarization, Tokenization.

Smooth it up!: Extractive summary coherence enhancement

The paper focuses on the importance of coherence and preserving the breadth of content in summaries generated by the extractive text summarization method. The study utilized the dataset containing 16,772 pairs of extractive and corresponding abstractive summaries of scientific papers specifically tailored to increase text coherence. We smoothed the extractive summaries with a Large Language Model (LLM) fine-tuning approach and evaluated our results by applying the coefficient of variation approach. The statistical significance of the results was assessed using the Kolmogorov-Smirnov test and Z-test. We observed an increase in coherence in the predicted texts, highlighting the effectiveness of our proposed methods.

Improving Faithfulness in Abstractive Text Summarization with EDUs Using BART (Student Abstract)

Abstractive text summarization uses the summarizer’s own words to capture the main information of a source document in a summary. While it is more challenging to automate than extractive text summarization, recent advancements in deep learning approaches and pre-trained language models have improved its performance. However, abstractive text summarization still has issues such as unfaithfulness. To address this problem, we propose a new approach that utilizes important Elementary Discourse Units (EDUs) to guide BART-based text summarization. Our approach showed the improvement in truthfulness and source document coverage in comparison to some previous studies.

A Multi-Metric Model for analyzing and comparing extractive text summarization approaches and algorithms on scientific papers

In today's world, where data and information are increasingly proliferating, text summarization and technologies play a critical role in making large amounts of text data more accessible and meaningful. In business, the news industry, academic research, and many other fields, text summarization helps make quick decisions, access information faster, and manage resources more effectively. Additionally, text summarization research is conducted to further improve these technologies and develop new methods and algorithms to provide better summarization of texts. Therefore, text summarization and research in this field are of great importance in the information age. In this study, a new operating model for text summarization that can be applied to different algorithms is proposed and evaluated. Sixteen summarization algorithms covering six approaches (statistical, graph-based, content-based, pointer-based, position-based, and user-oriented) were implemented and tested on 50 different full-text article datasets. Four evaluation criteria (BLEU, Rouge-N, Rouge-L, METEOR) were used to assess the similarity between the generated summaries and the original summaries. The performance of the algorithms within each approach was averaged and the overall best-performing algorithm was selected. This best algorithm was subjected to further analysis through Topic Modelling and Keyword Extraction to identify key topics and keywords within the summarised text. The proposed model provides a standardized workflow for developing and thoroughly testing summarization algorithms across datasets and evaluation metrics to determine the most appropriate summarization approach. This study demonstrates the effectiveness of the model on a variety of algorithm types and text sources.

Enhancing extractive text summarization using natural language processing with an optimal deep learning model

<abstract> <p>Natural language processing (NLP) performs a vital function in text summarization, a task targeted at refining the crucial information from the massive quantity of textual data. NLP methods allow computers to comprehend and process human language, permitting the development of advanced summarization methods. Text summarization includes the automatic generation of a concise and coherent summary of a specified document or collection of documents. Extracting significant insights from text data is crucial as it provides advanced solutions to end-users and business organizations. Automatic text summarization (ATS) computerizes text summarization by decreasing the initial size of the text without the loss of main data features. Deep learning (DL) approaches exhibited significant performance in abstractive and extractive summarization tasks. This research designed an extractive text summarization using NLP with an optimal DL (ETS-NLPODL) model. The major goal of the ETS-NLPODL technique was to exploit feature selection with a hyperparameter-tuned DL model for summarizing the text. In the ETS-NLPODL technique, an initial step of data preprocessing was involved to convert the input text into a compatible format. Next, a feature extraction process was carried out and the optimal set of features was chosen by the hunger games search optimization (HGSO) algorithm. For text summarization, the ETS-NLPODL model used an attention-based convolutional neural network with a gated recurrent unit (ACNN-GRU) model. Finally, the mountain gazelle optimization (MGO) algorithm was employed for the optimal hyperparameter selection of the ACNN-GRU model. The experimental results of the ETS-NLPODL system were examined under the benchmark dataset. The experimentation outcomes pointed out that the ETS-NLPODL technique gained better performance over other methods concerning diverse performance measures.</p> </abstract>

Interpretable extractive text summarization with meta-learning and BI-LSTM: A study of meta learning and explainability techniques

Text summarization is a widely-researched problem among scholars in the field of natural language processing. Multiple techniques have been proposed to help tackle this problem, yet some of these methodologies may still exhibit limitations such as the requirements for large training datasets, which might not always be possible, but more importantly, the lack of interpretability or transparency of the model. In this paper, we propose using meta-learning algorithm to train a deep learning model for extractive text summarization and then using various explanatory techniques such as SHAP (Shapley, 1953), linear regression (Lederer, 2022), decision trees (Fürnkranz, 2010), and input modification to gain insights into the model’s decision making process. To evaluate the effectiveness of our approach, we will compare it to other popular natural language processing models like BERT (Miller, 2019) or XLNET (Yang et al., 2020) using the ROUGE metrics (Lin, 2004). Overall, our proposed approach provides a promising solution to the limitations of existing methods and a framework for improving the explainability of deep learning models in other natural language processing tasks.

Retracted: N-GPETS: Neural Attention Graph-Based Pretrained Statistical Model for Extractive Text Summarization.

[This retracts the article DOI: 10.1155/2022/6241373.].

Decomposition–based multi-objective differential evolution for extractive multi-document automatic text summarization

The central challenge in Automatic Text Summarization (ATS) involves efficiently generating machine-generated text summaries through optimization algorithms. An ATS is a critical component for systems dealing with textual information processing. However, the current approach faces a significant hurdle due to the computational intensity of the process, particularly when employing complex optimization techniques like swarm intelligence optimization alongside a costly ATS repair operator. While the current approach yields impressive Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics for the generated summary, it struggles with inefficiencies, mainly attributed to the substantial optimization time consumed by the ATS repair operator scheme. In order to address this, a novel solution called Decomposition-based Multi-Objective Differential Evolution (MODE/D) is proposed. It is built upon the foundation of Differential Evolution for Multi-Objective Optimization (DEMO) and the weighted sum method (WS), coupled with an innovative ATS repair operator scheme. Through experimentation on Document Understanding Conferences (DUC) datasets, the novel approach of MODE/D – WS is validated by evaluating the results using ROUGE metrics. The outcomes are twofold: a remarkable reduction in serial execution time and a noteworthy enhancement over existing techniques in the scholarly domain, as evidenced by improved ROUGE-1, ROUGE-2, and ROUGE-L scores.

Extractive social media text summarization based on MFMMR-BertSum

The advancement of computer technology has led to an overwhelming amount of textual information, hindering the efficiency of knowledge intake. To address this issue, various text summarization techniques have been developed, including statistics, graph sorting, machine learning, and deep learning. However, the rich semantic features of text often interfere with the abstract effects and lack effective processing of redundant information. In this paper, we propose the Multi-Features Maximal Marginal Relevance BERT (MFMMR-BertSum) model for Extractive Summarization, which utilizes the pre-trained model BERT to tackle the text summarization task. The model incorporates a classification layer for extractive summarization. Additionally, the Maximal Marginal Relevance (MMR) component is utilized to remove information redundancy and optimize the summary results. The proposed method outperforms other sentence-level extractive summarization baseline methods on the CNN/DailyMail dataset, thus verifying its effectiveness.

Enhanced sentence representation for extractive text summarization: Investigating the syntactic and semantic features and their contribution to sentence scoring

Enhanced sentence representation for extractive text summarization: Investigating the syntactic and semantic features and their contribution to sentence scoring

Extractive Summarization of Telugu Text Using Modified Text Rank and Maximum Marginal Relevance

With the rapid growth of digital content, there is a need for an automatic text summarizer to provide short text from a long text document. Many research works have been presented for extractive text summarization (ETS). This article mainly focuses on the graph-based ETS approach for multiple Telugu text documents. A modified Text-Rank algorithm is employed with the noun and verb count of each sentence in the text as the initial score of each node. To get the optimal features, a novel feature selection algorithm called improved Flamingo Search Algorithm is proposed in this article. Though graph-based ETS is an important approach, the generated summaries are redundant. To reduce the redundancy in the generated summary, maximum marginal relevance is combined with the modified Text-Rank. Different word-embedding techniques such as Fast-Text, Word2vec, TF-IDF, and one-hot encoding are utilized to experiment with the proposed approach. The performance of the proposed text summarization approach is evaluated with BLEU and ROUGE in terms of F-measure, precision, and recall.

Automated Extractive Text Summarization using Genetic and Simulated Annealing Algorithms and their Hybridization

Automated Extractive Text Summarization using Genetic and Simulated Annealing Algorithms and their Hybridization

A parallel optimization and transfer learning approach for summarization in electrical power systems

Transfer learning approaches in natural language processing have been explored and evolved as a potential solution for solving many problems in recent days. The current research on aspect-based summarization shows unsatisfactory accuracy and low-quality generated summaries. Additionally, the potential advantages of combining language models with parallel processing have not been explored in the existing literature. This paper aims to address the problem of aspect-based extractive text summarization using a transfer learning approach and an optimization method based on map reduce. The proposed approach utilizes transfer learning with language models to extract significant aspects from the text. Subsequently, an optimization process using map reduce is employed. This optimization framework includes an in-node mapper and reducer algorithm to generate summaries for important aspects identified by the language model. This enhances the quality of the summary, leading to improved accuracy, particularly when applied to electrical power system documents. By leveraging the strengths of natural language models and parallel data processing techniques, this model presents an opportunity to achieve better text summary generation. The performance metric used is accuracy, measured with the ROUGE tool, incorporating precision, recall and f-measure. The proposed model demonstrates a 6% improvement in scores compared to state-of-the-art techniques.

TxLASM: A novel language agnostic summarization model for text documents

In Natural Language Processing (NLP) domain, the majority of automatic text summarization approaches depend on a prior knowledge of the language and/or the domain of the text being summarized. Such approaches requires language dependent part-of-speech taggers, parsers, databases, pre-structured lexicons, etc. In this research, we propose a novel automatic text summarization model, Text Documents - Language Agnostic Summarization Model (TxLASM), which is able to perform extractive text summarization task in language/domain agnostic manner. TxLASM depends on specific characteristics of the major elements of the text being summarized rather than its domain, context, or language and thus rules out the need for language dependent pre-processing tools, taggers, parsers, lexicons or databases. Within TxLASM, we present a novel technique for encoding the shapes of major text elements (paragraphs, sentences, n-grams and words); moreover, we present language independent preprocessing algorithms to normalize words and perform relative stemming or lemmatization. Those algorithms and its Shape-Coding technique enable the TxLASM to extract intrinsic features of text elements and score them statistically, and subsequently extract a representative summary that is independent of the text language, domain and context. TxLASM was applied on an English and Portuguese benchmark datasets, and the results were compared to twelve state-of-the-art approaches presented in recent literature. In addition, the model was applied on French and Spanish news datasets, and the results were compared to those obtained by standard commercial summarization tools. TxLASM has outperformed all the SOTA approaches as well as the commercial tools in all four languages while maintaining its language and domain agnostic nature.

Summarization of scholarly articles using BERT and BiGRU: Deep learning-based extractive approach

Extractive text summarization involves selecting and combining key sentences directly from the original text, rather than generating new content. While various methods, both statistical and graph-based, have been explored for this purpose, accurately capturing the intended meaning remains a challenge. To address this, researchers are investigating innovative techniques that harness deep learning models like BERT (Bidirectional Encoder Representations from Transformers). However, BERT has limitations in summarizing lengthy documents due to input length constraints. To find a more effective solution, we propose a novel approach. This approach combines the power of BERT, a transformer network pre-trained on extensive self-supervised datasets, with BiGRU (Bidirectional Gated Recurrent Units), a recurrent neural network that captures sequential dependencies within the text for extracting salient information. Our method involves using BERT to generate sentence-level embeddings, which are then fed into the BiGRU network. This allows us to achieve a comprehensive understanding of the complete document’s context. In experimental analysis conducted on arXiv and PubMed datasets, the proposed approach outperformed several state-of-the-art models. It achieved remarkable ROUGE-F scores of (46.7, 19.4, 35.4) and (47.0, 21.3, 39.7) on these datasets respectively. The proposed fusion of BERT and BiGRU significantly enhances extractive text summarization. It shows promising potential for summarizing lengthy documents and benefiting various domains that require concise and informative summaries.

Graph-Based Extractive Text Summarization Sentence Scoring Scheme for Big Data Applications

The recent advancements in big data and natural language processing (NLP) have necessitated proficient text mining (TM) schemes that can interpret and analyze voluminous textual data. Text summarization (TS) acts as an essential pillar within recommendation engines. Despite the prevalent use of abstractive techniques in TS, an anticipated shift towards a graph-based extractive TS (ETS) scheme is becoming apparent. The models, although simpler and less resource-intensive, are key in assessing reviews and feedback on products or services. Nonetheless, current methodologies have not fully resolved concerns surrounding complexity, adaptability, and computational demands. Thus, we propose our scheme, GETS, utilizing a graph-based model to forge connections among words and sentences through statistical procedures. The structure encompasses a post-processing stage that includes graph-based sentence clustering. Employing the Apache Spark framework, the scheme is designed for parallel execution, making it adaptable to real-world applications. For evaluation, we selected 500 documents from the WikiHow and Opinosis datasets, categorized them into five classes, and applied the recall-oriented understudying gisting evaluation (ROUGE) parameters for comparison with measures ROUGE-1, 2, and L. The results include recall scores of 0.3942, 0.0952, and 0.3436 for ROUGE-1, 2, and L, respectively (when using the clustered approach). Through a juxtaposition with existing models such as BERTEXT (with 3-gram, 4-gram) and MATCHSUM, our scheme has demonstrated notable improvements, substantiating its applicability and effectiveness in real-world scenarios.

An Extractive Text Summarization Model for Generating Extended Abstracts of Medical Papers in Turkish

The rapid growth of technology has led to an increase in the amount of data available in the digital environment. This situation makes it difficult for users to find the information they are looking for within this vast dataset, making it time-consuming. To alleviate this difficulty, automatic text summarization systems have been developed as a more efficient way to access relevant information in texts compared to traditional summarization techniques. This study aims to extract extended summaries of Turkish medical papers written about COVID-19. Although scientific papers already have abstracts, more comprehensive summaries are still needed. To the best of our knowledge, automatic summarization of academic studies related to COVID-19 in the Turkish language has not been done before. A dataset was created by collecting 84 Turkish papers from DergiPark. Extended summaries of 2455 and 1708 characters were obtained using widely used extractive methods such as Term Frequency and LexRank algorithms, respectively. The performance of the text summarization model was evaluated based on Recall, Precision, and F-score criteria, and the algorithms were shown to be effective for Turkish. The results of the study showed similar accuracy rates to previous studies in the literature.