Recall-Oriented Understudy For Gisting Evaluation Research Articles

GPT-Driven Radiology Report Generation with Fine-Tuned Llama 3.

The integration of deep learning into radiology has the potential to enhance diagnostic processes, yet its acceptance in clinical practice remains limited due to various challenges. This study aimed to develop and evaluate a fine-tuned large language model (LLM), based on Llama 3-8B, to automate the generation of accurate and concise conclusions in magnetic resonance imaging (MRI) and computed tomography (CT) radiology reports, thereby assisting radiologists and improving reporting efficiency. A dataset comprising 15,000 radiology reports was collected from the University of Medicine and Pharmacy of Craiova's Imaging Center, covering a diverse range of MRI and CT examinations made by four experienced radiologists. The Llama 3-8B model was fine-tuned using transfer-learning techniques, incorporating parameter quantization to 4-bit precision and low-rank adaptation (LoRA) with a rank of 16 to optimize computational efficiency on consumer-grade GPUs. The model was trained over five epochs using an NVIDIA RTX 3090 GPU, with intermediary checkpoints saved for monitoring. Performance was evaluated quantitatively using Bidirectional Encoder Representations from Transformers Score (BERTScore), Recall-Oriented Understudy for Gisting Evaluation (ROUGE), Bilingual Evaluation Understudy (BLEU), and Metric for Evaluation of Translation with Explicit Ordering (METEOR) metrics on a held-out test set. Additionally, a qualitative assessment was conducted, involving 13 independent radiologists who participated in a Turing-like test and provided ratings for the AI-generated conclusions. The fine-tuned model demonstrated strong quantitative performance, achieving a BERTScore F1 of 0.8054, a ROUGE-1 F1 of 0.4998, a ROUGE-L F1 of 0.4628, and a METEOR score of 0.4282. In the human evaluation, the artificial intelligence (AI)-generated conclusions were preferred over human-written ones in approximately 21.8% of cases, indicating that the model's outputs were competitive with those of experienced radiologists. The average rating of the AI-generated conclusions was 3.65 out of 5, reflecting a generally favorable assessment. Notably, the model maintained its consistency across various types of reports and demonstrated the ability to generalize to unseen data. The fine-tuned Llama 3-8B model effectively generates accurate and coherent conclusions for MRI and CT radiology reports. By automating the conclusion-writing process, this approach can assist radiologists in reducing their workload and enhancing report consistency, potentially addressing some barriers to the adoption of deep learning in clinical practice. The positive evaluations from independent radiologists underscore the model's potential utility. While the model demonstrated strong performance, limitations such as dataset bias, limited sample diversity, a lack of clinical judgment, and the need for large computational resources require further refinement and real-world validation. Future work should explore the integration of such models into clinical workflows, address ethical and legal considerations, and extend this approach to generate complete radiology reports.

HybridEval: An Improved Novel Hybrid Metric for Evaluation of Text Summarization

The present work re-evaluates the evaluation method for text summarization tasks. Two state-of-the-art assessment measures e.g., Recall-Oriented Understudy for Gisting Evaluation (ROUGE) and Bilingual Evaluation Understudy (BLEU) are discussed along with their limitations before presenting a novel evaluation metric. The evaluation scores are significantly different because of the length and vocabulary of the sentences, this suggests that the primary restriction is its inability to preserve the semantics and meaning of the sentences and consistent weight distribution over the whole sentence. To address this, the present work organizes the phrases into six different groups and to evaluate “text summarization” problems, a new hybrid approach (HybridEval) is proposed. Our approach uses a weighted sum of cosine scores from InferSent’s SentEval algorithms combined with original scores, achieving high accuracy. HybridEval outperforms existing state-of-the-art models by 10-15% in evaluation scores.

A Transformer-Based Yoruba to English Machine Translation (TYEMT) System with Rouge Score

Automated translation systems for some indigenous Nigerian languages like the Yoruba, have historically been limited by the lack of large, high- quality bilingual text and effective approaches to modeling. This paper presents introduces an approach to bi-directional Yoruba-English text-to-text machine translation utilizing deep learning technique, specifically Transformer models. Transformer models, which utilizes self-attention mechanisms to improve translation quality and efficiency. The system was trained and evaluated on a newly curated Yoruba- English parallel corpus, which significantly augments existing resources. Experimental results demonstrate that the Transformer-based model performs translation accurately and fluently, achieving a ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score improvement of 0.4649. This work not only advances the frontiers of Yoruba-English machine translation but also enriches a wider domain in the field of multilingual Natural Language processing (NLP) by addressing challenges associated with translating between languages with limited resources. Future studies include enhancing the available parallel corpus and exploring hybrid models that combine the strengths of both RNN and Transformer architectures.

An open-source fine-tuned large language model for radiological impression generation: a multi-reader performance study

BackgroundThe impression section integrates key findings of a radiology report but can be subjective and variable. We sought to fine-tune and evaluate an open-source Large Language Model (LLM) in automatically generating impressions from the remainder of a radiology report across different imaging modalities and hospitals.MethodsIn this institutional review board-approved retrospective study, we collated a dataset of CT, US, and MRI radiology reports from the University of California San Francisco Medical Center (UCSFMC) (n = 372,716) and the Zuckerberg San Francisco General (ZSFG) Hospital and Trauma Center (n = 60,049), both under a single institution. The Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score, an automatic natural language evaluation metric that measures word overlap, was used for automatic natural language evaluation. A reader study with five cardiothoracic radiologists was performed to more strictly evaluate the model’s performance on a specific modality (CT chest exams) with a radiologist subspecialist baseline. We stratified the results of the reader performance study based on the diagnosis category and the original impression length to gauge case complexity.ResultsThe LLM achieved ROUGE-L scores of 46.51, 44.2, and 50.96 on UCSFMC and upon external validation, ROUGE-L scores of 40.74, 37.89, and 24.61 on ZSFG across the CT, US, and MRI modalities respectively, implying a substantial degree of overlap between the model-generated impressions and impressions written by the subspecialist attending radiologists, but with a degree of degradation upon external validation. In our reader study, the model-generated impressions achieved overall mean scores of 3.56/4, 3.92/4, 3.37/4, 18.29 s,12.32 words, and 84 while the original impression written by a subspecialist radiologist achieved overall mean scores of 3.75/4, 3.87/4, 3.54/4, 12.2 s, 5.74 words, and 89 for clinical accuracy, grammatical accuracy, stylistic quality, edit time, edit distance, and ROUGE-L score respectively. The LLM achieved the highest clinical accuracy ratings for acute/emergent findings and on shorter impressions.ConclusionsAn open-source fine-tuned LLM can generate impressions to a satisfactory level of clinical accuracy, grammatical accuracy, and stylistic quality. Our reader performance study demonstrates the potential of large language models in drafting radiology report impressions that can aid in streamlining radiologists’ workflows.

An indicator-based multi-objective variable neighborhood search approach for query-focused summarization

Currently, automatic multi-document summarization is an interesting subject in numerous fields of study. As a part of it, query-focused summarization is becoming increasingly important in recent times. These methods can automatically produce a summary based on a query given by the user, including the most relevant information from the query at the same time as the redundancy among sentences is reduced. This can be achieved by developing and applying a multi-objective optimization approach. In this paper, an Indicator-based Multi-Objective Variable Neighborhood Search (IMOVNS) algorithm has been designed, implemented, and tested for the query-focused extractive multi-document summarization problem. Experiments have been carried out with datasets from Text Analysis Conference (TAC). The results were evaluated using the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. IMOVNS has greatly improved the results presented in the scientific literature, providing improvement percentages in ROUGE metric reaching up to 69.24% in ROUGE-1, up to 57.70% in ROUGE-2, and up to 77.37% in ROUGE-SU4 scores. Hence, the proposed IMOVNS offers a promising solution to the query-focused summarization problem, thus highlighting its efficacy and potential for enhancing automatic summarization techniques.

Beyond ROUGE: A Comprehensive Evaluation Metric for Abstractive Summarization Leveraging Similarity, Entailment, and Acceptability

A vast amount of textual information on the internet has amplified the importance of text summarization models. Abstractive summarization generates original words and sentences that may not exist in the source document to be summarized. Such abstractive models may suffer from shortcomings such as linguistic acceptability and hallucinations. Recall-Oriented Understudy for Gisting Evaluation (ROUGE) is a metric commonly used to evaluate abstractive summarization models. However, due to its n-gram-based approach, it ignores several critical linguistic aspects. In this work, we propose Similarity, Entailment, and Acceptability Score (SEAScore), an automatic evaluation metric for evaluating abstractive text summarization models using the power of state-of-the-art pre-trained language models. SEAScore comprises three language models (LMs) that extract meaningful linguistic features from candidate and reference summaries and a weighted sum aggregator that computes an evaluation score. Experimental results show that our LM-based SEAScore metric correlates better with human judgment than standard evaluation metrics such as ROUGE-N and BERTScore.

Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: Benchmark Study.

Comprehensive session summaries enable effective continuity in mental health counseling, facilitating informed therapy planning. However, manual summarization presents a significant challenge, diverting experts' attention from the core counseling process. Leveraging advances in automatic summarization to streamline the summarization process addresses this issue because this enables mental health professionals to access concise summaries of lengthy therapy sessions, thereby increasing their efficiency. However, existing approaches often overlook the nuanced intricacies inherent in counseling interactions. This study evaluates the effectiveness of state-of-the-art large language models (LLMs) in selectively summarizing various components of therapy sessions through aspect-based summarization, aiming to benchmark their performance. We first created Mental Health Counseling-Component-Guided Dialogue Summaries, a benchmarking data set that consists of 191 counseling sessions with summaries focused on 3 distinct counseling components (also known as counseling aspects). Next, we assessed the capabilities of 11 state-of-the-art LLMs in addressing the task of counseling-component-guided summarization. The generated summaries were evaluated quantitatively using standard summarization metrics and verified qualitatively by mental health professionals. Our findings demonstrated the superior performance of task-specific LLMs such as MentalLlama, Mistral, and MentalBART evaluated using standard quantitative metrics such as Recall-Oriented Understudy for Gisting Evaluation (ROUGE)-1, ROUGE-2, ROUGE-L, and Bidirectional Encoder Representations from Transformers Score across all aspects of the counseling components. Furthermore, expert evaluation revealed that Mistral superseded both MentalLlama and MentalBART across 6 parameters: affective attitude, burden, ethicality, coherence, opportunity costs, and perceived effectiveness. However, these models exhibit a common weakness in terms of room for improvement in the opportunity costs and perceived effectiveness metrics. While LLMs fine-tuned specifically on mental health domain data display better performance based on automatic evaluation scores, expert assessments indicate that these models are not yet reliable for clinical application. Further refinement and validation are necessary before their implementation in practice.

Hybrid model for extractive single document summarization: utilizing BERTopic and BERT model

Extractive text summarization has been a popular research area for many years. The goal of this task is to generate a compact and coherent summary of a given document, preserving the most important information. However, current extractive summarization methods still face several challenges such as semantic drift, repetition, redundancy, and lack of coherence. A novel approach is presented in this paper to improve the performance of an extractive summarization model based on bidirectional encoder representations from transformers (BERT) by incorporating topic modeling using the BERTopic model. Our method first utilizes BERTopic to identify the dominant topics in a document and then employs a BERT-based deep neural network to extract the most salient sentences related to those topics. Our experiments on the cable news network (CNN)/daily mail dataset demonstrate that our proposed method outperforms state-of-the-art BERT-based extractive summarization models in terms of recall-oriented understudy for gisting evaluation (ROUGE) scores, which resulted in an increase of 32.53% of ROUGE-1, 47.55% of ROUGE-2, and 16.63% of ROUGE-L when compared to baseline BERT-based extractive summarization models. This paper contributes to the field of extractive text summarization, highlights the potential of topic modeling in improving summarization results, and provides a new direction for future research.

Bidirectional recommendation in HR analytics through text summarization

For over a decade, online job portals have been providing their services to both job seekers and employers in search of hiring opportunities. Because of the high demand for recruitment, it is insufficient to use conventional hiring methods to find a suitable candidate to fill the position. Validating resumes online is challenging due to the potential for manual errors, making the process inherently risky. The bidirectional method comprises named entity recognition (NER) for extracting the required resumes for recruiters. Cosine similarity shows the match percentage of resumes for the job requirements and vice versa. In an attempt to tackle an issue of unregistered words, a solution called decoder attention with pointer network (DA-PN) has been introduced. This method incorporates the use of coverage mechanism to prevent word repetition through generated text summary. DA-PN+Cover method with mixed learning objective (MLO) (DA-PN+Cover+MLO) is utilized for protecting grow of increasing faults in generated text summary. Performance of proposed method is estimated using evaluation indicator recall oriented understudy for gisting evaluation (ROUGE) and attains an average of 27.47 which is comparatively higher than existing methods.

Video Transcripts Summarization using OpenAI Whisper and GPT Model

Abstract: In today’s digital age, a vast amount of video content is generated and shared on the internet every minute. However, extracting relevant information from these videos can be time-consuming and challenging. This is where video transcript summarization comes in, providing a concise summary of video content without the need to watch the entire video. The video transcript summarization system aims to streamline the process of extracting key insights and information from video content by generating concise and informative summaries from their transcripts. In the dynamic landscape of video content, existing approaches to transcript summarization have encountered challenges that limit their effectiveness. Issues such as compromised accuracy, prolonged processing times and a restricted focus on YouTube's captioned videos have prompted the exploration of alternative solutions. This paper introduces a novel approach that tackles these limitations head-on, employing innovative technologies to redefine the landscape of video transcript summarization. To address the identified limitations, we propose an alternative strategy that hinges on the utilization of advanced technologies like OpenAI Whisper Model Automatic Speech Recognition(ASR) for video’s Transcription and Generative Pre-trained Transformer (GPT) model for Summarization. Through the integration of the OpenAI Whisper model for transcription and GPT models for summarization, our proposed approach strives to redefine standards in accuracy, processing efficiency, multilingual support, and exhibit the capability to generate either extractive or abstractive summaries. In the context of natural language processing (NLP) and text summarization ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metrics will be used to evaluate the quality of generated summaries compared to human referenced summaries

Exploring the potential of data augmentation in poetry generation with small-scale corpora

Poetry generation is a complex task in the field of natural language processing, especially when working with small datasets. Data augmentation techniques have been shown to be an effective way to improve the performance of deep learning models in various tasks, including image classification and speech recognition. Therefore, this study focuses on exploring the impact of four different data augmentation methods - Synonym Replacement, Random Insertion, Random Swap, and Random Deletion - on the performance of poetry generation with a small poetry dataset. The results of the study reveal that Random Insertion performed well in terms of Bilingual Evaluation Understudy (BLEU), Recall-Oriented Understudy for Gisting Evaluation (ROUGE), and manual evaluation when compared to other data augmentation techniques. Synonym Replacement performed poorly in all three evaluations. This study confirms the potential value of data augmentation technology in poetry generation tasks and provides innovative perspectives and directions for future research in this area. Data augmentation can be employed to help address the problem of limited data in poetry generation tasks and enhance the efficiency of deep learning models. Future research could focus on exploring more advanced data augmentation techniques and their impact on poetry generation tasks.

An Abstractive Text Summarization using Decoder Attention with Pointer Network

In contemporary times, an abundance of unstructured data prevails across social media and the web. Text summarization, a process aimed at distilling relevant information concisely without altering its core meaning, has become crucial. Manual text summarization is resource-intensive, prompting the exploration of automated methods. While deep learning algorithms, particularly in abstractive text summarization, have gained popularity, further research is needed to understand their integration with semantic-based or structure-based approaches. This research leverages a dataset of 1,735 resumes sourced from Kaggle to propose a novel framework. The framework combines semantic data transformations and deep learning approaches to enhance abstractive text summarization. A key focus is addressing the challenge of handling unregistered words. The proposed solution, Decoder Attention with Pointer Network (DA-PN), is introduced. DA-PN incorporates a coverage mechanism to mitigate word repetition in generated text summaries, thereby improving the quality of summaries. The method aims to safeguard against the propagation of errors in generated text summaries. The performance of the proposed approach is evaluated using the Recall Oriented Understudy for Gisting Evaluation (ROUGE) indicator. Notably, the proposed method achieves an average ROUGE score of 26.28, surpassing existing methods. The emphasis on combining semantic data transformations, deep learning, and addressing specific challenges like word repetition sets this research apart in the field of abstractive text summarization.

Vision-Language Model for Generating Textual Descriptions From Clinical Images: Model Development and Validation Study.

The automatic generation of radiology reports, which seeks to create a free-text description from a clinical radiograph, is emerging as a pivotal intersection between clinical medicine and artificial intelligence. Leveraging natural language processing technologies can accelerate report creation, enhancing health care quality and standardization. However, most existing studies have not yet fully tapped into the combined potential of advanced language and vision models. The purpose of this study was to explore the integration of pretrained vision-language models into radiology report generation. This would enable the vision-language model to automatically convert clinical images into high-quality textual reports. In our research, we introduced a radiology report generation model named ClinicalBLIP, building upon the foundational InstructBLIP model and refining it using clinical image-to-text data sets. A multistage fine-tuning approach via low-rank adaptation was proposed to deepen the semantic comprehension of the visual encoder and the large language model for clinical imagery. Furthermore, prior knowledge was integrated through prompt learning to enhance the precision of the reports generated. Experiments were conducted on both the IU X-RAY and MIMIC-CXR data sets, with ClinicalBLIP compared to several leading methods. Experimental results revealed that ClinicalBLIP obtained superior scores of 0.570/0.365 and 0.534/0.313 on the IU X-RAY/MIMIC-CXR test sets for the Metric for Evaluation of Translation with Explicit Ordering (METEOR) and the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) evaluations, respectively. This performance notably surpasses that of existing state-of-the-art methods. Further evaluations confirmed the effectiveness of the multistage fine-tuning and the integration of prior information, leading to substantial improvements. The proposed ClinicalBLIP model demonstrated robustness and effectiveness in enhancing clinical radiology report generation, suggesting significant promise for real-world clinical applications.

Deep sequential pattern mining for readability enhancement of Indonesian summarization

In text summarization research, readability is a great issue that must be addressed. Our hypothesis is readability can be accomplished by using text representations that keep the meaning of text documents intact. Therefore, this study aims to combine sequential pattern mining (SPM) in producing a sequence of a word as text representation with unsupervised deep learning to produce an Indonesian text summary called DeepSPM. This research uses PrefixSpan as an SPM algorithm and deep belief network (DBN) as an unsupervised deep learning method. This research uses 18,774 Indonesian news text from IndoSum. The readability aspect is evaluated by recall-oriented understudy for gisting evaluation (ROUGE) as a co-selection-based analysis; Dwiyanto Djoko Pranowo metrics, Gunning fog index (GFI), and Flesch-Kincaid grade level (FKGL) as content-based analysis; and human readability evaluation with two experts. The experiment result shows that DeepSPM yields better than DBN, with the F-measure value of ROUGE-1 enhanced to 0.462, ROUGE-2 is 0.37, and ROUGE-L is 0.41. The significance of ROUGE results also be tested using T-Test. The content-based analysis and human readability evaluation findings are conformable with the findings of co-selection-based analysis that generated summaries are only partially readable or have a medium level of readability aspect.

Ar-CM-ViMETA: Arabic Image Captioning based on Concept Model and Vision-based Multi-Encoder Transformer Architecture

Image captioning is a major artificial intelligence research field that involves visual interpretation and linguistic description of a corresponding image. Successful image captioning relies on acquiring as much information as feasible from the original image. One of these essential bits of knowledge is the topic or the concept that the image is associated with. Recently, concept modeling technique has been utilized in English image captioning for completely capturing the image contexts and make use of these contexts to produce more accurate image descriptions. In this paper, a concept-based model is proposed for Arabic Image Captioning (AIC). A novel Vision-based Multi-Encoder Transformer Architecture (ViMETA) is proposed for handling the multi-outputs result from the concept modeling technique while producing the image caption. BiLingual Evaluation Understudy (BLEU) and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) standard metrics have been used to evaluate the proposed model using the Flickr8K dataset with Arabic captions. Furthermore, qualitative analysis has been conducted to compare the produced captions of the proposed model with the ground truth descriptions. Based on the experimental results, the proposed model outperformed the related works both quantitatively, using BLEU and ROUGE metrics, and qualitatively

Fully automatic summarization of radiology reports using natural language processing with large language models

PurposeNatural language processing using language models has yielded promising results in various fields. Language models can help improve the workflow of radiologists. This retrospective study aimed to construct and evaluate language models for automatic summarization of radiology reports. MethodsTwo radiology report datasets from the MIMIC Chest X-ray (MIMIC-CXR) database and the Japan Medical Image Database (JMID) were included in this study. The MIMIC-CXR is an open database comprising chest radiograph reports. The JMID is a large database comprising computed tomography and magnetic resonance imaging reports from 10 academic medical centers in Japan. A total of 128,032 and 1,101,271 reports were included in this study from the MIMIC-CXR database and JMID, respectively. Four Text-to-Text Transfer Transformer (T5) models were constructed. Recall-Oriented Understudy for Gisting Evaluation (ROUGE), a quantitative metric, was used to evaluate the quality of the text summarized from 19,205 and 58,043 test sets from the MIMIC-CXR and JMID, respectively. The Wilcoxon signed-rank test was used to evaluate the differences among the ROUGE values of the four T5 models. Moreover, the subsets of automatically summarized text in the test sets were manually evaluated by two radiologists. The best T5 models were selected for automatic summarization using the Wilcoxon signed-rank test. ResultsThe quantitative metrics of the best T5 models were as follows: ROUGE-1 = 57.75 ± 30.99, ROUGE-2 = 49.96 ± 35.36, and ROUGE-L = 54.07 ± 32.48 in the MIMIC-CXR; and ROUGE-1 = 50.00 ± 29.24, ROUGE-2 = 39.66 ± 30.21, and ROUGE-L = 47.87 ± 29.44 in the JMID. The radiologists’ evaluations revealed 86% and 85% of the texts automatically summarized from the MIMIC-CXR and JMID, respectively, to be clinically useful. ConclusionThe T5 models constructed in this study were able to perform automatic summarization of the radiology reports. The radiologists’ evaluations demonstrated most of the automatically summarized texts to be clinically valuable.

CANBLWO: A Novel Hybrid Approach for Semantic Text Generation

Semantic text generation is critical in Natural Language Processing (NLP) as it faces challenges such as maintenance of coherence among texts, contextual relevance, and quality output. Traditional language models often produce grammatically inconsistent text. To address these issues, we introduce Convolutional Attention Bi-LSTM with Whale Optimization (CANBLWO), a novel hybrid model that integrates a Convolutional Attention Network (CAN), Bidirectional Long Short-Term Memory (Bi-LSTM), and Whale Optimization Algorithm (WOA). CANBLWO aims to generate semantically rich and coherent text and outperforms the traditional models like Long Short-Term Memory (LSTM), Recurrent Neural Networks (RNN), Bi-LSTM, and Bi-LSTM with attention, Bidirectional Encoder Representations from Transformers (BERT), and Generative Pre-trained Transformer 2 (GPT-2). Our model achieved 0.79, 0.78, 0.76, and 0.82 scores in Metric for Evaluation of Translation with Explicit Ordering (METEOR), Bi-Lingual Evaluation Understudy (BLEU), Consensus-based Image Description Evaluation (Ciders), and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics, respectively. The proposed model also demonstrates 97% and 96% accuracy on Wiki-Bio and Code/Natural Language Challenge (CoNaLa) datasets, highlighting its effectiveness against Large Language Models (LLMs). This study underscores the potential capability of hybrid approaches in enhancing semantic text generation

Automatic Update Summarization by a Multiobjective Number-One-Selection Genetic Approach.

Currently, the explosive growth of the information available on the Internet makes automatic text summarization systems increasingly important. A particularly relevant challenge is the update summarization task. Update summarization differs from traditional summarization in its dynamic nature. While traditional summarization is static, that is, the document collections about a specific topic remain unchanged, update summarization addresses dynamic document collections based on a specific topic. Therefore, update summarization consists of summarizing the new document collection under the assumption that the user has already read a previous summarization and only the new information is interesting. The multiobjective number-one-selection genetic algorithm (MONOGA) has been designed and implemented to address this problem. The proposed algorithm produces a summary that is relevant to the user's given query, and it also contains updates information. Experiments were conducted on Text Analysis Conference (TAC) datasets, and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics were considered to assess the model performance. The results obtained by the proposed approach outperform those from the existing approaches in the scientific literature, obtaining average percentage improvements between 12.74% and 55.03% in the ROUGE scores.

KurdSum: A new benchmark dataset for the Kurdish text summarization

Summarizing a text is the process of condensing its content while still maintaining its essential information. With the abundance of digital information available, summarization has become a significant task in various fields, including information retrieval, NLP (Natural Language Processing), and machine learning. This task has been extensively studied in languages such as English and Chinese, but research on Kurdish language summarization is lacking. Therefore, we present the first-ever Kurdish summarization news dataset, KurdSum, which includes over 40,000 texts. We collected news articles from Kurdish websites, preprocessed the data, and manually created a summary for each article. We further assessed the performance of our benchmark dataset on four extractive systems (LEXRANK, TEXTRANK, ORACLE, and LEAD0-3) and three abstractive methods (Pointer-Generator, Sequence-to-Sequence and transformer-abstractive). Our experiments showed that the Pointer-Generator approach yielded superior ROUGE (Recall-Oriented Understudy for Gisting Evaluation) scores compared to other techniques and ORACLE outperformed other extractive methods. Our findings offer a promising direction for the summarization of Kurdish text and can contribute to developing NLP tools for processing the Kurdish language. Likewise, the dataset can serve as a benchmark dataset for Kurdish language summarization and a valuable resource for researchers interested in developing Kurdish summarization models.

Decomposition–based multi-objective differential evolution for extractive multi-document automatic text summarization

The central challenge in Automatic Text Summarization (ATS) involves efficiently generating machine-generated text summaries through optimization algorithms. An ATS is a critical component for systems dealing with textual information processing. However, the current approach faces a significant hurdle due to the computational intensity of the process, particularly when employing complex optimization techniques like swarm intelligence optimization alongside a costly ATS repair operator. While the current approach yields impressive Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics for the generated summary, it struggles with inefficiencies, mainly attributed to the substantial optimization time consumed by the ATS repair operator scheme. In order to address this, a novel solution called Decomposition-based Multi-Objective Differential Evolution (MODE/D) is proposed. It is built upon the foundation of Differential Evolution for Multi-Objective Optimization (DEMO) and the weighted sum method (WS), coupled with an innovative ATS repair operator scheme. Through experimentation on Document Understanding Conferences (DUC) datasets, the novel approach of MODE/D – WS is validated by evaluating the results using ROUGE metrics. The outcomes are twofold: a remarkable reduction in serial execution time and a noteworthy enhancement over existing techniques in the scholarly domain, as evidenced by improved ROUGE-1, ROUGE-2, and ROUGE-L scores.