Generative Adversarial Network Model Research Articles

A novel digital twin strategy to enable the translation of evidence from randomized control trials to new populations

Abstract Background Randomized clinical trials (RCTs) are essential to guide medical practice; however, their generalizability to a given population is often uncertain. Between the SPRINT and ACCORD trials, only SPRINT found a significant reduction in major cardiovascular events (MACE) with intensive blood pressure control, but ACCORD did not, without clear evidence of whether this reflected differences in enrolled patients. Purpose To use a novel RCT digital twin-based approach to demonstrate the translation of an RCT to a different population, leveraging the differences in populations and treatment effects in SPRINT and ACCORD. Methods We developed a statistically informed Generative Adversarial Network (GAN) model that leverages relationships between covariates and outcomes using Directed Acyclic Graphs (DAGs). This model can generate a digital twin of an RCT (RCT-Twin) conditioned on covariate distributions from a second patient population while maintaining these relationships (Fig A). Using this, we generated the (i) RCT-Twin of SPRINT, which was conditioned on the 10 most disparate covariates drawn from ACCORD, and (ii) an RCT-Twin of ACCORD conditioned on SPRINT. To demonstrate treatment effect estimates of each RCT conditioned on the other RCT’s patient population, we evaluated the cardiovascular event-free survival of 10 iterations of these cross-trained RCT-Twins against the original trials. Results Using 9361 SPRINT participants and conditioning on 4733 ACCORD participants, we generated 10 ACCORD-conditioned SPRINT-Twin datasets. We confirmed that the RCT-Twins were consistently balanced between treatment and controls across all measured covariates (mean absolute standardized mean difference (MASMD) 0.019, SD 0.018), as expected for real RCTs. We also confirmed that covariates of the ACCORD-conditioned SPRINT-Twins were closer to ACCORD than SPRINT (MASMD 0.0082 vs 0.46, SD 0.016 vs 0.20), suggestive of successful conditioning. Most importantly, across iterations, ACCORD-conditioned SPRINT-Twin datasets reproduced the overall non-significant effect size seen in ACCORD (5-year MACE hazard ratio (95% confidence interval) of 0.88 (0.73-1.06) in ACCORD vs median 0.87 (0.68-1.13) in the ACCORD-conditioned SPRINT-Twins, Fig B), while the SPRINT-conditioned ACCORD-Twin datasets reproduced the significant effect size seen in SPRINT (0.75 (0.64-0.89) vs median 0.79 (0.72-0.86) in the SPRINT-conditioned ACCORD-Twins, Fig C). Conclusions Our novel conditioned digital twin approach simulates RCT-derived effects in different patient populations by translating these effects to the covariate distributions of the patients. This key methodological advance may enable the direct translation of RCT-derived effects into disparate patient populations and may enable causal inference in real-world settings.

Deep learning approaches for image restoration in invasive coronary angiography

Abstract Background Despite being the gold standard for diagnosing coronary artery disease and guiding percutaneous coronary interventions (PCI), image restoration during invasive coronary angiography (ICA) has rarely been explored. Generative adversarial network (GAN), a deep learning model, has demonstrated its effectiveness in image restoration. Our hypothesis was that we could restore ICA images, damaged or deleted to varying degrees, using the inpainting GAN method, and we undertook efforts to test this hypothesis. Methods We extracted 22,659 ICA patch images (128 × 128 size) from 60 patients, focusing on sparse vessels for model training. To assess our inpainting GAN model, we degraded the extracted images to varying degrees: minimal, mild, moderate, severe, and completely blocked. The performance was quantitatively analyzed using metrics such as the peak signal-to-noise ratio (PSNR), structured similarity indexing methods (SSIM), and mean squared error (MSE). Results The inpainting GAN method significantly improved the image quality in all degraded images. For the blocked images, PSNR and SSIM were enhanced by 161.9% and 14.3%, respectively, with a 98% MSE reduction. In severely, moderately, mildly, and minimally damaged images, PSNR and SSIM increased by 14.9% and 14.8%, 15.7% and 14.7%, 19.3% and 15.6%, and 12.7% and 5.3%, respectively, with MSE reductions of 61.5%, 64.1%, 72.7%, and 33.9%, respectively. Conclusions Inpainting GAN can restore ICA images with varying damage levels, including deletions. Our deep-learning model can instantaneously improve ICA image quality, potentially aiding in emergency or chronic total occlusion PCI.Table 2.Quantitative Analysis of ImageFigure 1.Sample Cases Pre and Post-Rest

Generative adversarial networks (GANs) to produce synthetic 12-lead electrocardiogram signals for specific and rare diseases: a novel, powerful tool towards clinical advancements

Abstract Background Remarkable technological advancements in electrocardiogram (ECG) analysis face challenges due to insufficiently diverse and extensive real-world datasets. This limitation hampers both the development of sophisticated deep learning (DL) models for ECG analysis and the exposure of new physicians to substantial training examples. Purpose To leverage Generative Adversarial Networks (GANs), a novel AI algorithm type, to generate synthetic yet realistic ECG signals resembling specific diseases. Methods We initially employed a general-purpose GAN on more than 21,000 ECG samples from the PTB-XL database, encompassing normal and abnormal ECGs across various diseases. The GAN outputs served as inputs for training "DeceptionECG", a novel GAN model inspired by the InceptionTime architecture (code available online). DeceptionECG focused on generating disease-specific ECG signals, trained on annotated ECGs validated by two electrophysiologists. Post-training, we generated synthetic ECGs for common diseases like atrial fibrillation (AF) and anterior ST-segment elevation myocardial infarction (STEMI), as well as rarer conditions like Wolf-Parkinson White (WPW) syndrome. Authentication involved an independent discriminator DL model and two cardiologists who were asked to distinguish real from synthetic ECGs. Additionally, a DL model, a variant of the InceptionTime architecture, discerned specific diseases in the ECG signals to assess their realism. Finally, we trained a DL model for detecting abnormalities in ECGs with the augmented dataset to evaluate potential performance improvement. Results We generated sets of 5,000 synthetic ECGs for various abnormalities, at a rate of 2,500 12-lead ECG signals per minute. The generated ECGs were indistinguishable from real ones, evidenced by low accuracy (Accuracy, ACC: 48.9%) of the discriminator model and two clinicians (ACC: 56% and 51%), on test sets with anterior STEMI. A DL model trained on real ECGs accurately differentiated between synthetic normal ECGs and those hosting the target disease in nearly 9 out of 10 cases (ACC: 88.7%). Furthermore, the employed DL model detector trained on the augmented dataset, encompassing both real (n=1,589) and synthetic (n=5,000) ECGs, outperformed its counterpart trained solely on real ECGs, when tested on a 1,000-sample set of real signals (ACC: 94.6% vs. 89.3%). Conclusion Our GAN-based approach showcases the potential of generating high-quality synthetic ECGs tailored for specific diseases, overcoming data scarcity challenges, and enhancing the training and performance of machine learning models. Additionally, it holds promise in advancing medical professional training by providing diverse signals with distinct "disease stamps" at massive amounts, addressing privacy concerns associated with extensive data exposure. Further testing is warranted for the applicability of the approach, especially in exceedingly rare conditions with limited available training data.Project workflow and outcomesGAN-generated ECGs: Which is fake?

Crystal Structure Prediction Using Generative Adversarial Network with Data-Driven Latent Space Fusion Strategy.

Crystal structure prediction (CSP) is an important field of material design. Herein, we propose a novel generative adversarial network model, guided by a data-driven approach and incorporating the real physical structure of crystals, to address the complexity of high-dimensional data and improve prediction accuracy in materials science. The model, termed GAN-DDLSF, introduces a novel sampling method called data-driven latent space fusion (DDLSF), which aims to optimize the latent space of generative adversarial networks (GANs) by combining the statistical properties of real data with a standard Gaussian distribution, effectively mitigating the "mode collapse" problem prevalent in GANs. Our approach introduces a more refined generation mechanism specifically for binary crystal structures such as gallium nitride (GaN). By optimizing for the specific crystallographic features of GaN while maintaining structural rationality, we achieve higher precision and efficiency in predicting and designing structures for this particular material system. The model generates 9321 GaN binary crystal structures, with 16.59% reaching a stable state and 24.21% found to be metastable. These results can significantly enhance the accuracy of crystal structure predictions and provide valuable insights into the potential of the GAN-DDLSF approach for the discovery and design of binary, ternary, and multinary materials, offering new perspectives and methods for materials science research and applications.

Deep learning assisted cancer disease prediction from gene expression data using WT-GAN

Several diverse fields including the healthcare system and drug development sectors have benefited immensely through the adoption of deep learning (DL), which is a subset of artificial intelligence (AI) and machine learning (ML). Cancer makes up a significant percentage of the illnesses that cause early human mortality across the globe, and this situation is likely to rise in the coming years, especially when non-communicable illnesses are not considered. As a result, cancer patients would greatly benefit from precise and timely diagnosis and prediction. Deep learning (DL) has become a common technique in healthcare due to the abundance of computational power. Gene expression datasets are frequently used in major DL-based applications for illness detection, notably in cancer therapy. The quantity of medical data, on the other hand, is often insufficient to fulfill deep learning requirements. Microarray gene expression datasets are used for training procedures despite their extreme dimensionality, limited volume of data samples, and sparsely available information. Data augmentation is commonly used to expand the training sample size for gene data. The Wasserstein Tabular Generative Adversarial Network (WT-GAN) model is used for the data augmentation process for generating synthetic data in this proposed work. The correlation-based feature selection technique selects the most relevant characteristics based on threshold values. Deep FNN and ML algorithms train and classify the gene expression samples. The augmented data give better classification results (> 97%) when using WT-GAN for cancer diagnosis.

Comparison Between Gans and Diffusion Models in the Generation of Synthetic Images for Enhancing Tree Species Recognition

Objective: This study investigates the application of Generative Adversarial Networks (GANs) and Diffusion Models in data augmentation to improve the classification of tree species images, which is essential for sustainable forest management. Theoretical Framework: Fundamental concepts of machine learning, generative and classifier networks, as well as data augmentation techniques through synthetic image generation, are presented, establishing a solid foundation for the research. Method: The research utilized 2,178 images of cross-sections of wood from 18 species, applying Deep Convolutional GAN (DCGAN) and U-Net with diffusion on a rare species, evaluated using FID and IS metrics. The generated images were used to train and validate classification models, assessed by F1-score. Results and Discussion: The results revealed that Diffusion Models generated more realistic images and performed better in the classification of the rare species. The discussion contextualizes these results in light of the theoretical framework, exploring their implications for environmental management. Limitations, such as the impact of indiscriminate addition of synthetic data, are also addressed. Research Implications: The practical and theoretical implications of this research are discussed, providing insights into how the results can be applied or influence practices in the field of forest management and environmental management. These implications may cover areas such as sustainable innovation and biodiversity conservation. Originality/Value: This study contributes to the literature by demonstrating the potential of Diffusion Models to outperform GANs in generating synthetic images for sustainable forest management purposes. The relevance and value of this research are evidenced by its practical application in promoting innovative and sustainable practices in environmental management.

An efficient conditional GAN-based framework for high-resolution prediction of tyre-pavement contact stresses – a contribution towards a digital twin of the road system

ABSTRACT The high-resolution prediction of tyre-pavement contact stresses is crucial for advancing pavement design and vehicle safety. A significant knowledge gap exists in accurately and quickly predicting tyre-pavement contact stresses, especially under varying conditions. Addressing this challenge, this study is part of the concerted effort within the project CRC/TRR 339, contributing to the series ‘A Contribution towards a Digital Twin of the Road System’. It aims to bridge this gap by introducing a novel Multimodal Conditional Deep Convolutional Generative Adversarial Network (MC-DCGAN) model for forecasting tyre-pavement contact stresses, offering an efficient and accurate method. The incorporation of a multimodal conditional module within the GAN-based framework allows for controllable outputs tailored to specific tyre types, inflation pressures, and loads. Leveraging the DCGAN architecture significantly enhances prediction accuracy. Experimental validation confirms the method’s high accuracy, strong generalisation capabilities, and a substantial reduction in computational time – nearly two orders of magnitude lower compared to traditional Finite Element simulations. This research not only fills a critical void in the high-precision prediction of tyre-pavement contact stresses, but also holds significant implications for optimising pavement design and enhancing road traffic safety, marking a significant step towards realising a digital twin of the road system.

Derivative dynamic time warping algorithm with introduced correction for varying load fault diagnosis of nuclear power system steam turbine units

The Derivative Dynamic Time Warping (DDTW) algorithm is improved to address the multi-parameters time series classification problem faced by nuclear power system steam turbine units varying load fault diagnosis. Firstly, the entire load changing process is treated as a single sample rather than multiple time-step-samples. This ensures the complete information on the load changing processes, while avoiding interference from normal data fluctuations during faults. Secondly, Time Series Position Coefficient and Time Series Length Coefficient are proposed to correct the DDTW algorithm from two perspectives: the sequences lengths and the data positions in the sequences. This solves the singularities and timeline scaling problems, thereby preventing interference introduced by data sequences' lengths differences and ''similar data appearing at different times'' problem. The nuclear power system steam turbine unit simulation model was built to obtain load changing processes data under normal and faults statuses. In the varying load fault diagnosis test based on these data, the improved DDTW algorithm achieved an accuracy of 1.38%–12.06% higher than other methods, reaching 87.50%. Finally, The Deep Convolutional Generative Adversarial Networks (DCGAN) model was used to generate data to supplement the limited samples of complete load changing processes, and the accuracy of the novel method increased to 95.51% with the increase of data used to support the comparison.

Malware Detection Based on API Call Sequence Analysis: A Gated Recurrent Unit–Generative Adversarial Network Model Approach

Malware remains a major threat to computer systems, with a vast number of new samples being identified and documented regularly. Windows systems are particularly vulnerable to malicious programs like viruses, worms, and trojans. Dynamic analysis, which involves observing malware behavior during execution in a controlled environment, has emerged as a powerful technique for detection. This approach often focuses on analyzing Application Programming Interface (API) calls, which represent the interactions between the malware and the operating system. Recent advances in deep learning have shown promise in improving malware detection accuracy using API call sequence data. However, the potential of Generative Adversarial Networks (GANs) for this purpose remains largely unexplored. This paper proposes a novel hybrid deep learning model combining Gated Recurrent Units (GRUs) and GANs to enhance malware detection based on API call sequences from Windows portable executable files. We evaluate our GRU–GAN model against other approaches like Bidirectional Long Short-Term Memory (BiLSTM) and Bidirectional Gated Recurrent Unit (BiGRU) on multiple datasets. Results demonstrated the superior performance of our hybrid model, achieving 98.9% accuracy on the most challenging dataset. It outperformed existing models in resource utilization, with faster training and testing times and low memory usage.

Improving Non-Line-of-Sight Identification in Cellular Positioning Systems Using a Deep Autoencoding and Generative Adversarial Network Model

Positioning service is a critical technology that bridges the physical world with digital information, significantly enhancing efficiency and convenience in life and work. The evolution of 5G technology has proven that positioning services are integral components of current and future cellular networks. However, positioning accuracy is hindered by non-line-of-sight (NLoS) propagation, which severely affects the measurements of angles and delays. In this study, we introduced a deep autoencoding channel transform-generative adversarial network model that utilizes line-of-sight (LoS) samples as a singular category training set to fully extract the latent features of LoS, ultimately employing a discriminator as an NLoS identifier. We validated the proposed model in 5G indoor and indoor factory (dense clutter, low base station) scenarios by assessing its generalization capability across different scenarios. The results indicate that, compared to the state-of-the-art method, the proposed model markedly diminished the utilization of device resources and achieved a 2.15% higher area under the curve while reducing computing time by 12.6%. This approach holds promise for deployment in future positioning terminals to achieve superior localization precision, catering to commercial and industrial Internet of Things applications.

GenAI for Scientific Discovery in Electrochemical Energy Storage: State-of-the-Art and Perspectives from Nano- and Micro-Scale.

The transition to electric vehicles (EVs) and the increased reliance on renewable energy sources necessitate significant advancements in electrochemical energy storage systems. Fuel cells, lithium-ion batteries, and flow batteries play a key role in enhancing the efficiency and sustainability of energy usage in transportation and storage. Despite their potential, these technologies face limitations such as high costs, material scarcity, and efficiency challenges. This research introduces a novel integration of Generative AI (GenAI) within electrochemical energy storage systems to address these issues. By leveraging advanced GenAI techniques like Generative Adversarial Networks, autoencoders, diffusion and flow-based models, and multimodal large language models, this paper demonstrates significant improvements in material discovery, battery design, performance prediction, and lifecycle management across different types of electrochemical storage systems. The research further emphasizes the importance of nano- and micro-scale interactions, providing detailed insights into optimizing these interactions for improved efficiency and longevity. Additionally, the paper discusses the challenges and future directions for integrating GenAI in energy storage research, highlighting the importance of data quality, model transparency, workflow integration, scalability, and ethical considerations. By addressing these aspects, this research sets a new benchmark for the use of GenAI in battery development, promoting sustainable, efficient, and safer energy solutions.

Denoising astronomical images using an enhanced Pix2Pix model

Astronomical images are frequently affected by sensor noise, which can negatively impact the accuracy of subsequent data analysis. To address this issue, this study proposes an enhanced Pix2Pix generative adversarial network model that incorporates Residual Blocks and Self-Attention mechanisms to improve denoising performance. The effectiveness of the proposed model is evaluated by comparing it with traditional denoising methods, standard Pix2Pix, Pix2Pix with Residual Blocks, and Pix2Pix with Self-Attention using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) metrics. The findings demonstrate that the Pix2Pix model, when combined with both Residual Blocks and Self-Attention, significantly outperforms other models in noise reduction and detail preservation. This improved approach offers a robust solution for high-quality processing of astronomical images, providing clearer and more reliable data for scientific analysis. The results highlight the potential of advanced deep learning techniques in overcoming the challenges posed by sensor noise in astronomical imaging.

Intelligent Lost Circulation Monitoring Method Based on Data Augmentation and Temporal Models

Deep and offshore drilling operations face complex geological formations, uncertain formation pressures, and narrow safety density windows, making them susceptible to lost circulation risks. To address these challenges, this paper introduces an innovative, intelligent lost circulation monitoring model that incorporates geological lithology information. This model not only utilizes real-time drilling parameters, but also encodes geological information such as rock type as inputs to the model. By combining these key lithological features, the model can comprehensively assess wellbore stability and reduce the lost circulation risks. In this paper, the Conditional Tabular Generative adversarial network (CTGAN) model is used to enhance the data of small-sample risk data, which can effectively expand the data distribution space and improve the performance of the model. This paper conducts a comparative analysis of intelligent monitoring results using artificial neural networks (ANNs), long short-term memory (LSTM), and temporal convolutional networks (TCNs). The results show that the TCN achieves an identification accuracy of 93.7%. Furthermore, the analysis reveals that the inclusion of lithology information significantly enhances the model’s performance, resulting in a 7.1% increase in accuracy. The false alarm rate of the model can be reduced by 10.2%, considering the fluctuation of the logging curve caused by the on/off condition of the pump. This indicates that the introduction of lithology information and the condition of the pump on−off provide advantages in monitoring and identifying lost circulation risks, enabling a more precise assessment of wellbore stability and a reduction in lost circulation incidents. The method of lost circulation monitoring proposed in this paper provides an important safety guarantee for the oil drilling industry.

Unsupervised Representation Learning using Wasserstein Generative Adversarial Network

In recent years, representational learning has attracted considerable attention. However, unsupervised representation learning has received less attention compared to supervised representation learning. This paper introduces a combination of a deep neural network (DNN) and a generative adversarial network (GAN) that can learn features through unsupervised learning. Essentially, the Generative Adversarial Network (GAN) is a deep learning architecture that engages two neural networks in a framework similar to a zero-sum game. Generating new, synthetic data that resembles a known data distribution is the aim of GANs. In June 2014, Ian Goodfellow and associates first developed the idea of Generative Adversarial Network (GAN). The research used a new type of GAN model which is called Wasserstein GAN. There are some distinct differences between traditional GAN and Wasserstein GAN. This paper highlights the differences and benefits of using Wasserstein GAN, as well as the architecture of Wasserstein GAN. This study trained the model on an image dataset to extract features, and subsequently tested it on another dataset, demonstrating that the GAN model learns a hierarchy of representation from object parts in the discriminator. The purpose of this paper is to use unsupervised learning like Convolutional Neural Network (CNN) and Wasserstein Generative Adversarial Network (WGAN) for feature extraction from unlabeled dataset.

The deep learning-based physical education course recommendation system under the internet of things

This study aims to propose a deep learning (DL)-based physical education course recommendation system by combining the Internet of Things (IoT) technology and DL, to improve the accuracy and personalization of recommendation. Firstly, IoT devices such as smart bracelets and smart clothing are used to monitor students' physiological data in real-time, and IoT sensors are utilized to sense the environment around students. Secondly, IoT devices capture students' social interactions with their peers, recommending socially oriented courses. Meanwhile, by integrating IoT data with students' academic data, course recommendations are optimized to match students' learning progress and schedule. Finally, Generative Adversarial Network (GAN) models, especially the improved Regularization Penalty Conditional Feature Generative Adversarial Network (RP-CFGAN) model, deal with data sparsity and cold start problems. The experimental results show that this model performs well in TopN evaluation and is markedly enhanced compared with traditional models. This study denotes that integrating IoT technology and GAN models can more accurately understand student needs and provide personalized recommendations. Although the model performs well, there is still room for improvement, such as exploring more regularization techniques, protecting user privacy, and extending the system to diverse platforms and scenarios.

Text2shape: Intelligent computational design of car outer contour shapes based on improved conditional Wasserstein generative adversarial network

To provide technical support for the initial design of products, we propose an innovative text2shape-based technology for intelligent computational design, which can map engineering semantics to functional/structural/Kansei feature spaces to generate product shapes. New energy vehicles were selected as the application object of this technology, as there are many creative ideas for the outer contour design of new energy vehicles. Firstly, a dataset with 2900 + samples was built based on feature engineering (FE) and Kansei engineering (KE). Each sample contains the car’s outer contour shape’s functional, structural, and Kansei features. Secondly, we proposed an improved conditional Wasserstein generative adversarial network (CWGAN) model suitable to the dataset. The generator’s loss in the model is designed to evaluate the authenticity of the generated results, while the discriminator’s loss assesses the conditional matching of these results. Finally, in case studies, the trained CWGAN was compared with the conditional variational auto-encoder (C-VAE), diffusion, Wasserstein generative adversarial network with gradient penalty (WGAN-GP) and style generative adversarial network (StyleGAN) models, demonstrating superior performance.

3D microstructure reconstruction and characterization of porous materials using a cross-sectional SEM image and deep learning

Accurate assessment of the three-dimensional (3D) pore characteristics within porous materials and devices holds significant importance. Compared to high-cost experimental approaches, this study introduces an alternative method: utilizing a generative adversarial network (GAN) to reconstruct a 3D pore microstructure. Unlike some existing GAN models that require 3D images as training data, the proposed model only requires a single cross-sectional image for 3D reconstruction. Using porous ceramic electrode materials as a case study, a comparison between the GAN-generated microstructures and those reconstructed through focused ion beam-scanning electron microscopy (FIB-SEM) reveals promising consistency. The GAN-based reconstruction technique demonstrates its effectiveness by successfully characterizing pore attributes in porous ceramics, with measurements of porosity, pore size, and tortuosity factor exhibiting notable agreement with the results obtained from mercury intrusion porosimetry.

CT image reconstruction: integrating iterative methods with Ml-EM algorithm and deep learning models

Computed Tomography (CT) imaging faces limitations, including low spatial resolution and noise, particularly in low-radiation-dose imaging. To address these challenges, researchers are exploring CT image reconstruction from sinogram data. Sinograms represent X-ray absorption throughout the body, and sophisticated image reconstruction methods, including machine learning algorithms and generative adversarial networks (GANs), can improve precision and resolution without increasing patient radiation exposure. This study proposes an iterative reconstruction approach that combines filters from deep learning models (Convolutional Neural Networks and UNet) with the Maximum Likelihood Expectation Maximization (ML-EM) algorithm and the Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN) model. Our method aims to enhance image quality and reconstruction speed. Experimental results show significant improvements in image quality and resolution, with the proposed method (DL-MLEM-IR-UNET-ESRGAN) achieving an average SSIM of 0.9980 and PSNR of 53.2119, outperforming other methods. Additionally, our method reduces reconstruction time, with an average runtime of 131 seconds.

Layout Congestion Prediction Based on Regression-ViT

To accelerate the back-end design flow of integrated circuit (IC), numerous studies have made exploratory advancements in machine learning (ML) for electronic design automation (EDA). However, most research works are limited to deep learning (DL) models predominantly based on convolutional neural networks, and the models often suffer from poor generalization due to the scarcity of data. In this study, we propose the Double generative adversarial networks (D-GAN) model to enrich the dataset and propose the Regression Vision Transformer (R-ViT) model to predict layout congestion information. Compared to the baseline model, experimental results show improvements of 3.03% and 2.64% in Receiver Operating Characteristic-Area under Curve (ROC-AUC) and Precision-Recall Curve-Area under Curve (PRC-AUC) respectively. To further enhance the prediction accuracy of the model, an adaptive Huber loss function is designed to optimize the training process, resulting in an improvement of up to 11.03% in ROC-AUC compared with the baseline model. Lastly, extended experiments are conducted to study the effects of parameters and convolutional kernel size on performance, which find a better configuration.

Three-stage binarization of color document images based on discrete wavelet transform and generative adversarial networks

The efficient extraction of text information from the background in degraded color document images is an important challenge in the preservation of ancient manuscripts. The imperfect preservation of ancient manuscripts has led to different types of degradation over time, such as page yellowing, staining, and ink bleeding, seriously affecting the results of document image binarization. This work proposes an effective three-stage network method to image enhancement and binarization of degraded documents using generative adversarial networks (GANs). Specifically, in Stage-1, we first split the input images into multiple patches, and then split these patches into four single-channel patch images (gray, red, green, and blue). Then, three single-channel patch images (red, green, and blue) are processed by the discrete wavelet transform (DWT) with normalization. In Stage-2, we use four independent generators to separately train GAN models based on the four channels on the processed patch images to extract color foreground information. Finally, in Stage-3, we train two independent GAN models on the outputs of Stage-2 and the resized original input images (512 × 512) as the local and global predictions to obtain the final outputs. The experimental results show that the Avg-Score metrics of the proposed method are 77.64, 77.95, 79.05, 76.38, 75.34, and 77.00 on the (H)-DIBCO 2011, 2013, 2014, 2016, 2017, and 2018 datasets, which are at the state-of-the-art level. The implementation code for this work is available at https://github.com/abcpp12383/ThreeStageBinarization.