Generative Adversarial Network Architecture Research Articles

Advancements and Applications of Generative Adversarial Networks: A Comprehensive Review

Abstracts: Our paper offers a comprehensive exploration of Generative Adversarial Networks (GANs), tracing their evolution from Ian Goodfellow's seminal work to their current state-of-the-art status. Delving into the intricacies of GAN architecture and training dynamics, we illuminate their pivotal role in diverse applications such as image synthesis, style transfer, and text-toimage conversion. Through an exhaustive literature review, we dissect the progression of GAN architectures, from Vanilla GANs [1] to advanced variants like Progressive GANs [7] and StyleGANs [8], highlighting their techniques, contributions, and performance across benchmark datasets. Moreover, we confront challenges such as training instability and mode collapse, while also presenting a meticulously curated repository of contemporary generative model advancements. This repository encapsulates the cutting edge of GAN research, showcasing innovative approaches across domains ranging from financial forecasting to image restoration. Despite hurdles and ethical considerations, GANs persist as the vanguard of generative modeling, propelling forward the frontiers of artificial intelligence and creative synthesis.

GAN-Based Super Resolution Algorithm for High-Quality Image Enhancement

Abstract: Super Resolution: Leveraging investigates the trans-formative potential of Generative Adversarial Networks (GANs) in the context of super resolution, utilizing state-of-the-art algorithms to achieve high-quality image enhancement. The study encompasses a comprehensive analysis of GAN architectures suchas SRGAN, ESRGAN, and others, exploring their strengths, limitations, and practical applications. The proposed methodology delves into dataset selection, preprocessing techniques, and the integration of advanced algorithms within the GAN framework. It evaluates the quantitative and perceptual performance of these techniques, addressing challenges and proposing avenues for future research. The findings contribute to the evolving landscapeof GAN-based super resolution, offering insights for researchers and practitioners in advancing image enhancement technologies

Improved brain metastases segmentation using generative adversarial network and conditional random field optimization mask R-CNN.

In radiotherapy, the delineation of the gross tumor volume (GTV) in brain metastases using computed tomography (CT) simulation localization is very important. However, despite the criticality of this process, a pronounced gap exists in the availability of tools tailored for the automatic segmentation of the GTV based on CT simulation localization images. This study aims to fill this gap by devising an effective tool specifically for the automatic segmentation of the GTV using CT simulation localization images. A dual-network generative adversarial network (GAN) architecture was developed, wherein the generator focused on refining CT images for more precise delineation, and the discriminator differentiated between real and augmented images. This architecture was coupled with the Mask R-CNN model to achieve meticulous GTV segmentation. An end-to-end training process facilitated the integration between the GAN and Mask R-CNN functionalities. Furthermore, a conditional random field (CRF) was incorporated to refine the initial masks generated by the Mask R-CNN model to ensure optimal segmentation accuracy. The performance was assessed using key metrics, namely, the Dice coefficient (DSC), intersection over union (IoU), accuracy, specificity, and sensitivity. The GAN+Mask R-CNN+CRF integration method in this study performs well in GTV segmentation. In particular, the model has an overall average DSC of 0.819±0.102 and an IoU of 0.712±0.111 in the internal validation. The overall average DSC in the external validation data is 0.726±0.128 and the IoU is 0.640±0.136. It demonstrates favorable generalization ability. The integration of the GAN, Mask R-CNN, and CRF optimization provides a pioneering tool for the sophisticated segmentation of the GTV in brain metastases using CT simulation localization images. The method proposed in this study can provide a robust automatic segmentation approach for brain metastases in the absence of MRI.

Image inpainting based on GAN-driven structure- and texture-aware learning with application to object removal

In this paper, a novel deep learning-based image inpainting framework consisting of restoring image structure and reconstructing image details from corrupted images is proposed. Most image inpainting methods in the literature aim at restoring image details, outlines, and colors, simultaneously, which may suffer from blurring, deformation, and unreasonable content recovery due to interference among various information. To solve these problems, a two-stage image inpainting deep neural network based on GAN (generative adversarial network) architecture is proposed. The proposed inpainting framework consists of two modules: (1) the first stage, called the structure-aware learning stage, aims at learning a GAN-based structure restoration network, focusing on recovering the low-frequency image component, including colors and outlines of the missing regions of the input corrupted image; and (2) the second stage, called the texture-aware learning stage, aims at learning a GAN-based detail refinement network, focusing on rebuilding the high-frequency image details and texture information. In particular, we also propose to remove details from the training images to better train the structure restoration network to avoid inadequate image structure recovery induced by richer image textures, where the detail reconstruction task is left to the second stage. This strategy achieves to balance the workload between the two stages and the image quality can be progressively enhanced through the two stages. Experimental results have shown that the proposed deep inpainting framework quantitatively and qualitatively achieves state-of-the-art performance on the well-known datasets, including the CelebA, Places2, and ImageNet datasets, compared with existing deep learning-based image inpainting approaches. More specifically, in terms of the two well-known image quality assessment metrics, PSNR (peak signal-to-noise ratio) and SSIM (structural similarity), the improvement percentage of the proposed method, compared with the baseline approach, respectively, ranges from 3.23 % to 11.12 %, and 1.95 % to 13.39 %. The improvements have been shown to stably and significantly outperform the compared state-of-the-art methods in most types of inpainting mask. We also show that the proposed method is applicable to image editing in object removal from a single image.

Transfer-Learning-Enhanced Regression Generative Adversarial Networks for Optimal eVTOL Takeoff Trajectory Prediction

Electric vertical takeoff and landing (eVTOL) aircraft represent a crucial aviation technology to transform future transportation systems. The unique characteristics of eVTOL aircraft include reduced noise, low pollutant emission, efficient operating cost, and flexible maneuverability, which in the meantime pose critical challenges to advanced power retention techniques. Thus, optimal takeoff trajectory design is essential due to immense power demands during eVTOL takeoffs. Conventional design optimizations, however, adopt high-fidelity simulation models in an iterative manner resulting in a computationally intensive mechanism. In this work, we implement a surrogate-enabled inverse mapping optimization architecture, i.e., directly predicting optimal designs from design requirements (including flight conditions and design constraints). A trained inverse mapping surrogate performs real-time optimal eVTOL takeoff trajectory predictions with no need for running optimizations; however, one training sample requires one design optimization in this inverse mapping setup. The excessive training cost of inverse mapping and the characteristics of optimal eVTOL takeoff trajectories necessitate the development of the regression generative adversarial network (regGAN) surrogate. We propose to further enhance regGAN predictive performance through the transfer learning (TL) technique, creating a scheme termed regGAN-TL. In particular, the proposed regGAN-TL scheme leverages the generative adversarial network (GAN) architecture consisting of a generator network and a discriminator network, with a combined loss of the mean squared error (MSE) and binary cross-entropy (BC) losses, for regression tasks. In this work, the generator employs design requirements as input and produces optimal takeoff trajectory profiles, while the discriminator differentiates the generated profiles and real optimal profiles in the training set. The combined loss facilitates the generator training in the dual aspects: the MSE loss targets minimum differences between generated profiles and training counterparts, while the BC loss drives the generated profiles to share analogous patterns with the training set. We demonstrated the utility of regGAN-TL on optimal takeoff trajectory designs for the Airbus A3 Vahana and compared its performance against representative surrogates, including the multi-output Gaussian process, the conditional GAN, and the vanilla regGAN. Results showed that regGAN-TL reached the 99.5% generalization accuracy threshold with only 200 training samples while the best reference surrogate required 400 samples. The 50% reduction in training expense and reduced standard deviations of generalization accuracy achieved by regGAN-TL confirmed its outstanding predictive performance and broad engineering application potential.

MuLA-GAN: Multi-Level Attention GAN for Enhanced Underwater Visibility

The underwater environment presents unique challenges (color distortions, reduced contrast, blurriness) hindering accurate analysis. This work introduces MuLA-GAN, a novel approach leveraging Generative Adversarial Networks (GANs) and specifically adapted Multi-Level Attention for comprehensive underwater image enhancement. MuLA-GAN integrates Multi-Level Attention within the GAN architecture to prioritize learning discriminative features crucial for precise image restoration. These relevant features encompass information on local details within image regions leveraged by spatial attention and features at various scales across the entire image captured by multi-level attention. This allows MuLA-GAN to identify and enhance objects, textures, and edges obscured by underwater distortions while also reconstructing a more accurate and visually clear representation of the underwater scene by analyzing low-level information like edges and textures, as well as high-level information like object shapes and global scene information. By selectively focusing on these relevant features, MuLA-GAN excels at capturing and preserving intricate details in underwater imagery, which is essential for various marine research, exploration, and resource management applications. Extensive evaluations on diverse datasets (UIEB test, UIEB challenge, U45, UCCS) demonstrate MuLA-GAN's superior performance compared to existing methods. Additionally, a specialized bio-fouling and aquaculture dataset confirms the model's robustness in challenging environments. On the UIEB test dataset, MuLA-GAN achieves exceptional Peak Signal-to-Noise Ratio (PSNR) (25.59) and Structural Similarity Index (SSIM) (0.893) scores, surpassing Water-Net (24.36 PSNR, 0.885 SSIM). This work addresses a significant research gap in underwater image enhancement by demonstrating the effectiveness of combining GANs with specifically adapted Multi-Level Attention mechanisms. This tailored approach offers a novel and comprehensive framework for restoring underwater image quality, providing valuable insights for accurate underwater scene analysis. The source code for MuLA-GAN is publicly available on GitHub at https://github.com/AhsanBaidar/MuLA_GAN.git

Generative Adversarial Networks for Malware Detection in Cloud Computing Environments

In recent years, the proliferation of sophisticated malware threats has necessitated the development of advanced detection techniques to safeguard computer systems and networks. Generative Adversarial Networks (GANs), a class of deep learning models comprising a generator and a discriminator trained in an adversarial manner, have emerged as a promising tool for improving malware detection capabilities. Traditional signature-based malware detection struggles to keep pace with the ever-evolving threat landscape in cloud computing environments. This paper proposes a novel approach utilizing Generative Adversarial Networks (GANs) for enhanced malware detection. The GAN architecture generates realistic malware samples, refining the discriminator's ability to differentiate between legitimate and malicious software. This fosters a model adept at identifying zero-day attacks and unseen malware variants. Evaluation in a cloud environment assesses detection accuracy, efficiency, and generalizability. Findings underscore GANs' potential to enhance cloud-based malware detection, securing the future of cloud computing.

Deep learning-based whole-body PSMA PET/CT attenuation correction utilizing Pix-2-Pix GAN.

Sequential PET/CT studies oncology patients can undergo during their treatment follow-up course is limited by radiation dosage. We propose an artificial intelligence (AI) tool to produce attenuation-corrected PET (AC-PET) images from non-attenuation-corrected PET (NAC-PET) images to reduce need for low-dose CT scans. A deep learning algorithm based on 2D Pix-2-Pix generative adversarial network (GAN) architecture was developed from paired AC-PET and NAC-PET images. 18F-DCFPyL PSMA PET-CT studies from 302 prostate cancer patients, split into training, validation, and testing cohorts (n = 183, 60, 59, respectively). Models were trained with two normalization strategies: Standard Uptake Value (SUV)-based and SUV-Nyul-based. Scan-level performance was evaluated by normalized mean square error (NMSE), mean absolute error (MAE), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR). Lesion-level analysis was performed in regions-of-interest prospectively from nuclear medicine physicians. SUV metrics were evaluated using intraclass correlation coefficient (ICC), repeatability coefficient (RC), and linear mixed-effects modeling. Median NMSE, MAE, SSIM, and PSNR were 13.26%, 3.59%, 0.891, and 26.82, respectively, in the independent test cohort. ICC for SUVmax and SUVmean were 0.88 and 0.89, which indicated a high correlation between original and AI-generated quantitative imaging markers. Lesion location, density (Hounsfield units), and lesion uptake were all shown to impact relative error in generated SUV metrics (all p < 0.05). The Pix-2-Pix GAN model for generating AC-PET demonstrates SUV metrics that highly correlate with original images. AI-generated PET images show clinical potential for reducing the need for CT scans for attenuation correction while preserving quantitative markers and image quality.

Creating realistic anterior segment optical coherence tomography images using generative adversarial networks

AimsTo develop a generative adversarial network (GAN) capable of generating realistic high-resolution anterior segment optical coherence tomography (AS-OCT) images.MethodsThis study included 142 628 AS-OCT B-scans from the American University of...

An improved hybrid solar cell defect detection approach using Generative Adversarial Networks and weighted classification

Quality control has a vital role in manufacturing processes. Electroluminescence (EL) imaging is one of the main non-destructive inspection methods for quality assessment in the Photovoltaic (PV) module production industry. EL test reveals PV cell defects such as micro cracks, broken cells, finger interruptions and provides detailed information about production quality. In recent years, automated detection and classification systems using deep neural networks for PV module inspection have gained increasing attention. However, deep learning-based systems usually require large amount of labeled data and high computational power. In this work, we proposed a compact classification framework based on hybrid data augmentation and deep learning models for detection of the defective solar cells. In the proposed method, the limited and imbalanced EL datasets were augmented through various Generative Adversarial Networks (GAN), and defect detection was achieved by customized pre-trained Convolutional Neural Networks (CNN). A novel hybrid EL dataset for training was formed by combining public ELPV dataset, our custom real-world dataset and synthetic images created with different GAN architectures such as GAN, cGAN and WGAN-GP. The datasets were classified through proposed customized VGG-16 and other state-of-art CNN models. The best results obtained by the proposed CNN model with WGAN-GP augmented dataset are 94.11% mean accuracy in the ELPV test dataset and 93.08% mean accuracy in the custom real-world test dataset. Therefore, the proposed detection system achieved superior performance with lesser resource usage and limited data.

Deep generative networks for multivariate fullstack seismic data inversion using inverse autoregressive flows

The simultaneous prediction of the subsurface distribution of facies and acoustic impedance (IP) from fullstack seismic data requires solving an inverse problem and is fundamental in natural resources exploration, carbon capture and storage, and environmental risk management. In recent years, deep generative models (DGM), such as variational autoencoders (VAE) and generative adversarial networks (GAN), were proposed to reproduce complex facies patterns honoring prior geological information. Variational Bayesian inference using inverse autoregressive flows (IAF) can be performed to infer the solution to a geophysical inverse problem from the encoded latent space of such pre-trained DGM. Successful applications of such approach on crosshole ground-penetrating radar synthetic data inversion demonstrated that the technique's accuracy is comparable to that of Markov chain Monte Carlo (MCMC) inference methods, while significantly reducing the computational cost. Nonetheless, these application examples did not account for the spatial uncertainty affecting the facies-dependent continuous physical property, from which the geophysical data are calculated. This uncertainty can significantly affect the inversion accuracy and its applicability to real data. In this work, specific VAE and GAN architectures are proposed to simultaneously predict facies and co-located IP, while accounting for their spatial uncertainties. The two types of generative networks are used in Bayesian inversion with IAF for the inversion of seismic data. The results are found to reproduce the statistics of the training images and solve the seismic inversion problem accurately, comparably to MCMC inversion. Furthermore, advantages and limitations of the two DGMs are evaluated by comparing the results obtained.

A conditional generative adversarial networks and Yolov5 Darknet -based skin lesion localization and classification using independent component analysis model

Automated diagnosis systems significantly improve the early detection of skin cancer, thereby contributing to its successful treatment. Skin lesion segmentation represents a crucial yet challenging issue within the realm of computer-assisted dermoscopy image analysis. Numerous segmentation techniques reliant on convolutional neural networks falter in achieving precise lesion boundaries due to the diminishing spatial scale of feature maps as they traverse through network layers. Nonetheless, neural networks demand a substantial amount of data, which presents a notable obstacle in the medical field. More data, and possibly a pre-trained model tailored to medical images, are imperative. Researchers have resorted to extensive augmentation and preprocessing methods, as well as fine-tuning the network using pre-trained models on unrelated images, in order to address the challenge of lesion segmentation as the initial step in skin cancer analysis. This arises from the fact that datasets are limited in size and encompass a diverse array of images, varying in terms of lighting, color, scale, and markings. The proposed method addresses the issue of precisely locating and classifying skin lesions in dermatological images. The architecture of Conditional Generative Adversarial Networks (cGAN) enables the generation of high-resolution, contextually pertinent images, thereby enhancing the discriminative properties of skin lesions. YOLOv5 Darknet has an effective object detection mechanism that aids in precise lesion localization. Independent component analysis (ICA), a signal processing technique that identifies statistically separate components from diverse data sources, is employed to improve the model's robustness and interpretability further. This facilitates extracting latent features from the combined dermatological images and data collection, thereby enhancing the model's understanding of complex lesion characteristics. According to experimental results on an extensive dermatological data set, the proposed cGAN-YOLOv5 technique is effective. In pinpointing and classifying lesion locations, the hybrid model outperforms the competition. The synergistic integration of cGANs, YOLOv5 Darknet, and ICA demonstrates the effectiveness of multifaceted integration in enhancing medical image analysis. Using high-resolution feature blocks and a noise-suppressing attention method, cGAN-YOLOv5 is proposed. Therefore, the proposed strategy segments skin lesions effectively.

Physically‐Based Lighting for 3D Generative Models of Cars

AbstractRecent work has demonstrated that Generative Adversarial Networks (GANs) can be trained to generate 3D content from 2D image collections, by synthesizing features for neural radiance field rendering. However, most such solutions generate radiance, with lighting entangled with materials. This results in unrealistic appearance, since lighting cannot be changed and view‐dependent effects such as reflections do not move correctly with the viewpoint. In addition, many methods have difficulty for full, 360° rotations, since they are often designed for mainly front‐facing scenes such as faces. We introduce a new 3D GAN framework that addresses these shortcomings, allowing multi‐view coherent 360° viewing and at the same time relighting for objects with shiny reflections, which we exemplify using a car dataset. The success of our solution stems from three main contributions. First, we estimate initial camera poses for a dataset of car images, and then learn to refine the distribution of camera parameters while training the GAN. Second, we propose an efficient Image‐Based Lighting model, that we use in a 3D GAN to generate disentangled reflectance, as opposed to the radiance synthesized in most previous work. The material is used for physically‐based rendering with a dataset of environment maps. Third, we improve the 3D GAN architecture compared to previous work and design a careful training strategy that allows effective disentanglement. Our model is the first that generate a variety of 3D cars that are multi‐view consistent and that can be relit interactively with any environment map.

Generated realistic noise and rotation-equivariant models for data-driven mesh denoising

3D mesh denoising is a crucial pre-processing step in many graphics applications. However, existing data-driven mesh denoising models, primarily trained on synthetic white noise, are less effective when applied to real-world meshes with the noise of complex intensities and distributions. Moreover, how to comprehensively capture information from input meshes and apply suitable denoising models for feature-preserving mesh denoising remains a critical and unresolved challenge. This paper presents a rotation-Equivariant model-based Mesh Denoising (EMD) model and a Realistic Mesh Noise Generation (RMNG) model to address these issues. Our EMD model leverages rotation-equivariant features and self-attention weights of geodesic patches for more effective feature extraction, thereby achieving SOTA denoising results. The RMNG model, based on the Generative Adversarial Networks (GANs) architecture, generates massive amounts of realistic noisy and noiseless mesh pairs data for data-driven mesh denoising model training, significantly benefiting real-world denoising tasks. To address the smooth degradation and loss of sharp edges commonly observed in captured meshes, we further introduce varying levels of Laplacian smoothing to input meshes during the paired training data generation, endowing the trained denoising model with feature recovery capabilities. Experimental results demonstrate the superior performance of our proposed method in preserving fine-grained features while removing noise on real-world captured meshes.

Generating Synthetic Electronic Health Record Data Using Generative Adversarial Networks: Tutorial

Synthetic electronic health record (EHR) data generation has been increasingly recognized as an important solution to expand the accessibility and maximize the value of private health data on a large scale. Recent advances in machine learning have facilitated more accurate modeling for complex and high-dimensional data, thereby greatly enhancing the data quality of synthetic EHR data. Among various approaches, generative adversarial networks (GANs) have become the main technical path in the literature due to their ability to capture the statistical characteristics of real data. However, there is a scarcity of detailed guidance within the domain regarding the development procedures of synthetic EHR data. The objective of this tutorial is to present a transparent and reproducible process for generating structured synthetic EHR data using a publicly accessible EHR data set as an example. We cover the topics of GAN architecture, EHR data types and representation, data preprocessing, GAN training, synthetic data generation and postprocessing, and data quality evaluation. We conclude this tutorial by discussing multiple important issues and future opportunities in this domain. The source code of the entire process has been made publicly available.

A novel approach for improving open scene text translation with modified GAN

AbstractText, as a vital tool for communication, is playing an imperative role in modern society. Precise high-level text translation systems are essential requirements in a wide range of real-world applications, such as robot navigation, industrial automation, image search, and instant translation. Regardless of improved research, a series of grand challenges may still become upon when translating text automatically in the real-world from open scene images. The difficulties mainly stem from multiplicity and inconsistency of text in open scenes, complication and obstruction of backgrounds, and deficient imaging conditions in uncontrolled circumstances for open scene images. The existing deep learning-based text translation systems do not eliminate the text for translation, and these applications just replace text on the reconstructed scene. To address the abovementioned shortcomings, this study proposed a novel approach for open scene text translation. Our system consists of five modules including scene text detection, text recognition, text elimination, text translation, and text insertion along with scene reconstruction. The novelty presented by our model lies in the idea of first eliminating the text from the open scene for accurate translation and then reconstructs the translated text on the image for its proper alignment. We specifically modified the existing generative adversarial network (GAN) architecture for improved performance of text elimination by introducing a novel strategy of text and scene concatenation to reduce the overall loss function. For this purpose, we created a synthetic dataset to train our GAN for text elimination module. Experiments on various standard text translation systems demonstrate that our integrated system is able to outperform state-of-the-art approaches in terms of result quality. We have achieved 90.87% of precision, 83.66% of recall, 87.116% of F1-score, and reduced both losses ($$l_1$$ l 1 and $$l_2$$ l 2 ) up to 50% which is remarkable upon state-of-the-art translation systems.

GANs in the Panorama of Synthetic Data Generation Methods

This paper focuses on the creation and evaluation of synthetic data to address the challenges of imbalanced datasets in machine learning applications (ML), using fake news detection as a case study. We conducted a thorough literature review on generative adversarial networks (GANs) for tabular data, synthetic data generation methods, and synthetic data quality assessment. By augmenting a public news dataset with synthetic data generated by different GAN architectures, we demonstrate the potential of synthetic data to improve ML models’ performance in fake news detection. Our results show a significant improvement in classification performance, especially in the underrepresented class. We also modify and extend a data usage approach to evaluate the quality of synthetic data and investigate the relationship between synthetic data quality and data augmentation performance in classification tasks. We found a positive correlation between synthetic data quality and performance in the underrepresented class, highlighting the importance of high-quality synthetic data for effective data augmentation.

Audio-deepfake detection: Adversarial attacks and countermeasures

Audio has always been a powerful resource for biometric authentication: thus, numerous AI-based audio authentication systems (classifiers) have been proposed. While these classifiers are effective in identifying legitimate human-generated input their security, to the best of our knowledge, has not been explored thoroughly when confronted with advanced attacks that leverage AI-generated deepfake audio. This issue presents a serious concern regarding the security of these classifiers because, e.g., samples generated using adversarial attacks might fool such classifiers, resulting in incorrect classification. In this study, we prove the point: we demonstrate that state-of-the-art audio deepfake classifiers are vulnerable to adversarial attacks. In particular, we design two adversarial attacks on a state-of-the-art audio-deepfake classifier, i.e., the Deep4SNet classification model, which achieves 98.5% accuracy in detecting fake audio samples. The designed adversarial attacks11The code of the attacks will be released open-source in the camera ready. leverage a generative adversarial network architecture and reduce the detector’s accuracy to nearly 0%. In particular, under graybox attack scenarios, we demonstrate that when starting from random noise, we can reduce the accuracy of the state-of-the-art detector from 98.5% to only 0.08%. To mitigate the effect of adversarial attacks on audio-deepfake detectors, we propose a highly generalizable, lightweight, simple, and effective add-on defense mechanism that can be implemented in any audio-deepfake detector. Finally, we discuss promising research directions.

Garbage In [formula omitted] Garbage Out: Exploring GAN Resilience to Image Training Set Degradations

Generative Adversarial Networks (GANs) have received immense attention in recent years due to their ability to capture complex, high-dimensional data distributions without the need for extensive labeling. Since their conception in 2014, a wide array of GAN variants have been proposed featuring alternative architectures, optimizers, and loss functions with the goal of improving performance and training stability. This manuscript focuses on quantifying the resilience of a GAN architecture to specific modes of image degradation. We conduct systematic experimentation to empirically determine the effects of 10 fundamental image degradation modes, applied to the training image dataset, on the Fréchet inception distance (FID) of images generated by a conditional deep convolutional GAN (cDCGAN). We find that at the α=0.05 level, brightening, darkening, and blurring are statistically significantly more detrimental to the resulting GAN image quality than removing the degraded data completely, while other degradations are typically safe to keep in training datasets. Additionally, we find that in the case of randomized partial occlusion, the FID of the resulting GAN images approaches that of the degraded training set for increasing levels of occlusion, with the surprising result that GAN FID performance is equal to that of the training set at 75% degradation.

Robust deep learning for eye fundus images: Bridging real and synthetic data for enhancing generalization

Deep learning applications for assessing medical images are limited because the datasets are often small and imbalanced. The use of synthetic data has been proposed in the literature, but neither a robust comparison of the different methods nor generalizability has been reported. Our approach integrates a retinal image quality assessment model and StyleGAN2 architecture to enhance Age-related Macular Degeneration (AMD) detection capabilities and improve generalizability. This work compares ten different Generative Adversarial Network (GAN) architectures to generate synthetic eye-fundus images with and without AMD. We combined subsets of three public databases (iChallenge-AMD, ODIR-2019, and RIADD) to form a single training and test set. We employed the STARE dataset for external validation, ensuring a comprehensive assessment of the proposed approach. The results show that StyleGAN2 reached the lowest Fréchet Inception Distance (166.17), and clinicians could not accurately differentiate between real and synthetic images. ResNet-18 architecture obtained the best performance with 85% accuracy and outperformed the two human experts (80% and 75%) in detecting AMD fundus images. The accuracy rates were 82.8% for the test set and 81.3% for the STARE dataset, demonstrating the model’s generalizability. The proposed methodology for synthetic medical image generation has been validated for robustness and accuracy, with free access to its code for further research and development in this field.