Generative Adversarial Networks Research Articles

Transfer learning enabled transformer-based generative adversarial networks for modeling and generating terahertz channels

Terahertz communications are envisioned as a promising technology for the sixth generation and beyond wireless systems, which can support wireless links with Terabits-per-second (Tbps) data rates. As the foundation of designing terahertz communications, channel modeling and characterization are crucial to scrutinize the potential of this spectrum. However, current channel modeling in the terahertz band heavily relies on time-consuming and costly measurements. Here, we propose a transfer learning enabled transformer based generative adversarial network to mitigate this problem in terahertz channel modeling. Specifically, as a fundamental building block, a generative adversarial network is exploited to generate channel parameters. To improve the accuracy, a transformer structure with a self-attention mechanism is incorporated in generative adversarial network. Still incurring errors compared with ground-truth measurement, a transfer learning is designed to solve the mismatch between the formulated network and measurement. The proposed method can achieve high accuracy in channel modeling, while requiring only rather limited amount of measurement, which is a promising complement of current channel modeling techniques.

Coinciding Diabetic Retinopathy and Diabetic Macular Edema Grading With Rat Swarm Optimization Algorithm for Enhanced Capsule Generation Adversarial Network.

In the worldwide working-age population, visual disability and blindness are common conditions caused by diabetic retinopathy (DR) and diabetic macular edema (DME). Nowadays, due to diabetes, many people are affected by eye-related issues. Among these, DR and DME are the two foremost eye diseases, the severity of which may lead to some eye-related problems and blindness. Early detection of DR and DME is essential to preventing vision loss. Therefore, an enhanced capsule generation adversarial network (ECGAN) optimized with the rat swarm optimization (RSO) approach is proposed in this article to coincide with DR and DME grading (DR-DME-ECGAN-RSO-ISBI 2018 IDRiD). The input images are obtained from the ISBI 2018 unbalanced DR grading data set. Then, the input fundus images are preprocessed using the Savitzky-Golay (SG) filter filtering technique, which reduces noise from the input image. The preprocessed image is fed to the discrete shearlet transform (DST) for feature extraction. The extracting features of DR-DME are given to the ECGAN-RSO algorithm to categorize the grading of DR and DME disorders. The proposed approach is implemented in Python and achieves better accuracy by 7.94%, 36.66%, and 4.88% compared to the existing models, such as the combined DR with DME grading for the cross-disease attention network (DR-DME-CANet-ISBI 2018 IDRiD), category attention block for unbalanced grading of DR (DR-DME-HDLCNN-MGMO-ISBI 2018 IDRiD), combined DR-DME classification with a deep learning-convolutional neural network-based modified gray-wolf optimizer with variable weights (DR-DME-ANN-ISBI 2018 IDRiD).

Wearing myopia glasses on GANs: Mitigating bias for pre-trained Generative Adversarial Networks via online prior perturbation

Pre-trained Generative Adversarial Networks (GANs) can provide rich information and make various downstream tasks beneficial. However, the training process of GANs potentially learns pre-existing biases in the historical data, which are not only inherited, but also amplified to some extent when using the generated data for prediction. Most available methods for bias mitigation utilize pre-processing or in-processing strategies to retrain GANs, which needs to access the raw data and also increases the time and computational cost required to train a fully expressive GAN. Therefore, we propose a novel post-processing mechanism to achieve fairness outputs of GANs through online prior perturbation. Unlike traditional output-based offline processing, our method mitigates biases from model inputs by designing a prior perturbation network, called “prior perturber”, to form a combined network with the pre-trained GAN. Specifically, we introduce a bias prediction network, called “bias predictor”, for online adversarial training with the prior perturber, which can decouple the generated representations from the bias features and improve the fairness of the downstream prediction results. In addition, the training process is independent of specific target labels, and the generated fair representations have good transferability. Experimental evaluations on five real datasets validate the effectiveness of the proposed method and the best utility is obtained on the same fairness level as compared to other output-based baseline methods. The result on fairness-utility trade-offs significantly improves the fairness of original samples by 56.40%, while the utility is reduced by only 2.60%.

Missing Data Imputation for Industrial Time Series With Adaptive Median Iteration Based on Generative Adversarial Networks

Missing Data Imputation for Industrial Time Series With Adaptive Median Iteration Based on Generative Adversarial Networks

Image-Based Generative Artificial Intelligence in Radiology: Comprehensive Updates.

Generative artificial intelligence (AI) has been applied to images for image quality enhancement, domain transfer, and augmentation of training data for AI modeling in various medical fields. Image-generative AI can produce large amounts of unannotated imaging data, which facilitates multiple downstream deep-learning tasks. However, their evaluation methods and clinical utility have not been thoroughly reviewed. This article summarizes commonly used generative adversarial networks and diffusion models. In addition, it summarizes their utility in clinical tasks in the field of radiology, such as direct image utilization, lesion detection, segmentation, and diagnosis. This article aims to guide readers regarding radiology practice and research using image-generative AI by 1) reviewing basic theories of image-generative AI, 2) discussing the methods used to evaluate the generated images, 3) outlining the clinical and research utility of generated images, and 4) discussing the issue of hallucinations.

Advancements in Data Augmentation and Transfer Learning: A Comprehensive Survey to Address Data Scarcity Challenges

Abstract: Deep Learning (DL) models have demonstrated remarkable proficiency in image classification and recognition tasks, surpassing human capabilities. The observed enhancement in performance can be attributed to the utilization of extensive datasets. Nevertheless, DL models have huge data requirements. Widening the learning capability of such models from limited samples even today remains a challenge, given the intrinsic constraints of small datasets. The trifecta of challenges, encompassing limited labeled datasets, privacy, poor generalization performance, and the costliness of annotations, further compounds the difficulty in achieving robust model performance. Overcoming the challenge of expanding the learning capabilities of Deep Learning models with limited sample sizes remains a pressing concern even today. To address this critical issue, our study conducts a meticulous examination of established methodologies, such as Data Augmentation and Transfer Learning, which offer promising solutions to data scarcity dilemmas. Data Augmentation, a powerful technique, amplifies the size of small datasets through a diverse array of strategies. These encompass geometric transformations, kernel filter manipulations, neural style transfer amalgamation, random erasing, Generative Adversarial Networks, augmentations in feature space, and adversarial and meta- learning training paradigms. : Furthermore, Transfer Learning emerges as a crucial tool, leveraging pre-trained models to facilitate knowledge transfer between models or enabling the retraining of models on analogous datasets. Through our comprehensive investigation, we provide profound insights into how the synergistic application of these two techniques can significantly enhance the performance of classification tasks, effectively magnifying scarce datasets. This augmentation in data availability not only addresses the immediate challenges posed by limited datasets but also unlocks the full potential of working with Big Data in a new era of possibilities in DL applications.

Partial Discharge Data Augmentation and Pattern Recognition Method Based on DAE-GAN

Accurate identification of partial discharge (PD) and its types is essential for assessing the operating conditions of electrical equipment. To enhance PD pattern recognition under imbalanced and limited sample conditions, a method based on a Deep Autoencoder-embedded Generative Adversarial Network (DAE-GAN) is proposed. First, the Deep Autoencoder (DAE) is embedded within the Generative Adversarial Network (GAN) to improve the realism of generated samples. Then, complementary PD data samples are introduced during GAN training to address the issue of limited sample size. Lastly, the model’s discriminator is fine-tuned with augmented and balanced training data to enable PD pattern recognition. The DAE-GAN method is used to augment data and recognize patterns in experimental PD signals. The results demonstrate that, under imbalanced and small sample conditions, DAE-GAN generates more authentic PD samples with improved probability distribution fitting compared to other algorithms, leading to varying levels of enhancement in pattern recognition accuracy.

A systematic review of deep learning-based denoising for low-dose computed tomography from a perceptual quality perspective.

Low-dose computed tomography (LDCT) scans are essential in reducing radiation exposure but often suffer from significant image noise that can impair diagnostic accuracy. While deep learning approaches have enhanced LDCT denoising capabilities, the predominant reliance on objective metrics like PSNR and SSIM has resulted in over-smoothed images that lack critical detail. This paper explores advanced deep learning methods tailored specifically to improve perceptual quality in LDCT images, focusing on generating diagnostic-quality images preferred in clinical practice. We review and compare current methodologies, including perceptual loss functions and generative adversarial networks, addressing the significant limitations of current benchmarks and the subjective nature of perceptual quality evaluation. Through a systematic analysis, this study underscores the urgent need for developing methods that balance both perceptual and diagnostic quality, proposing new directions for future research in the field.

Integrated deep learning approach for generating cross-polarized images and analyzing skin melanin and hemoglobin distributions.

Cross-polarized images are beneficial for skin pigment analysis due to the enhanced visualization of melanin and hemoglobin regions. However, the required imaging equipment can be bulky and optically complex. Additionally, preparing ground truths for training pigment analysis models is labor-intensive. This study aims to introduce an integrated approach for generating cross-polarized images and creating skin melanin and hemoglobin maps without the need for ground truth preparation for pigment distributions. We propose a two-component approach: a cross-polarized image generation module and a skin analysis module. Three generative adversarial networks (CycleGAN, pix2pix, and pix2pixHD) are compared for creating cross-polarized images. The regression analysis network for skin analysis is trained with theoretically reconstructed ground truths based on the optical properties of pigments. The methodology is evaluated using the VISIA VAESTRO clinical system. The cross-polarized image generation module achieved a peak signal-to-noise ratio of 35.514dB. The skin analysis module demonstrated correlation coefficients of 0.942 for hemoglobin and 0.922 for melanin. The integrated approach yielded correlation coefficients of 0.923 for hemoglobin and 0.897 for melanin, respectively. The proposed approach achieved a reasonable correlation with the professional system using actually captured images, offering a promising alternative to existing professional equipment without the need for additional optical instruments or extensive ground truth preparation.

Modeling complex polycrystalline alloys using a Generative Adversarial Network enabled computational platform

Creating statistically equivalent virtual microstructures (SEVM) for polycrystalline materials with complex microstructures that encompass multi-modal morphological and crystallographic distributions is a challenging enterprise. Cold spray-formed (CSF) AA7050 alloy containing coarse-grained prior particles and ultra-fine grains (UFG) and additively manufactured (AM) Ti64 alloys with alpha laths in beta substrates. The paper introduces an approach strategically integrating a Generative Adversarial Network (GAN) for multi-modal microstructures with a synthetic microstructure builder DREAM.3D for packing grains conforming to statistics in electron backscatter diffraction (EBSD) maps for generating SEVMs of CSF and AM alloy microstructures. A robust multiscale model is subsequently developed for self-consistent coupling of crystal plasticity finite element model (CPFEM) for coarse-grained crystals with an upscaled constitutive model for UFGs. Sub-volume elements are simulated for efficient computations and their responses are averaged for overall stress-strain response. The methods developed are important for image-based micromechanical modeling that is necessary for microstructure-property relations.

Style-Controlled Image Synthesis of Concrete Damages Based on Fusion of Convolutional Encoder and Attention-Enhanced Conditional Generative Adversarial Network

Style-Controlled Image Synthesis of Concrete Damages Based on Fusion of Convolutional Encoder and Attention-Enhanced Conditional Generative Adversarial Network

Fluid Grey 2: How Well Does Generative Adversarial Networks Learn Deeper Topology Structure in Architecture That Matches Images?

Fluid Grey 2: How Well Does Generative Adversarial Networks Learn Deeper Topology Structure in Architecture That Matches Images?

A StarGAN and transformer-based hybrid classification-regression model for multi-institution VMAT patient-specific quality assurance.

The field of artificial intelligence (AI)-based patient-specific quality assurance (PSQA) for volumetric modulated arc therapy (VMAT) faces challenges in terms of developing general models across institutions due to the prevalence of multi-institution data collection and multivariate heterogeneity. Building a general model that is capable of handling diverse multi-institution data is critical for enabling large-scale integration and analysis. This study aims to develop a star generative adversarial network (StarGAN) and transformer-based hybrid classification-regression PSQA framework to address unification of heterogeneous data from different institutions. A StarGAN and transformer-based hybrid classification-regression model was developed as a general PSQA framework to predict gamma passing rates (GPRs) and classify quality assurance (QA) results as "Pass" or "Fail" at multiple institutions. A total of 1815 VMAT plans were collected from eight institutions to develop the general PSQA framework and perform clinical commissioning and implementation. Among them, 20 independent clinical plans from each of eight institutions, for a total of 160 plans, were used for the clinical commissioning, and 205 new clinical plans from eight institutions were used for clinical implementation. For the 3%/3, 3%/2, and 2%/2mm gamma criteria, the sensitivity of the proposed PSQA framework with pretraining was 90.13%, 92.03%, and 95.84%, respectively, while the specificity was 76.01%, 76.12%, and 85.34%, respectively. The mean absolute errors (MAEs) of the proposed PSQA framework with pretraining were 1.36%, 2.37%, and 3.96%, respectively, while the root-mean-square errors (RMSEs) were 2.31%, 3.89%, and 5.17%, respectively. The results demonstrated visible improvement at multiple institutions. For clinical commissioning, the deviations between the predicted and measured results were all within 3% for 3%/3 and 3%/2mm at eight institutions. For clinical implementation, all failure plans were correctly identified by the proposed PSQA framework. The general PSQA framework enables diverse clinical data sources to be handled to achieve enhanced model performance and generalizability, and provides a solution to the unification of heterogeneous data from different institutions to construct robust QA models. This approach can be clinically deployed for VMAT QA.

Restoration of X-ray phase-contrast imaging based on generative adversarial networks

For light-element materials, X-ray phase contrast imaging provides better contrast compared to absorption imaging. While the Fourier transform method has a shorter imaging time, it typically results in lower image quality; in contrast, the phase-shifting method offers higher image quality but is more time-consuming and involves a higher radiation dose. To rapidly reconstruct low-dose X-ray phase contrast images, this study developed a model based on Generative Adversarial Networks (GAN), incorporating custom layers and self-attention mechanisms to recover high-quality phase contrast images. We generated a simulated dataset using Kaggle’s X-ray data to train the GAN, and in simulated experiments, we achieved significant improvements in Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). To further validate our method, we applied it to fringe images acquired from three phase contrast systems: a single-grating phase contrast system, a Talbot-Lau system, and a cascaded grating system. The current results demonstrate that our method successfully restored high-quality phase contrast images from fringe images collected in experimental settings, though it should be noted that these results were achieved using relatively simple sample configurations.

Resolution-enhanced quantitative phase imaging of blood platelets using a generative adversarial network

We developed a new method to enhance the resolution of blood platelet aggregates imaged via quantitative phase imaging (QPI) using a Pix2Pix generative adversarial network (GAN). First, 1 µm polystyrene beads were imaged with low- and high-resolution QPI, to train the GAN model and validate its applicability. Testing on the polystyrene beads demonstrated a mean error of 4.14% in the generated high-resolution optical-path-delay values compared to the optically acquired ones. Next, blood platelets were collected with low- and high-resolution QPI, and a deep neural network was trained to predict the high-resolution platelet optical-path-delay profiles using the low-resolution profiles, achieving a mean error of 7.01% in the generated high-resolution optical-path-delay values compared to the optically acquired ones. These results highlight the potential of the method in enhancing QPI resolution of cell aggregates without the need for sophisticated optical equipment and optical system modifications for high-resolution microscopy, allowing for better understanding of platelet-related disorders and conditions such as thrombocytopenia and thrombocytosis.

Simultaneous single image super‐resolution and blind Gaussian denoising via slim ghost full‐frequency residual blocks

AbstractGiven that super‐resolution (SR) aims to recover lost information, and low‐resolution (LR) images in real‐world conditions might be corrupted with multiple degradations, considering basic bicubic down‐sampling as the sole degradation significantly limits the performance of most existing SR models. This paper presents a model for simultaneous super‐resolution and blind additive white Gaussian noise (AWGN) denoising with two components (netdeg and netSR) that is based on a generative adversarial network (GAN) to achieve detailed results. netdeg, featuring residual and innovative cost‐effective ghost residual blocks with a frequency separation module for obtaining long‐range information, blindly restores a clean version of the LR image. netSR leverages slim ghost full‐frequency residual blocks to process low‐frequency (LF) and high‐frequency (HF) information via static large convolutions and pixel‐wise highlighted input‐adaptive dynamic convolutions, respectively. To address the susceptibility of dynamic layers to noise and preserve feature diversity while reducing model’s costs, static and dynamic layer features are combined and highlighted. Diverse and non‐redundant features are then processed using ghost‐style blocks. The proposed model achieves comparable SR results in bicubic down‐sampling scenarios, outperform existing SR methods in the complex task of concurrent SR and AWGN denoising, and demonstrate robustness in handling images corrupted with varying levels of AWGN.

An Empirical Study of WGAN and WGAN-GP for Enhanced Image Generation

This paper aims to advance the Wasserstein Generative Adversarial Networks (WGANs) and their enhancements, particularly focusing on the gradient penalty. Generative Adversarial Networks (GANs), introduced by Goodfellow et al. in 2014, have revolutionized the domain of image generation. To address the limitations of GANs, the WGAN was proposed. However, WGANs rely on weight clipping, which introduces its own set of issues such as slow convergence and potential gradient vanishing. The inefficiency and instability of WGANs have troubled its users. To solve these problems, WGAN with Gradient Penalty (WGAN-GP) was developed to address these challenges. It provides more stable gradients and reduces the risk of mode collapse by using a gradient penalty to enforce the necessary constraints. In this paper, the author implemented both WGAN and WGAN with Gradient Penalty (WGAN-GP) and evaluated them using the CIFAR-10 and MNIST datasets. The results show that WGAN-GP's outputs are more stable and efficient in the early rounds, confirming the effectiveness of the gradient penalty in training image datasets.

A Review of Advancement in using Generative AI for Detecting Skin Cancer

Abstract: Skin cancer is a very common type of cancer that is asymptomatic in the early stage of development and is difficult to diagnose and treat. More and more people have used generative AI methods to enhance the speed and accuracy of skin cancer prediction in the recent past. Current techniques of artificial intelligence and deep learning algorithms are performing quite effectively in the field of dermoscopy and identification of malignant lesions. This study overviews the recent developments in Skin Cancer Detection Approaches based on emerging AI methods like Generative Adversarial Networks (GAN); Variational AutoEncoders (VAE); and derivatives of both. In this study, we discussed several approaches to implementing generative AI for skin cancer detection, the advantages and disadvantages of each approach, and identified research that has revealed potential in the field

Active Machine Learning for Pre-procedural Prediction of Time-Varying Boundary Condition After Fontan Procedure Using Generative Adversarial Networks.

The Fontan procedure is the definitive palliation for pediatric patients born with single ventricles. Surgical planning for the Fontan procedure has emerged as a promising vehicle toward optimizing outcomes, where pre-operative measurements are used prospectively as post-operative boundary conditions for simulation. Nevertheless, actual post-operative measurements can be very different from pre-operative states, which raises questions for the accuracy of surgical planning. The goal of this study is to apply machine leaning techniques to describing pre-operative and post-operative vena caval flow conditions in Fontan patients in order to develop predictions of post-operative boundary conditions to be used in surgical planning. Based on a virtual cohort synthesized by lumped-parameter models, we proposed a novel diversity-aware generative adversarial active learning framework to successfully train predictive deep neural networks on very limited amount of cases that are generally faced by cardiovascular studies. Results of 14 groups of experiments uniquely combining different data query strategies, metrics, and data augmentation options with generative adversarial networks demonstrated that the highest overall prediction accuracy and coefficient of determination were exhibited by the proposed method. This framework serves as a first step toward deep learning for cardiovascular flow prediction/regression with reduced labeling requirements and augmented learning space.

Super-resolution imaging quality enhancement method for distributed array infrared camera

Abstract To address issues related to low resolution and high noise in infrared cameras, a distributed array infrared camera imaging system utilizing four cameras is proposed. The four cameras are arranged in an unconstrained array, and the combination algorithm of Projections onto Convex Sets (POCS) and Real-Enhanced Super-Resolution Generative Adversarial Networks (Real-ESRGAN) is applied to achieve high-quality super-resolution infrared imaging. The wavelet fusion algorithm is used to preprocess four low-resolution infrared images to reduce noise. Then, the POCS algorithm is used to reconstruct the preprocessed image. Finally, the Real-ESRGAN is employed for image reconstruction, resulting in an ultra-high-resolution infrared image. The results show that compared to single infrared camera imaging, the resolution of images reconstructed using the distributed infrared camera array is increased by 0.58 times, with the Modulation Transfer Function (MTF) increased by 1.2 times. Additionally, the entropy is increased by 18.87%, the standard deviation is increased by 13.51%, and the Naturalness Image Quality Evaluator (NIQE) is reduced by 18.87%. This demonstrates a significant enhancement in the super-resolution imaging quality of the distributed infrared camera array.