Conditional Generative Adversarial Network Research Articles

Fringe pattern normalization using conditional Generative Adversarial Networks

Fringe pattern normalization is an important pre-processing operation in optical fringe analysis, and is usually required as the initial step for several phase retrieval methods. However, noise and rapid amplitude fluctuations in the fringe pattern pose significant challenges for fringe pattern normalization. In this paper, we propose a robust approach for fringe normalization using conditional Generative Adversarial Network, which works on the principle of image-to-image translation with conditioning on input images. The performance of the proposed method under different noise conditions and amplitude variations is demonstrated using numerical simulations. Further, the practical applicability of the proposed method is also tested with experimental data obtained in digital holographic interferometry.

UWAFA-GAN: Ultra-Wide-Angle Fluorescein Angiography Transformation via Multi-Scale Generation and Registration Enhancement.

Fundus photography, in combination with the ultra-wide-angle fundus (UWF) techniques, becomes an indispensable diagnostic tool in clinical settings by offering a more comprehensive view of the retina. Nonetheless, UWF fluorescein angiography (UWF-FA) necessitates the administration of a fluorescent dye via injection into the patient's hand or elbow unlike UWF scanning laser ophthalmoscopy (UWF-SLO). To mitigate potential adverse effects associated with injections, researchers have proposed the development of cross-modality medical image generation algorithms capable of converting UWF-SLO images into their UWF-FA counterparts. Current image generation techniques applied to fundus photography encounter difficulties in producing high-resolution retinal images, particularly in capturing minute vascular lesions. To address these issues, we introduce a novel conditional generative adversarial network (UWAFA-GAN) to synthesize UWF-FA from UWF-SLO. This approach employs multi-scale generators and an attention transmit module to efficiently extract both global structures and local lesions. Additionally, to counteract the image blurriness issue that arises from training with misaligned data, a registration module is integrated within this framework. Our method performs non-trivially on inception scores and details generation. Clinical user studies further indicate that the UWF-FA images generated by UWAFA-GAN are clinically comparable to authentic images in terms of diagnostic reliability. Empirical evaluations on our proprietary UWF image datasets elucidate that UWAFA-GAN outperforms extant methodologies.

A data augmentation method for multiaxial fatigue life prediction based on physics-informed tabular generative adversarial network

Abstract Currently, in the multiaxial fatigue life prediction problem, machine learning (ML) algorithms are still limited by small samples. For ML tasks requiring large amounts of data, this paper proposes a Conditional Tabular Generative Adversarial Network (CTGAN) model using a physical equation as a generation constraint. The proposed method can make synthetic samples satisfy specific physical knowledge. The method is evaluated using two datasets of different materials. ML-based algorithms verify the feasibility of using synthetic datasets for life prediction. Verification analysis shows that the proposed method successfully synthesizes high-quality multiaxial fatigue datasets and effectively improves the prediction accuracy of ML algorithms.

Carbon market risk estimation using quantum conditional generative adversarial network and amplitude estimation

AbstractAccurately and efficiently estimating the carbon market risk is paramount for ensuring financial stability, promoting environmental sustainability, and facilitating informed decision‐making. Although classical risk estimation methods are extensively utilized, the implicit pre‐assumptions regarding distribution are predominantly contained and challenging to balance accuracy and computational efficiency. A quantum computing‐based carbon market risk estimation framework is proposed to address this problem with the quantum conditional generative adversarial network‐quantum amplitude estimation (QCGAN‐QAE) algorithm. Specifically, quantum conditional generative adversarial network (QCGAN) is employed to simulate the future distribution of the generated return rate, whereas quantum amplitude estimation (QAE) is employed to measure the distribution. Moreover, the quantum circuit of the QCGAN improved by reordering the data interaction layer and data simulation layer is coupled with the introduction of the quantum fully connected layer. The binary search method is incorporated into the QAE to bolster the computational efficiency. The simulation results based on the European Union Emissions Trading System reveals that the proposed framework markedly enhances the efficiency and precision of Value‐at‐Risk and Conditional Value‐at‐Risk compared to original methods.

ASD-GANNet: A Generative Adversarial Network-Inspired Deep Learning Approach for the Classification of Autism Brain Disorder.

The classification of a pre-processed fMRI dataset using functional connectivity (FC)-based features is considered a challenging task because of the set of high-dimensional FC features and the small dataset size. To tackle this specific set of FC high-dimensional features and a small-sized dataset, we propose here a conditional Generative Adversarial Network (cGAN)-based dataset augmenter to first train the cGAN on computed connectivity features of NYU dataset and use the trained cGAN to generate synthetic connectivity features per category. After obtaining a sufficient number of connectivity features per category, a Multi-Head attention mechanism is used as a head for the classification. We name our proposed approach "ASD-GANNet", which is end-to-end and does not require hand-crafted features, as the Multi-Head attention mechanism focuses on the features that are more relevant. Moreover, we compare our results with the six available state-of-the-art techniques from the literature. Our proposed approach results using the "NYU" site as a training set for generating a cGAN-based synthetic dataset are promising. We achieve an overall 10-fold cross-validation-based accuracy of 82%, sensitivity of 82%, and specificity of 81%, outperforming available state-of-the art approaches. A sitewise comparison of our proposed approach also outperforms the available state-of-the-art, as out of the 17 sites, our proposed approach has better results in the 10 sites.

FloorDiffusion: Diffusion model-based conditional floorplan image generation method using parameter-efficient fine-tuning and image inpainting

The conditional generation of high-quality floorplan images using deep-learning methods is challenging because the generated floorplans are required to match specific conditions, such as floorplan silhouettes and spatial layouts. Recently, diffusion models have emerged as alternatives of conditional generative adversarial networks in image generation, offering higher image quality, pairing-free training datasets, and adaptability to various image domains via parameter fine-tuning of pretrained diffusion models. However, diffusion models are rarely used for floorplan generation because when fine-tuning them on image domains that were not learned in pretraining, such as floorplans, the quality of the generated images is poor and tuning takes a long time. This phenomenon arises from the so-called catastrophic forgetting problem, where traditional fine-tuning methods that update all parameters easily destroy the knowledge of pretrained diffusion models. To address this problem, we propose FloorDiffusion, a diffusion model-based conditional floorplan generation method. In this method, only a few key parameters of the pre-trained diffusion model are fine-tuned, which allows adaptation to the floorplan domain while retaining its useful knowledge. Then, the fine-tuned diffusion model performs conditional floorplan generation by inpainting the unfinished regions of the input conditional image. Comparative experiments with existing methods demonstrate that our method can produce more architecturally realistic floorplan images with up to 72 % image quality improvement. It can also generate various floorplan images for a single input condition image. Finally, ablation studies show that all components of the proposed method are essential for optimal operation.

Leveraging Conditional Generative Adversarial Networks (cGANs) for enhanced artistic creation: Exploring quality improvement and content control through conditional inputs

Conditional Generative Adversarial Networks (cGANs) have revolutionized digital art by enabling the creation of high-quality, artistically coherent images guided by conditional inputs. This paper examines key factors influencing the performance of cGANs, including the quality of training data, network architecture improvements, and loss function optimization. We introduce a mathematical model to quantify training data quality, emphasizing dataset diversity, data augmentation, and cleaning. Network architectural enhancements such as residual connections, attention mechanisms, and progressive growing are explored for their impact on image quality. Additionally, we discuss the integration of conditional inputs, such as labels and textual descriptions, for precise content control. Challenges in balancing realism with artistic expression, managing mode collapse, and interpreting conditional inputs are also addressed. This study provides a comprehensive framework for enhancing cGAN-generated artworks, offering insights into applications in personalized art generation, art restoration, and collaborative art projects.

Circular synthetic aperture radar sub‐aperture angle information complementation based on azimuth‐controllable generative adversarial network

AbstractA conditional generative adversarial network (CGAN) framework is proposed to address the issue of incomplete circular synthetic aperture radar (CSAR) azimuthal information due to motion errors. Specifically, the authors propose a novel CGAN architecture that can control the azimuth angle for arbitrary angle generation, capable of complementing missing CSAR sub‐aperture information. The network incorporates angular labels for various scenarios and integrates a dynamic region‐aware convolution (DRconv) module. Additionally, to counteract the common challenge of mode collapse in GAN training, a mode seeking regularisation technique is innovativrly introduced into the authors’ loss function. The efficacy of the proposed network is rigorously tested using both the MSTAR dataset and an X‐band SAR dataset. The results demonstrate that the authors’ network can generate high‐fidelity SAR images with controllable azimuths, closely resembling authentic images. Furthermore, the proposed method excels in complementing missing CSAR sub‐aperture information, effectively supplying the lost angular information due to motion errors. A new technical approach for SAR image generation is not only offered but it also has the potential to significantly expand SAR datasets. This advancement is expected to enhance the quality and utility of SAR imagery in applications such as surveillance, reconnaissance, and environmental monitoring.

Green finance growth prediction model based on time-series conditional generative adversarial networks.

Climate change mitigation necessitates increased investment in green sectors. This study proposes a methodology to predict green finance growth across various countries, aiming to encourage such investments. Our approach leverages time-series Conditional Generative Adversarial Networks (CT-GANs) for data augmentation and Nonlinear Autoregressive Neural Networks (NARNNs) for prediction. The green finance growth predicting model was applied to datasets collected from forty countries across five continents. The Augmented Dickey-Fuller (ADF) test confirmed the non-stationary nature of the data, supporting the use of Nonlinear Autoregressive Neural Networks (NARNNs). CT-GANs were then employed to augment the data for improved prediction accuracy. Results demonstrate the effectiveness of the proposed model. NARNNs trained with CT-GAN augmented data achieved superior performance across all regions, with R-squared (R2) values of 98.8%, 96.6%, and 99% for Europe, Asia, and other countries respectively. While the RMSE for Europe, Asia, and other countries are 1.26e+2, 2.16e+2, and 1.16e+2 respectively. Compared to a baseline NARNN model without augmentation, CT-GAN augmentation significantly improved both R2 and RMSE. The R2 values for the Europe, Asia, and other countries models are 96%, 73%, and 97.2%, respectively. The RMSE values for the Europe, Asia, and various countries models are 2.24e+2, 7e+2, and 2.07e+2, respectively. The Nonlinear Autoregressive Exogenous Neural Network (NARX-NN) exhibited significantly lower performance across Europe, Asia, and other countries with R2 values of 74%, 52%, and 86%, and RMSE values of 1.11e+2, 3.63e+2, and 1.8e+2, respectively.

A new deep learning-based approach for concrete crack identification and damage assessment

Concrete building structures are prone to cracking as they are subjected to environmental temperatures, freeze-thaw cycles, and other operational environmental factors. Failure to detect cracks in the key building structure at the early stage can result in serious accidents and associated economic losses. A new method using the SE-U-Net model based on a conditional generative adversarial network (CGAN) has been developed to identify small cracks in concrete structures in this paper. This proposed method was a pixel-level U-Net model based on a generative network, that was integrated the original convolutional layer with an attention mechanism, and an SE module in the jump connection section was added to improve the identifiability of the model. The discriminative network compared the generated images with real images using the PatchGAN model. Through the adversarial training of generator and discriminator, the performance of generator in crack image segmentation task is improved, and the trained generation network is used to segment cracks. In damage assessments, the crack skeleton was represented by the individual pixel width and recognized using the binary morphological crack skeleton method, in which the final length, area, and average width of the crack could be determined through the geometric correction index. The results showed that compared with other methods, the proposed method could better identify subtle pixel-level cracks, and the identification accuracy is 98.48%. These methods are of great significance for the identification of cracks and the damage assessment of concrete structures in practice.

Novel hybrid integrated Pix2Pix and WGAN model with Gradient Penalty for binary images denoising

This paper introduces a novel approach to image denoising that leverages the advantages of Generative Adversarial Networks (GANs). Specifically, we propose a model that combines elements of the Pix2Pix model and the Wasserstein GAN (WGAN) with Gradient Penalty (WGAN-GP). This hybrid framework seeks to capitalize on the denoising capabilities of conditional GANs, as demonstrated in the Pix2Pix model, while mitigating the need for an exhaustive search for optimal hyperparameters that could potentially ruin the stability of the learning process. In the proposed method, the GAN’s generator is employed to produce denoised images, harnessing the power of a conditional GAN for noise reduction. Simultaneously, the implementation of the Lipschitz continuity constraint during updates, as featured in WGAN-GP, aids in reducing susceptibility to mode collapse. This innovative design allows the proposed model to benefit from the strong points of both Pix2Pix and WGAN-GP, generating superior denoising results while ensuring training stability. Drawing on previous work on image-to-image translation and GAN stabilization techniques, the proposed research highlights the potential of GANs as a general-purpose solution for denoising. The paper details the development and testing of this model, showcasing its effectiveness through numerical experiments. The dataset was created by adding synthetic noise to clean images. Numerical results based on real-world dataset validation underscore the efficacy of this approach in image-denoising tasks, exhibiting significant enhancements over traditional techniques. Notably, the proposed model demonstrates strong generalization capabilities, performing effectively even when trained with synthetic noise.

Predicting multiphase flow behavior of methane in shallow unconfined aquifers using conditional deep convolutional generative adversarial network

Numerical modeling is an essential tool for geoscience applications involving multiphase flow behavior. Performing numerical simulation is, however, computationally intensive due to multi-scale, non-linear, and multi-physics nature of mechanisms controlling multiphase flow dynamics. Additionally, the computational challenges associated with traditional numerical simulations limit their applicability for repetitive simulations required in inverse modeling, uncertainty analysis, and optimization tasks. To overcome these limitations, surrogate modeling has recently emerged as a viable solution. A surrogate model, which functions as a regression model, is trained using numerical simulation data. It approximates the input–output relationships within a dynamic fluid environment. We design surrogate models, based on conditional deep convolutional generative adversarial network (cDC-GAN), to map the cross-domain between input and output pairs in a multiphase multicomponent system, focusing on methane (CH4) migration in shallow unconfined aquifers. The cDC-GAN technique is selected due to its ability to generate high-fidelity surrogate models that effectively capture complex spatial distributions and heterogeneities. This technique excels in handling high-dimensional data, learning the intricate relationships between inputs and outputs, and producing realistic representations of subsurface phenomena. The trained cDC-GANs consider capillary entry pressure heterogeneity as input because this geologic attribute places first-order control on CH4 migration in subsurface. The outputs include CH4 saturation, water saturation, residual saturation of CH4, CH4 mole fraction, and CH4 molality. For each output, a separate cDC-GAN model is trained. Once trained, cDC-GANs can at any given time determine CH4 distribution in a shallow unconfined aquifer, both qualitatively and quantitatively. The cDC-GAN approach can be considered as a valuable complement to numerical simulations for predicting multiphase flow behavior in other geoscience-, energy-, and environmental applications such as contaminant transport, hydrocarbon production, and geological carbon dioxide and hydrogen storage.

Quick Prediction of Complex Temperature Fields Using Conditional Generative Adversarial Networks

Abstract Qualified thermal management is an important guarantee for the stable work of electronic devices. However, the increasingly complex cooling structure needs several hours or even longer to simulate, which hinders finding the optimal heat dissipation design in the limited space. Herein, an approach based on conditional generative adversarial network (cGAN) is reported to bridge complex geometry and physical field. The established end-to-end model not only predicted the maximum temperature with high precision but also captured real field details in the generated image. The impact of amount of training data on model prediction performance was discussed, and the performance of the models fine-tuned and trained from scratch was also compared in the case of less training data or using in new electronic devices. Furthermore, the high expansibility of geometrically encoded labels makes this method possible to be used in the heat dissipation analysis of more electronic devices. More importantly, this approach, compared to the grid-based simulation, accelerates the process by several orders of magnitude and saves a large amount of energy, which can vastly improve the efficiency of the thermal management design of electronic devices.

CGAN-based cyber deception framework against reconnaissance attacks in ICS

In recent years, Industrial Control Systems (ICSs) have faced increasing vulnerability to cyber attacks due to their integration with the Internet. Despite efforts to enhance cybersecurity, reconnaissance attacks remain a significant threat, prompting the need for innovative defensive strategies. This paper introduces a novel approach to strengthen the defensive capabilities of ICS networks against reconnaissance attacks using machine learning-driven cyber deception techniques. Leveraging Conditional Generative Adversarial Networks (CGANs), the proposed framework dynamically generates defensive network topologies to network shuffling and implement deception strategies, prioritizing system availability. Extensive simulations demonstrate the superior efficacy of the proposed framework in enhancing cybersecurity while minimizing computational overhead. By effectively mitigating reconnaissance attacks, this solution reinforces the resilience of ICS networks, safeguarding critical industrial infrastructure from evolving cyber threats. These findings underscore the significance of adopting machine learning-based cyber deception as a pragmatic security measure for protecting ICS networks in real-world industrial contexts.

Enhancing network intrusion detection performance using generative adversarial networks

Network intrusion detection systems (NIDS) play a pivotal role in safeguarding critical digital infrastructures against cyber threats. Machine learning-based detection models applied in NIDS are prevalent today. However, the effectiveness of these machine learning-based models is often limited by the evolving and sophisticated nature of intrusion techniques as well as the lack of diverse and updated training samples. In this research, a novel approach for enhancing the performance of an NIDS through the integration of Generative Adversarial Networks (GANs) is proposed. By harnessing the power of GANs in generating synthetic network traffic data that closely mimics real-world network behavior, we address a key challenge associated with NIDS training datasets, which is the data scarcity. Three distinct GAN models (Vanilla GAN, Wasserstein GAN and Conditional Tabular GAN) are implemented in this work to generate authentic network traffic patterns specifically tailored to represent the anomalous activity. We demonstrate how this synthetic data resampling technique can significantly improve the performance of the NIDS model for detecting such activity. By conducting comprehensive experiments using the CIC-IDS2017 benchmark dataset, augmented with GAN-generated data, we offer empirical evidence that shows the effectiveness of our proposed approach. Our findings show that the integration of GANs into NIDS can lead to enhancements in intrusion detection performance for attacks with limited training data, making it a promising avenue for bolstering the cybersecurity posture of organizations in an increasingly interconnected and vulnerable digital landscape.

Artificial intelligence-based joint attenuation and scatter correction strategies for multi-tracer total-body PET

BackgroundLow-dose ungated CT is commonly used for total-body PET attenuation and scatter correction (ASC). However, CT-based ASC (CT-ASC) is limited by radiation dose risks of CT examinations, propagation of CT-based artifacts and potential mismatches between PET and CT. We demonstrate the feasibility of direct ASC for multi-tracer total-body PET in the image domain.MethodsClinical uEXPLORER total-body PET/CT datasets of [18F]FDG (N = 52), [18F]FAPI (N = 46) and [68Ga]FAPI (N = 60) were retrospectively enrolled in this study. We developed an improved 3D conditional generative adversarial network (cGAN) to directly estimate attenuation and scatter-corrected PET images from non-attenuation and scatter-corrected (NASC) PET images. The feasibility of the proposed 3D cGAN-based ASC was validated using four training strategies: (1) Paired 3D NASC and CT-ASC PET images from three tracers were pooled into one centralized server (CZ-ASC). (2) Paired 3D NASC and CT-ASC PET images from each tracer were individually used (DL-ASC). (3) Paired NASC and CT-ASC PET images from one tracer ([18F]FDG) were used to train the networks, while the other two tracers were used for testing without fine-tuning (NFT-ASC). (4) The pre-trained networks of (3) were fine-tuned with two other tracers individually (FT-ASC). We trained all networks in fivefold cross-validation. The performance of all ASC methods was evaluated by qualitative and quantitative metrics using CT-ASC as the reference.ResultsCZ-ASC, DL-ASC and FT-ASC showed comparable visual quality with CT-ASC for all tracers. CZ-ASC and DL-ASC resulted in a normalized mean absolute error (NMAE) of 8.51 ± 7.32% versus 7.36 ± 6.77% (p < 0.05), outperforming NASC (p < 0.0001) in [18F]FDG dataset. CZ-ASC, FT-ASC and DL-ASC led to NMAE of 6.44 ± 7.02%, 6.55 ± 5.89%, and 7.25 ± 6.33% in [18F]FAPI dataset, and NMAE of 5.53 ± 3.99%, 5.60 ± 4.02%, and 5.68 ± 4.12% in [68Ga]FAPI dataset, respectively. CZ-ASC, FT-ASC and DL-ASC were superior to NASC (p < 0.0001) and NFT-ASC (p < 0.0001) in terms of NMAE results.ConclusionsCZ-ASC, DL-ASC and FT-ASC demonstrated the feasibility of providing accurate and robust ASC for multi-tracer total-body PET, thereby reducing the radiation hazards to patients from redundant CT examinations. CZ-ASC and FT-ASC could outperform DL-ASC for cross-tracer total-body PET AC.

Adversarial image-to-image model to obtain highly detailed wind fields from mesoscale simulations in mountainous and urban areas

The characterisation of wind is of great interest in multiple disciplines such as city planning, pedestrian comfort and energy generation. We propose a conditional Generative Adversarial Network (cGAN), based on the Pix2Pix model [1], that can generate detailed local wind fields in areas with complex orography or an urban layout, which are comparable in level of detail to those from Computational Fluid Dynamics (CFD) simulations, from coarser Numerical Weather Prediction (NWP) data.

Research on identification of flight cadets’ cognitive load based on multi-source physiological data and CGAN-DBN model

Modern aircraft cockpit system is highly information-intensive. Pilots often need to receive a large amount of information and make correct judgments and decisions in a short time. However, cognitive load can affect their ability to perceive, judge and make decisions accurately. Furthermore, the excessive cognitive load will induce incorrect operations and even lead to flight accidents. Accordingly, the research on cognitive load is crucial to reduce errors and even accidents caused by human factors. By using physiological acquisition systems such as eye movement, ECG, and respiration, multi-source physiological signals of flight cadets performing different flight tasks during the flight simulation experiment are obtained. Based on the characteristic indexes extracted from multi-source physiological data, the CGAN-DBN model is established by combining the conditional generative adversarial networks (CGAN) model with the deep belief network (DBN) model to identify the flight cadets’ cognitive load. The research results show that the flight cadets’ cognitive load identification based on the CGAN-DBN model established has high accuracy. And it can effectively identify the cognitive load of flight cadets. The research paper has important practical significance to reduce the flight accidents caused by the high cognitive load of pilots.

Super-Resolution Image Reconstruction of Wavefront Coding Imaging System Based on Deep Learning Network

Wavefront Coding (WFC) is an innovative technique aimed at extending the depth of focus (DOF) of optics imaging systems. In digital imaging systems, super-resolution digital reconstruction close to the diffraction limit of optical systems has always been a hot research topic. With the design of a point spread function (PSF) generated by a suitably phase mask, WFC could also be used in super-resolution image reconstruction. In this paper, we use a deep learning network combined with WFC as a general framework for images reconstruction, and verify its possibility and effectiveness. Considering the blur and additive noise simultaneously, we proposed three super-resolution image reconstruction procedures utilizing convolutional neural networks (CNN) based on mean square error (MSE) loss, conditional Generative Adversarial Networks (CGAN), and Swin Transformer Networks (SwinIR) based on mean absolute error (MAE) loss. We verified their effectiveness by simulation experiments. A comparison of experimental results shows that the SwinIR deep residual network structure based on MAE loss optimization criteria can generate more realistic super-resolution images with more details. In addition, we used a WFC camera to obtain a resolution test target and real scene images for experiments. Using the resolution test target, we demonstrated that the spatial resolution could be improved from 55.6 lp/mm to 124 lp/mm by the proposed super-resolution reconstruction procedure. The reconstruction results show that the proposed deep learning network model is superior to the traditional method in reconstructing high-frequency details and effectively suppressing noise, with the resolution approaching the diffraction limit.

Stochastic optimization for capacity configuration of data center microgrid thermal energy management equipment considering flexible resources

The rapid development of Internet and cloud computing technologies has led to the expansion of the scale of data centers, resulting in a continuous increase in the demand for data processing and storage. This not only results in higher energy consumption but also makes thermal management a key factor of energy management in data centers. A capacity configuration method is proposed for the core thermal management equipment of data center microgrids, electric boilers, cooling systems, and heat pumps. This method not only considers the uncertainty factors of the source and load but also effectively utilizes the characteristics of various flexible resources in data centers, such as the adjustable ability of different types of batch processing loads, air thermal inertia, and waste heat recovery, significantly improving the energy utilization efficiency and enhancing the system’s economy and flexibility. In addition, a wind power scenario generation method based on conditional least squares generative adversarial networks was proposed to improve the generation quality of wind power scenarios, and a random optimization model was constructed. Considering a certain province’s data center microgrid as a case study, the effectiveness of the model was verified, and a sensitivity analysis of the relevant parameters provided important references for microgrid planning. This study provides a new perspective and tool for the thermal management of data center microgrids and comprehensive utilization of renewable energy, which is of great significance for promoting the improvement of data center energy efficiency and sustainable development.