Generative Adversarial Networks Training Research Articles

A practical Approach to Handwritten Digit Generation with Generative Adversarial Networks on MNIST

Implementation of a Generative Adversarial Network (GAN) utilizing the MNIST (Modified National Institute of Standards and Technology database) dataset of handwritten digits, the GAN comprises a generator creating synthetic images from random noise and a discriminator to classifying these images as real or fake. The Generator utilizes leaky ReLU (rectified linear unit) activations, Batch normalization and reshaping to produce 28x28 grayscale images, while the Discriminator uses dense layers, leaky ReLU, and dropout to enhance classification accuracy. Both networks are trained using the Adam optimizer to improve stability and performance. The GAN is trained in an adversarial setup where the Generator seeks to create convincing images and the Discriminator aims to correctly classify them. Results demonstrate the GAN's ability to generate high-quality handwritten digit images, displaying its effectiveness and providing valuable insights into best practices for GAN implementation and training.

Partial Discharge Data Augmentation and Pattern Recognition Method Based on DAE-GAN

Accurate identification of partial discharge (PD) and its types is essential for assessing the operating conditions of electrical equipment. To enhance PD pattern recognition under imbalanced and limited sample conditions, a method based on a Deep Autoencoder-embedded Generative Adversarial Network (DAE-GAN) is proposed. First, the Deep Autoencoder (DAE) is embedded within the Generative Adversarial Network (GAN) to improve the realism of generated samples. Then, complementary PD data samples are introduced during GAN training to address the issue of limited sample size. Lastly, the model’s discriminator is fine-tuned with augmented and balanced training data to enable PD pattern recognition. The DAE-GAN method is used to augment data and recognize patterns in experimental PD signals. The results demonstrate that, under imbalanced and small sample conditions, DAE-GAN generates more authentic PD samples with improved probability distribution fitting compared to other algorithms, leading to varying levels of enhancement in pattern recognition accuracy.

DMFGAN: a multifeature data augmentation method for grape leaf disease identification.

The use of deep learning techniques to identify grape leaf diseases relies on large, high-quality datasets. However, a large number of images occupy more computing resources and are prone to pattern collapse during training. In this paper, a depth-separable multifeature generative adversarial network (DMFGAN) was proposed to enhance grape leaf disease data. First, a multifeature extraction block (MFEB) based on the four-channel feature fusion strategy is designed to improve the quality of the generated image and avoid the problem of poor feature learning ability of the adversarial generation network caused by the single-channel feature extraction method. Second, a depth-based D-discriminator is designed to improve the discriminator capability and reduce the number of model parameters. Third, SeLU activation function was substituted for DCGAN activation function to overcome the problem that DCGAN activation function was not enough to fit grape leaf disease image data. Finally, an MFLoss function with a gradient penalty term is proposed to reduce the mode collapse during the training of generative adversarial networks. By comparing the visual indicators and evaluation indicators of the images generated by different models, and using the recognition network to verify the enhanced grape disease data, the results show that the method is effective in enhancing grape leaf disease data. Under the same experimental conditions, DMFGAN generates higher quality and more diverse images with fewer parameters than other generative adversarial networks. The mode breakdown times of generative adversarial networks in training process are reduced, which is more effective in practical application.

Spectrum-guided GAN with density-directionality sampling: Diverse high-fidelity signal generation for fault diagnosis of rotating machinery

In the field of fault diagnosis for rotating machinery, where available fault data are limited, numerous studies have employed a generative adversarial network (GAN) for data generation. However, the limited fault data for training GAN exacerbate GAN’s inherent training instability and mode collapse issues, which are induced by adversarial training. Moreover, the stochastic nature of random sampling for latent vectors sampling often results in low-fidelity and poor diversity generation, which negatively affects the fault diagnosis models. To address these issues, this paper presents two novel approaches: a spectrum-guided GAN (SGAN) and density-directionality sampling (DDS). SGAN mitigates training instability and mode collapse through combinatorial data utilization, adversarial spectral loss, and a tailored model structure. DDS ensures the high-fidelity and high-diversity of the generated data by selectively sampling the latent vectors through two steps: density-based filtering and directionality-based sampling in the feature space. Validation on both rotor and rolling element bearing datasets demonstrates that SGAN-DDS considerably improves classification results under the limited fault data. Furthermore, fidelity and diversity analyses are conducted to validate DDS, which increase the credibility of the proposed method; and offer advancement toward the application of deep-learning and GAN in industrial fields.

Closing the Gap Between Theory and Practice During Alternating Optimization for GANs.

Synthesizing high-quality and diverse samples is the main goal of generative models. Despite recent great progress in generative adversarial networks (GANs), mode collapse is still an open problem, and mitigating it will benefit the generator to better capture the target data distribution. This article rethinks alternating optimization in GANs, which is a classic approach to training GANs in practice. We find that the theory presented in the original GANs does not accommodate this practical solution. Under the alternating optimization manner, the vanilla loss function provides an inappropriate objective for the generator. This objective forces the generator to produce the output with the highest discriminative probability of the discriminator, which leads to mode collapse in GANs. To address this problem, we introduce a novel loss function for the generator to adapt to the alternating optimization nature. When updating the generator by the proposed loss function, the reverse Kullback-Leibler divergence between the model distribution and the target distribution is theoretically optimized, which encourages the model to learn the target distribution. The results of extensive experiments demonstrate that our approach can consistently boost model performance on various datasets and network structures.

Three-stage binarization of color document images based on discrete wavelet transform and generative adversarial networks

The efficient extraction of text information from the background in degraded color document images is an important challenge in the preservation of ancient manuscripts. The imperfect preservation of ancient manuscripts has led to different types of degradation over time, such as page yellowing, staining, and ink bleeding, seriously affecting the results of document image binarization. This work proposes an effective three-stage network method to image enhancement and binarization of degraded documents using generative adversarial networks (GANs). Specifically, in Stage-1, we first split the input images into multiple patches, and then split these patches into four single-channel patch images (gray, red, green, and blue). Then, three single-channel patch images (red, green, and blue) are processed by the discrete wavelet transform (DWT) with normalization. In Stage-2, we use four independent generators to separately train GAN models based on the four channels on the processed patch images to extract color foreground information. Finally, in Stage-3, we train two independent GAN models on the outputs of Stage-2 and the resized original input images (512 × 512) as the local and global predictions to obtain the final outputs. The experimental results show that the Avg-Score metrics of the proposed method are 77.64, 77.95, 79.05, 76.38, 75.34, and 77.00 on the (H)-DIBCO 2011, 2013, 2014, 2016, 2017, and 2018 datasets, which are at the state-of-the-art level. The implementation code for this work is available at https://github.com/abcpp12383/ThreeStageBinarization.

Data Augmentation in Histopathological Classification: An Analysis Exploring GANs with XAI and Vision Transformers

Generative adversarial networks (GANs) create images by pitting a generator (G) against a discriminator (D) network, aiming to find a balance between the networks. However, achieving this balance is difficult because G is trained based on just one value representing D’s prediction, and only D can access image features. We introduce a novel approach for training GANs using explainable artificial intelligence (XAI) to enhance the quality and diversity of generated images in histopathological datasets. We leverage XAI to extract feature information from D and incorporate it into G via the loss function, a unique strategy not previously explored in this context. We demonstrate that this approach enriches the training with relevant information and promotes improved quality and more variability in the artificial images, decreasing the FID by up to 32.7% compared to traditional methods. In the data augmentation task, these images improve the classification accuracy of Transformer models by up to 3.81% compared to models without data augmentation and up to 3.01% compared to traditional GAN data augmentation. The Saliency method provides G with the most informative feature information. Overall, our work highlights the potential of XAI for enhancing GAN training and suggests avenues for further exploration in this field.

EcoFlow: Efficient Convolutional Dataflows on Low-Power Neural Network Accelerators

Dilated and transposed convolutions are widely used in modern convolutional neural networks (CNNs). These kernels are used extensively during CNN training and inference of applications such as image segmentation and high-resolution image generation. We find that commonly-used low-power CNN inference accelerators are <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">not</i> optimized for both these convolutional kernels. Dilated and transposed convolutions introduce significant zero padding when mapped to the underlying spatial architecture, significantly degrading performance and energy efficiency. Existing approaches that address this issue require significant design changes to the otherwise simple, efficient, and well-adopted architectures used to compute direct convolutions. To address this challenge, we propose EcoFlow, a new set of dataflows and mapping algorithms for dilated and transposed convolutions. These algorithms are tailored to execute efficiently on existing low-cost, small-scale spatial architectures and requires minimal changes to existing accelerators. At its core, EcoFlow eliminates zero padding through careful dataflow orchestration and data mapping tailored to the spatial architecture. We evaluate EcoFlow on CNN training workloads and Generative Adversarial Network (GAN) workloads. Experiments in our new cycle-accurate simulator show that, using a common CNN inference accelerator, EcoFlow 1) reduces end-to-end CNN training time between 7-85%, and 2) improves end-to-end GAN training performance between 29-42%, compared to state-of-the-art CNN dataflows.

Integrating terrain structure characteristics into generative adversarial nets for hillshade generation

A hillshade is a visualization technique that represents three-dimensional terrain in a two-dimensional plane by illumination mapping. The digital relief shading promotes the visualization of the terrain efficiently using DEM data. However, compared with manual shading, the digital algorithm-based method still has a gap in the visual effect of illumination strategy and terrain generalization. The typical landform characteristics and micro-geomorphic properties usually are destroyed. The reason is that the complexity of illumination rules is hard to summarize for different terrain phenomena. In this study, namely the data-driven artificial intelligent method, an alternative strategy based on generative adversarial networks (GANs) is proposed rather than the rule-based method. The DEM and the terrain skeleton lines are input to the model and part of the manual relief shading of Swisstopo is used for the training samples. Through the GAN training and learning, the manual skill imbedded in the hillshade is discovered in the generation model. The results show that the proposed model performs better than the digital relief shading on various landforms in aesthetic visualization and geo-scientific representation. Compared to other models, including other convolution neural network (CNN) based methods, terrain structure is maintained more significantly through the proposed model.

Geo-Hgan: Unsupervised anomaly detection in geochemical data via latent space learning

Reconstructing geochemical data for anomaly detection using Generative Adversarial Networks (GANs) has become a prevalent method in identifying geochemical anomalies. However, injecting random noise into GANs can induce model instability. To mitigate this issue, we propose a novel anomaly detection model, Geo-Hgan, which integrates a dual adversarial network architecture with a Latent Space Adversarial Module (LSAM) to learn the distribution of latent variables from arbitrary data and optimize the sample reconstruction process, thereby alleviating instability during GAN training. Additionally, an encoder guided by the LSAM-pretrained GAN is employed to extract variational features, facilitating rapid and effective sample mapping into the latent space defined by LSAM. Experimental results demonstrate that under unsupervised conditions, Geo-Hgan achieves an Area Under the Curve (AUC) score of 85% across three geochemical datasets, outperforming similar models in accuracy and reconstruction capabilities. To assess its versatility and generalization ability, we extend Geo-Hgan to anomaly detection tasks in computer vision, where it achieves an average AUC score of 98.7% on the MvtecAD dataset, setting a new state-of-the-art performance in the domain. Furthermore, we propose AnomFilter, a method for setting anomaly thresholds based on the clustering hypothesis. AnomFilter identifies high-confidence anomaly samples identified by Geo-Hgan in the source domain and iteratively transfers them to the target domain. These high-confidence anomaly samples, combined with a small number of known positive samples in the target domain, enhance the accuracy of supervised geochemical anomaly detection in the target domain, which achieved an AUC score of 94%. The utilization of anomaly detection models for sample transfer learning offers a novel perspective for future work.

Enhancing experimental image quality in two-phase bubbly systems with super-resolution using generative adversarial networks

Fluid Dynamics is a key scientific field to multitudes of engineering applications. Experimental work in this field requires careful set-up and expensive image-capturing equipment, particularly when considering the finer details of complex phenomena. In this work, we study the application of super-resolution Generative Adversarial Networks (GANs) to achieve high-resolution results by upscaling lower-resolution experimental images.We train GANs proposed for natural images on a bubbly flow experimental Fluid Dynamics dataset and compare common super-resolution evaluation metrics to domain expert assessments of the upscaled images. We find that these models achieve promising results, as evaluated by experts, and transfer learning from natural images translates to better performance overall. Attention mechanisms are found to be particularly useful in recreating sharper details. On the other hand, traditional super-resolution evaluation metrics are found to align poorly with expert perception of quality, signaling the need for better systematic evaluation methodologies in this domain.

Hybrid encryption based on a generative adversarial network

In today’s world, encryption is crucial for protecting sensitive data. Neural networks can provide security against adversarial attacks, but meticulous training and vulnerability analysis are required to ensure their effectiveness. Hence, this research explores hybrid encryption based on a generative adversarial network (GAN) for improved message encryption. A neural network was trained using the GAN method to defend against adversarial attacks. Various GAN training parameters were tested to identify the best model system, and various models were evaluated concerning their accuracy against different configurations. Neural network models were developed for Alice, Bob, and Eve using random datasets and encryption. The models were trained adversarially using the GAN to find optimal parameters, and their performance was analyzed by studying Bob’s and Eve’s accuracy and bits error. The parameters of 8,000 epochs, a batch size of 4,096, and a learning rate of 0.0008 resulted in 100% accuracy for Bob and 52.14% accuracy for Eve. This implies that Alice and Bob’s neural network effectively secured the messages from Eve’s neural network. The findings highlight the advantages of employing neural network-based encryption methods, providing valuable insights for advancing the field of secure communication and data protection.

Melanoma classification using generative adversarial network and proximal policy optimization.

In oncology, melanoma is a serious concern, often arising from DNA changes caused mainly by ultraviolet radiation. This cancer is known for its aggressive growth, highlighting the necessity of early detection. Our research introduces a novel deep learning framework for melanoma classification, trained and validated using the extensive SIIM-ISIC Melanoma Classification Challenge-ISIC-2020 dataset. The framework features three dilated convolution layers that extract critical feature vectors for classification. A key aspect of our model is incorporating the Off-policy Proximal Policy Optimization (Off-policy PPO) algorithm, which effectively handles data imbalance in the training set by rewarding the accurate classification of underrepresented samples. In this framework, the model is visualized as an agent making a series of decisions, where each sample represents a distinct state. Additionally, a Generative Adversarial Network (GAN) augments training data to improve generalizability, paired with a new regularization technique to stabilize GAN training and prevent mode collapse. The model achieved an F-measure of 91.836% and a geometric mean of 91.920%, surpassing existing models and demonstrating the model's practical utility in clinical environments. These results demonstrate its potential in enhancing early melanoma detection and informing more accurate treatment approaches, significantly advancing in combating this aggressive cancer.

Enhancing capabilities of generative models through VAE-GAN integration: A review

Our review explores the integration of Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), which are pivotal in the realm of generative models. VAEs are renowned for their robust probabilistic foundations and capacity for complex data representation learning, while GANs are celebrated for generating high-fidelity images. Despite their strengths, both models have limitations: VAEs often produce less sharp outputs, and GANs face challenges with training stability. The hybrid VAE-GAN models harness the strengths of both architectures to overcome these limitations, enhancing output quality and diversity. We provide a comprehensive overview of VAEs and GANs technology developments, their integration strategies, and resultant performance improvements. Applications across various fields, such as artistic creation, medical imaging, e-commerce, and video gaming, highlight the transformative potential of these models. However, challenges in model robustness, ethical concerns, and computational demands persist, posing significant hurdles. Future research directions are poised to transform the VAE-GAN landscape significantly. Enhancing training stability remains a priority, with new approaches such as incorporating self-correcting mechanisms into GANs training being tested. Addressing ethical issues is also critical, as policymakers and technologists work together to develop standards that prevent misuse. Moreover, reducing computational costs is fundamental to democratizing access to these technologies. Projects such as the development of MobileNetV2 have made strides in creating more efficient neural network architectures that maintain performance while being less resource-intensive. Further, the exploration of VAE-GAN applications in fields like augmented reality and personalized medicine offers exciting opportunities for growth, as evidenced by recent pilot studies.

DM-GAN: CNN hybrid vits for training GANs under limited data

Generative adversarial network (GAN) training demands substantial data and computational resources. This paper aims to explore an economical approach for generating novel images with limited image data, addressing the challenge of data scarcity. Our contributions involve resolving the few-shot image generation challenge through the development of an unsupervised hybrid generative adversarial network named DM-GAN. We introduce a lightweight hybrid module (DC-Vit) comprising convolution and visual transformation, merging local and global features to enhance image perception, expressiveness, and ensure stable image generation. Additionally, a multi-scale adaptive skip connection module is incorporated to effectively mitigate the feature loss problem arising from inter-layer jumps, thereby producing more complete and regular images. To enhance the texture learning process and improve the quality and realism of synthesized images, we integrate the gray conjugate matrix into the loss function. Empirical evaluations are conducted on small sample datasets at various resolutions, including publicly accessible collections of art paintings, real-life photographs, and proprietary artifact image datasets. The experimental results unequivocally demonstrate the qualitative and quantitative superiority of our model over existing methods, underscoring its efficacy and robustness.

Parallel diffusion models promote high detail-fidelity photoacoustic microscopy in sparse sampling

Reconstructing sparsely sampled data is fundamental for achieving high spatiotemporal resolution photoacoustic microscopy (PAM) of microvascular morphology in vivo. Convolutional networks (CNN) and generative adversarial networks (GAN) have been introduced to high-speed PAM, but due to the use of upsampling in CNN-based networks to restore details and the instability in GAN training, they struggle to learn the entangled microvascular network structure and vascular texture features, resulting in only achieving low detail-fidelity imaging of microvascular. The diffusion models is richly sampled and can generate high-quality images, which is very helpful for the complex vascular features in PAM. Here, we propose an approach named parallel diffusion models (PDM) with parallel learning of Noise task and Image task, where the Noise task optimizes through variational lower bounds to generate microvascular structures that are visually realistic, and the Image task improves the fidelity of the generated microvascular details through image-based loss. With only 1.56% of fully sampled pixels from photoacoustic human oral data, PDM achieves an LPIPS of 0.199. Additionally, using PDM in high-speed 16x PAM prevents breathing artifacts and image distortion issues caused by low-speed sampling, reduces the standard deviation of the Row-wise Self-Correlation Coefficient, and maintains high image quality. It achieves high confidence in reconstructing detailed information from sparsely sampled data and will promote the application of reconstructed sparsely sampled data in realizing high spatiotemporal resolution PAM.

Multi deep learning-based stochastic microstructure reconstruction and high-fidelity micromechanics simulation of time-dependent ceramic matrix composite response

A multi deep learning-based framework is developed for efficient, automated microstructure reconstruction and generation of stochastic representative volume elements (SRVEs) with periodic boundary conditions (PBCs) for accurate modeling of ceramic matrix composite (CMC) response. The methodology comprises a convolutional neural network coupled with regression layers to act as a vanilla regression network for semantic segmentation of the microstructure, allowing accurate characterization of the phases and their distributions at the microscale. Scanning electron microscope and confocal microscope are used to obtain C/SiNC and SiC/SiNC CMCs micrographs for vanilla regression testing. Microstructure variability in terms of fiber volume fraction and porosity are quantified through the output regression layer, ensuring accurate representation of material variability in SRVE construction. Generative adversarial network (GAN) and its variants are designed to produce high-fidelity SRVE, spanning CMCs microstructure variability space. A circular padding algorithm is developed to generate SRVEs with PBCs during training of GANs. The accuracy of the generated SRVEs is established through micromechanics simulations, where an efficient formulation of the high-fidelity generalized methods of cells (HFGMC) approach is used to compute the effective mechanical properties. An iterative algorithm is implemented in the HFGMC solver to simulate time-dependent deformation of SiC/SiNC subjected to creep loading conditions.

Melanoma image synthesis: a review using generative adversarial networks

Melanoma is a highly malignant skin cancer that may be fatal if not promptly detected and treated. The limited availability of high-quality melanoma images, which are needed for training machine learning models, is one of the obstacles to detecting melanoma. Generative adversarial networks (GANs) have grown in popularity as a strong technique for image synthesis. This research is also targeted at the sustainable development goal (SDG) for health care. In this study, we survey existing GAN-based melanoma image synthesis methods. In this work, we briefly introduce GANs and how they may be used for generating synthetic images. Ensuring healthy lifestyles and promoting well-being for everyone, regardless of age, is the main aim. A comparative study is carried out on how GANs are used in current research to generate melanoma images and how they improve the classification performance of neural networks. Various public and proprietary datasets for training GANs in melanoma image synthesis are also discussed. Lastly, we assess the examined studies' performance using measures like the Frechet Inception distance (FID), Inception score, structural similarity ındex (SSIM), and various classification performance metrics. We compare the evaluated findings and suggest further GAN-based melanoma image-creation research.

Image synthesis of interictal SPECT from MRI and PET using machine learning.

Cross-modality image estimation can be performed using generative adversarial networks (GANs). To date, SPECT image estimation from another medical imaging modality using this technique has not been considered. We evaluate the estimation of SPECT from MRI and PET, and additionally assess the necessity for cross-modality image registration for GAN training. We estimated interictal SPECT from PET and MRI as a single-channel input, and as a multi-channel input to the GAN. We collected data from 48 individuals with epilepsy and converted them to 3D isotropic images for consistence across the modalities. Training and testing data were prepared in native and template spaces. The Pix2pix framework within the GAN network was adopted. We evaluated the addition of the structural similarity index metric to the loss function in the GAN implementation. Root-mean-square error, structural similarity index, and peak signal-to-noise ratio were used to assess how well SPECT images were able to be synthesised. High quality SPECT images could be synthesised in each case. On average, the use of native space images resulted in a 5.4% percentage improvement in SSIM than the use of images registered to template space. The addition of structural similarity index metric to the GAN loss function did not result in improved synthetic SPECT images. Using PET in either the single channel or dual channel implementation led to the best results, however MRI could produce SPECT images close in quality. Synthesis of SPECT from MRI or PET can potentially reduce the number of scans needed for epilepsy patient evaluation and reduce patient exposure to radiation.

Stochastic Modeling of Thin Mud Drapes Inside Point Bar Reservoirs With ALLUVSIM‐GANSim

AbstractModeling inclined fine‐scale mud drapes inside point bars, deposited on accretion surfaces during stages of low energy or slack water, is critical to modeling fluid flow in complex sedimentary environments (e.g., fluvial and turbidity flows). These features have been modeled using deterministic or geostatistical modeling tools (e.g., object‐, event‐, and pixel‐based). However, this is a non‐trivial task due to the need to preserve geological realism (e.g., connectivity within sedimentary features and facies hierarchy), while being able to condition the generated models to point data (e.g., well data). Generative Adversarial Networks (GAN) have been successfully applied to reproduce several large‐scale scenarios (e.g., braided rivers and carbonate reservoirs), yet their potential for capturing small‐scale and hierarchical features remains largely unexplored. Here, we propose a geo‐modeling workflow for fast modeling of small‐scale conditional mud drapes based on ALLUVSIM and GANSim. Initially, improved ALLUVSIM produces realistic unconditional models of mud drapes along accretionary surfaces, serving as GAN training data. GANSim is then employed to achieve conditioning to well data and probability maps derived from geophysical modeling. Finally, temporal pressure data observed in wells are further conditioned via a Markov chain Monte Carlo sampling method. The proposed geo‐modeling workflow is validated in a two‐dimensional synthetic example as the pre‐trained generator extracts mud‐drapes‐features and generates multiple facies realizations conditioned to diverse information. A field application example in a modern meandering river verifies the effectiveness and practicability of the proposed workflow in real case application examples. The application examples illustrate the potential of the proposed method to predict mud drapes inside point bar reservoirs.