Generative Adversarial Networks Research Articles

ACGAN: adaptive conditional generative adversarial network architecture predicting skin lesion using collaboration of transfer learning models

Skin cancer has become a serious disease which has the potential to scale up if it is not identified earlier. It is imperative to detect and give treatment to skin cancer promptly. Diagnosing skin cancer manually takes a lot of time and it is costly, and the probability of false diagnosis has increased due to the outstanding resemblances among various skin lesions. Enhancing the classification of multi-class lesions of skin needs the development of investigative systems which should be automated. Data augmentation with GANs and Adaptive Conditional Generative Adversarial Network strategies improves performance. The performance is tested using balanced and unbalanced datasets. Using a proper process of augmentation of data, the suggested system attains a 94% accuracy for the VGG16, 93% for the ResNet50 and 94.25% for ResNet101. The process of collaboration of all such methods improves accuracy further to 95%. In summary, the novelty of the work lies in its holistic approach to automated skin lesion classification, incorporating advanced deep learning models, novel data augmentation techniques and comprehensive performance evaluation on real-world datasets. These contributions collectively advance the field of computer-aided diagnosis for the detection of skin cancer and treatment.

VFLGAN: Vertical Federated Learning-based Generative Adversarial Network for Vertically Partitioned Data Publication

In the current artificial intelligence (AI) era, the scale and quality of the dataset play a crucial role in training a high-quality AI model. However, good data is not a free lunch and is always hard to access due to privacy regulations like the General Data Protection Regulation (GDPR). A potential solution is to release a synthetic dataset with a similar distribution to that of the private dataset. Nevertheless, in some scenarios, it has been found that the attributes needed to train an AI model belong to different parties, and they cannot share the raw data for synthetic data publication due to privacy regulations. In PETS 2023, Xue et al. [29] proposed the first generative adversary network-based model, VertiGAN, for vertically partitioned data publication. However, after thoroughly investigating, we found that VertiGAN is less effective in preserving the correlation among the attributes of different parties. This article proposes a Vertical Federated Learning-based Generative Adversarial Network, VFLGAN, for vertically partitioned data publication to address the above issues. Our experimental results show that compared with VertiGAN, VFLGAN significantly improves the quality of synthetic data. Taking the MNIST dataset as an example, the quality of the synthetic dataset generated by VFLGAN is 3.2 times better than that generated by VertiGAN w.r.t. the Frechet Distance. We also designed a more efficient and effective Gaussian mechanism for the proposed VFLGAN to provide the synthetic dataset with a differential privacy guarantee. On the other hand, differential privacy only gives the upper bound of the worst-case privacy guarantee. This article also proposes a practical auditing scheme that applies membership inference attacks to estimate privacy leakage through the synthetic dataset.

Performance Improvement of Rice Leaf Disease Prediction Model using Enhanced Super-Resolution Generative Adversarial Networks

The objective of this paper is to propose a method to improve the performance of Rice Leaf Disease Prediction Deep Learning Model. The primary goal is to empower farmers with precise information about crop disease, enabling them to judiciously apply pesticides suitably to their crop's needs. The method proposed here uses image enhancement using Enhanced Super Resolution Generative Adversarial Network. Experiment was carried out using Rice Leaf Disease dataset which includes 16,000 rice leaf images categorized into four groups. Among these groups, three contain 4,000 images each, showcasing three different diseases and another 4000 images of healthy leaves. The best accuracy of the model came out to be 99.68. Also all individual diseases are identified very well and other model evaluation aspects also giving good results. Average precision obtained is 99.67, average recall value=99.68 and average f1-score came out to be 99.67. Conclusion: The performance results obtained is the highest what has been achieved of all the models that have been developed till now for this task.

Creating realistic anterior segment optical coherence tomography images using generative adversarial networks

AimsTo develop a generative adversarial network (GAN) capable of generating realistic high-resolution anterior segment optical coherence tomography (AS-OCT) images.MethodsThis study included 142 628 AS-OCT B-scans from the American University of...

Synthesis of Comic-Style Portraits Using Combination of CycleGAN and Pix2Pix

This paper proposes a system for converting face photos into portraits with specific charcoal sketch-like style. NPR (non-photorealistic rendering) for converting real photo images into anime-style images have been studied. Its promising application is the creation of users’ portraits on social networks for preventing leaks of personal information. An image-to-image transformation using GAN (Generative Adversarial Network), such as CycleGAN or Pix2Pix, is expected to be used for this goal. However, it is difficult to generate portraits with specific comic styles that satisfy two conditions at the same time: preserving photorealistic features and reproducing the comic styles. For example, replacing too many photorealistic features on the face with comic style will destroy users’ identity, such as eyes shape. To solve this problem, the proposed system combines CycleGAN and Pix2Pix. CycleGAN is used to generate paired examples for Pix2Pix and Pix2Pix learns photo-to-comic transformation. To further emphasize a comic style, this paper also proposes two extensions, which divides face images into several parts such as hair and eye parts. The quality of generated images are evaluated with questionnaire by 50 answerers. The results show that the images generated by the proposed system are highly evaluated than the images generated by CycleGAN in terms of preserving the features in a photo and reproducing a comic style. Although the segmentation tend to collapse the generated images, it can reproduce a comic style effectively, especially on facial parts.

Generative Adversarial Networks-Based Ground-Motion Model for Crustal Earthquakes in Japan Considering Detailed Site Conditions

ABSTRACT We develop a ground-motion model (GMM) for crustal earthquakes in Japan that can directly model the probability distribution of ground-motion acceleration time histories based on generative adversarial networks (GANs). The proposed model can generate ground motions conditioned on moment magnitude, rupture distance, and detailed site conditions defined by the average shear-wave velocity in the top 5, 10, and 20 m (VS5, VS10, and VS20) and the depth to shear-wave velocities of 1.0 km/s and 1.4 km/s (Z1.0 and Z1.4). We construct the neural networks based on styleGAN2 and introduce a novel neural network architecture to generate ground motions considering the effect of source, path, and such detailed site conditions. The resulting 5% damped spectral acceleration from the proposed GMM is consistent with empirical GMMs in terms of magnitude and distance scaling. The proposed GMM can also generate ground motions accounting for the shear-wave velocity profiles of surface soil with different magnitudes and distances and represent characteristics that are not explained solely byVS30.

Improving synthetic media generation and detection using generative adversarial networks

Improving synthetic media generation and detection using generative adversarial networks

Digital ray: enhancing cataractous fundus images using style transfer generative adversarial networks to improve retinopathy detection

Background/aimsThe aim of this study was to develop and evaluate digital ray, based on preoperative and postoperative image pairs using style transfer generative adversarial networks (GANs), to enhance cataractous fundus...

Lightweight dual-path octave generative adversarial networks for few-shot image generation

Lightweight dual-path octave generative adversarial networks for few-shot image generation

Variational Autoencoders and Generative Adversarial Networks for Multivariate Scenario Generation

When making decisions with lasting implications over a medium to long timeframe, it is essential to consider not only the most probable scenario, possibly obtained through a forecasting model, but also a range of potential outcomes. This approach allows for effective risk mitigation across a spectrum of scenarios, including less probable ones, and enhances the resilience of planning strategies. In this paper, we demonstrate the development of a generative model capable of learning the multivariate joint probability distribution of link speeds on a road network, using real sensor data. To further enhance the performance of our Generative Adversarial Network model, we employed a Variational AutoEncoder for pre-training the generator network. Experimental results, conducted on three distinct benchmark datasets, highlight the potential of the proposed model in generating new scenario samples of multivariate variables. The Wasserstein distance between the generated distribution and the real data, confirms the good performance of our model compared to state-of-the-art models, based on copulae. The proposed model has shown its ability to generate scenarios that preserve correlations among variables, while producing samples that faithfully represent the empirical marginal distributions.

Deep Learning-Driven Robot Arm Control Fusing Convolutional Visual Perception and Predictive Modeling for Motion Planning

The wide application of robotic technology in various industries from industrial automation to medical assistance is gradually changing our production and lifestyle, and has attracted widespread attention in many fields. However, existing robotic control systems often grapple with limited flexibility and poor adaptability to complex environments, particularly in highly dynamic operational contexts. Against this backdrop, the integration of deep learning technologies offers new possibilities in enhancing robotic perception and decision-making, especially in visual perception and motion planning. To address these challenges, we have introduced a novel robotic arm control network model, MPC-WGAN-Faster R-CNN, which combines Model Predictive Control concepts with Wasserstein Generative Adversarial Networks and Faster R-CNN visual recognition technology. This integration aims to improve the precision and adaptability of robotic arm operations in complex environments.

Automotive Consumer Loans Risk Assessment Predictive Modeling Using Generative Adversarial Network and Stacked Autoencoder Neural Networks

In this study, we propose a predictive model for automotive consumer loans risk assessment, leveraging Generative Adversarial Networks (GANs) and Stacked Autoencoder Neural Networks (SAEs). We address issues of high dimensionality and sparsity inherent in big data environments and tackle the class imbalance problem using GANs. Feature selection is effectively carried out using SAEs. Experimental results prove the superiority of our approach over traditional neural networks and our model without GANs or SAEs. The proposed model shows significant potential for application in personal credit risk assessment within automotive finance and beyond. Future work is aimed at extending and improving our model and applying it to other domains.

Object image reconstruction: method for reconstructing images from digital off-axis holograms using a generative adversarial network

Object image reconstruction: method for reconstructing images from digital off-axis holograms using a generative adversarial network

Research on Pattern Generation and Innovative Design of Chinese Mongolian Embroidery based on AIGC Technology

This study aims to investigate the potential of generative artificial intelligence (AI) technology in the generation and innovation of Mongolian embroidery patterns. It seeks to address the limitations of traditional embroidery pattern design, which is often time- consuming and inefficient, and to enhance the market competitiveness of Mongolian embroidery products. A substantial corpus of Mongolian embroidery patterns has been assembled, and image processing software has been employed to enhance the processing. A generative adversarial network (GAN) and a variational self-encoder model are constructed to learn and train Mongolian embroidery patterns, with the objective of generating new patterns that exhibit a combination of traditional style and modern design concepts. The experimental results demonstrate that the Mongolian embroidery patterns generated by AIGC technology retain the defining characteristics of traditional patterns in terms of form, while also exhibiting a greater degree of diversification in design style through the creative generation of the model.

Detecting anomalous anatomic regions in spatial transcriptomics with STANDS

Detection and Dissection of Anomalous Tissue Domains (DDATD) from multi-sample spatial transcriptomics (ST) data provides unprecedented opportunities to characterize anomalous tissue domains (ATDs), revealing both population-level and individual-specific pathogenic factors for understanding pathogenic heterogeneities behind diseases. However, no current methods can perform de novo DDATD from ST data, especially in the multi-sample context. Here, we introduce STANDS, an innovative framework based on Generative Adversarial Networks which integrates three core tasks in multi-sample DDATD: detecting, aligning, and subtyping ATDs. STANDS incorporates multimodal-learning, transfer-learning, and style-transfer techniques to effectively address major challenges in multi-sample DDATD, including complications caused by unalignable ATDs, under-utilization of multimodal information, and scarcity of normal ST datasets necessary for comparative analysis. Extensive benchmarks from diverse datasets demonstrate STAND’s superiority in identifying both common and individual-specific ATDs and further dissecting them into biologically distinct subdomains. STANDS also provides clues to developing ATDs visually indistinguishable from surrounding normal tissues.

Malicious Participants and Fake Task Detection Incorporating Gaussian Bias

Mobile crowdsensing (MCS) is a combination of crowdsourcing ideas and mobile sensing devices, designed to enable rational allocation of resources at scale. However, the MCS platform is highly vulnerable to injection attacks from malicious participants and fake tasks that interfere with platform service capabilities and sensing activities. To this end, the participant and task submission process is modeled as a multivariate time series, and a detection model for malicious participants and fake tasks (MP-FTD) with a Gaussian prior on the attentional mechanism and a two-stage adversarial training process is proposed. The attention mechanism was corrected using Gaussian bias, and then the corrected attention mechanism was used to obtain the correlation discrepancies between the data. Using the adversarial training method of Generative Adversarial Networks (GAN), the output of the correlation discrepancy reconstruction phase is transformed into a focus score, to amplify the reconstruction error in the output of the focus score reconstruction phase, and to improve the differentiation between the injected data and normal data of malicious attackers. The detection of these malicious attackers will effectively improve the robustness of the sensing platform. Experiments on six real-world datasets showed that the average F1-score reached 93.44%, outperforming the current baseline method, and resulting in an average 12.07% improvement in participant assignment accuracy and an average 12.25% improvement in task assignment accuracy in task assignment experiments.

Research on improved underwater cable image processing technique based on CNN-GAN

In this study, we propose a CNN-GAN-based real-time processing technique for filtering images of underwater cables used in power systems. This addresses the excessive interference impurities that are frequently observed in images captured by remotely operated vehicles (ROVs). The process begins with the input of the original image into the convolutional neural network (CNN). Subsequently, the training outcomes, which serve as input parameters for the generative adversarial network (GAN), facilitate the filtering process. The system also calculates both the structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR), performing model updates via backward propagation. This technique utilizes deep learning technologies to achieve rapid, real-time filtering of underwater cable images. The experimental results reveal that the loss function of the CNN reaches 0.16 with an accuracy of 97.5%, while the loss function of the adversarial GAN network approaches 0.05. Compared with traditional methods such as DDN, JORDOR, RESCAN, and PRENet, the proposed CNN-GAN algorithm exhibits superior performance, as evidenced by the higher PSNR and SSIM values. Specifically, for clear water images, the PSNR reaches 29.86 dB and the SSIM is 0.9045. For severely polluted images, the PSNR is 28.67 dB and the SSIM is 0.8965, while for unevenly illuminated images, the PSNR and SSIM values are 24.37 dB and 0.88, respectively. These enhancements significantly benefit the monitoring and maintenance of power systems.

Robust Remote Sensing Scene Interpretation Based on Unsupervised Domain Adaptation

Deep learning models excel in interpreting the exponentially growing amounts of remote sensing data; however, they are susceptible to deception and spoofing by adversarial samples, posing catastrophic threats. The existing methods to combat adversarial samples have limited performance in robustness and efficiency, particularly in complex remote sensing scenarios. To tackle these challenges, an unsupervised domain adaptation algorithm is proposed for the accurate identification of clean images and adversarial samples by exploring a robust generative adversarial classification network that can harmonize the features between clean images and adversarial samples to minimize distribution discrepancies. Furthermore, linear polynomial loss as a replacement for cross-entropy loss is integrated to guide robust representation learning. Additionally, we leverage the fast gradient sign method (FGSM) and projected gradient descent (PGD) algorithms to generate adversarial samples with varying perturbation amplitudes to assess model robustness. A series of experiments was performed on the RSSCN7 dataset and SIRI-WHU dataset. Our experimental results illustrate that the proposed algorithm performs exceptionally well in classifying clean images while demonstrating robustness against adversarial perturbations.

Towards a fully automatic workflow for investigating the dynamics of lung cancer cachexia during radiotherapy using cone beam computed tomography

Abstract Objective. Cachexia is a devastating condition, characterized by involuntary loss of muscle mass with or without loss of adipose tissue mass. It affects more than half of patients with lung cancer, diminishing treatment effects and increasing mortality. Cone-beam computed tomography (CBCT) images, routinely acquired during radiotherapy treatment, might contain valuable anatomical information for monitoring body composition changes associated with cachexia. For this purpose, we propose an automatic artificial intelligence (AI)-based workflow, consisting of CBCT to CT conversion, followed by segmentation of pectoralis muscles. 
Approach. Data from 140 stage III non-small cell lung cancer patients was used. Two deep learning models, cycle-consistent generative adversarial network (CycleGAN) and contrastive unpaired translation (CUT), were used for unpaired training of CBCT to CT conversion, to generate synthetic CT (sCT) images. The no-new U-Net (nnU-Net) model was used for automatic pectoralis muscle segmentation. To evaluate tissue segmentation performance in the absence of ground truth labels, an uncertainty metric (UM) based on Monte Carlo dropout was developed and validated. 
Main results. Both CycleGAN and CUT restored the Hounsfield unit fidelity of the CBCT images compared to the planning CT (pCT) images and visually reduced streaking artefacts. The nnU-Net model achieved a Dice Similarity Coefficient (DSC) of 0.93, 0.94, 0.92 for the CT, sCT and CBCT images, respectively, on an independent test set. The UM showed a high correlation with DSC with a correlation coefficient of -0.84 for the pCT dataset and -0.89 for the sCT dataset. 
Significance. This paper shows a proof-of-concept for automatic AI-based monitoring of the pectoralis muscle area of lung cancer patients during radiotherapy treatment based on CBCT images, which provides an unprecedented time resolution of muscle mass loss during cachexia progression. Ultimately, the proposed workflow could provide valuable information for early intervention of cachexia, ideally resulting in improved cancer treatment outcome.

Exploring Generative Adversarial Networks for Augmenting Network Intrusion Detection Tasks

The advent of generative networks and their adoption in numerous domains and communities have led to a wave of innovation and breakthroughs in artificial intelligence and machine learning. Generative Adversarial Networks (GANs) have expanded the scope of what is possible with machine learning, allowing for new applications in areas such as computer vision, natural language processing, and creative AI. GANs, in particular, have been used for a wide range of tasks, including image and video generation, data augmentation, style transfer, and anomaly detection. They have also been used for medical imaging and drug discovery, where they can generate synthetic data to augment small datasets, reduce the need for expensive experiments, and lower the number of real patients that must be included in medical trials. Given these developments, we propose using the power of generative adversarial networks to create and augment flow-based network traffic datasets. We evaluate a series of GAN architectures, including Wasserstein, conditional, energy-based, gradient penalty, and LSTM GANs. We evaluate their performance on a set of flow-based network traffic data collected from 16 subjects who used their computers for home, work, and study purposes. The performance of these GAN architectures is described according to metrics that involve networking principles, data distribution among a collection of flows, and temporal data distribution. Given the tendency of network intrusion detection datasets to have a very imbalanced data distribution, i.e., a large number of samples in the “normal traffic” category and a comparatively low number of samples assigned to the “intrusion” categories, we test our GANs by augmenting the intrusion data and checking whether this helps intrusion detection neural networks in their task. We publish the resulting UPBFlow dataset and code on GitHub 1 .