Generative Adversarial Models Research Articles

Patient-specific placental vessel segmentation with limited data

A major obstacle in applying machine learning for medical fields is the disparity between the data distribution of the training images and the data encountered in clinics. This phenomenon can be explained by inconsistent acquisition techniques and large variations across the patient spectrum. The result is poor translation of the trained models to the clinic, which limits their implementation in medical practice. Patient-specific trained networks could provide a potential solution. Although patient-specific approaches are usually infeasible because of the expenses associated with on-the-fly labeling, the use of generative adversarial networks enables this approach. This study proposes a patient-specific approach based on generative adversarial networks. In the presented training pipeline, the user trains a patient-specific segmentation network with extremely limited data which is supplemented with artificial samples generated by generative adversarial models. This approach is demonstrated in endoscopic video data captured during fetoscopic laser coagulation, a procedure used for treating twin-to-twin transfusion syndrome by ablating the placental blood vessels. Compared to a standard deep learning segmentation approach, the pipeline was able to achieve an intersection over union score of 0.60 using only 20 annotated images compared to 100 images using a standard approach. Furthermore, training with 20 annotated images without the use of the pipeline achieves an intersection over union score of 0.30, which, therefore, corresponds to a 100% increase in performance when incorporating the pipeline. A pipeline using GANs was used to generate artificial data which supplements the real data, this allows patient-specific training of a segmentation network. We show that artificial images generated using GANs significantly improve performance in vessel segmentation and that training patient-specific models can be a viable solution to bring automated vessel segmentation to the clinic.

Diffusion probabilistic versus generative adversarial models to reduce contrast agent dose in breast MRI

BackgroundTo compare denoising diffusion probabilistic models (DDPM) and generative adversarial networks (GAN) for recovering contrast-enhanced breast magnetic resonance imaging (MRI) subtraction images from virtual low-dose subtraction images.MethodsRetrospective, ethically approved study. DDPM- and GAN-reconstructed single-slice subtraction images of 50 breasts with enhancing lesions were compared to original ones at three dose levels (25%, 10%, 5%) using quantitative measures and radiologic evaluations. Two radiologists stated their preference based on the reconstruction quality and scored the lesion conspicuity as compared to the original, blinded to the model. Fifty lesion-free maximum intensity projections were evaluated for the presence of false-positives. Results were compared between models and dose levels, using generalized linear mixed models.ResultsAt 5% dose, both radiologists preferred the GAN-generated images, whereas at 25% dose, both radiologists preferred the DDPM-generated images. Median lesion conspicuity scores did not differ between GAN and DDPM at 25% dose (5 versus 5, p = 1.000) and 10% dose (4 versus 4, p = 1.000). At 5% dose, both readers assigned higher conspicuity to the GAN than to the DDPM (3 versus 2, p = 0.007). In the lesion-free examinations, DDPM and GAN showed no differences in the false-positive rate at 5% (15% versus 22%), 10% (10% versus 6%), and 25% (6% versus 4%) (p = 1.000).ConclusionsBoth GAN and DDPM yielded promising results in low-dose image reconstruction. However, neither of them showed superior results over the other model for all dose levels and evaluation metrics. Further development is needed to counteract false-positives.Relevance statementFor MRI-based breast cancer screening, reducing the contrast agent dose is desirable. Diffusion probabilistic models and generative adversarial networks were capable of retrospectively enhancing the signal of low-dose images. Hence, they may supplement imaging with reduced doses in the future.Key points• Deep learning may help recover signal in low-dose contrast-enhanced breast MRI.• Two models (DDPM and GAN) were trained at different dose levels.• Radiologists preferred DDPM at 25%, and GAN images at 5% dose.• Lesion conspicuity between DDPM and GAN was similar, except at 5% dose.• GAN and DDPM yield promising results in low-dose image reconstruction.Graphical

Modelling Rate of Exogenous Glucose Appearance for Biomedical Applications Using Conditional Generative Models

The assessment of oral carbohydrate intake and its rate of exogenous glucose appearance is crucial for monitoring blood glucose in patients who suffer from diabetes and also for healthy individuals, as it is one of the major factors involved in human metabolism. Its accurate modelling is necessary when developing methodologies to mimic the physiological processes within the human body. Considering the recent advancements in data-driven methods that demand non-deterministic solutions to simulate real-life scenarios, this study proposes a novel approach based on conditional generative adversarial models to introduce realistic variability to the models in the state of the art, which are incapable of representing the full variety of scenarios due to their deterministic nature.

Enhancing DDoS Attack Detection in SDNs with GAN-Based Imbalanced Data Augmentation

Securing computer networks has become crucial due to the ongoing emergence of diverse network attacks. The popularity of Software Defined Networks (SDN) has risen because of its ability to enhance network agility, efficiency, and adaptability to recent networking challenges. However, it is essential to note that SDNs, which depend on centralized controllers, can be severely affected by Distributed Denial of Service (DDoS) attacks. The threat of DDoS attacks has grown exponentially, resulting in the evolution of robust Machine Learning-based DDoS attack detection systems within SDN. DDoS attack detection systems may deliver poor performance when trained on imbalanced datasets. Traditional techniques for handling imbalanced datasets need to be revised. Recent advances in generative adversarial networks (GANs) have revealed significant potential in generating synthetic data while preserving the probability distribution of the original data. This innovative procedure offers a promising solution to mitigate the challenges of imbalanced data in DDoS attack detection. To address challenges originating from imbalanced training datasets, we employed Generative Adversarial models to generate adversarial attacks from one viewpoint and evaluate their quality from another perspective. We chose Generative Adversarial Networks (GANs), Bidirectional GANs (Bi-GANs), and Wasserstein GANs (WGANs) based on extensive usage and reliability criteria in various domains. We conducted a comprehensive assessment to evaluate their effectiveness and resilience in generating high-quality attacks. It helps to develop, train, and fine-tune machine and deep learning models to estimate their impacts. We utilized NSL-KDD and CICIDS-2017 datasets to ensure generalization, implementing both ML and DL approaches. The outcomes demonstrate that the WGAN model outperformed GAN, Bi-GAN, and the models trained on the original imbalanced dataset and traditional sampling techniques in binary and multiclass classifications for both datasets.

Evaluating the impact of generative adversarial models on the performance of anomaly intrusion detection

AbstractWith the increasing rate and types of cyber attacks against information systems and communication infrastructures, many tools are needed to detect and mitigate against such attacks, for example, Intrusion Detection Systems (IDSs). Unfortunately, traditional Signature‐based IDSs (SIDSs) perform poorly against previously unseen adversarial attacks. Anomaly‐based IDSs (AIDSs) use Machine Learning (ML) and Deep Learning (DL) approaches to overcome these limitations. However, AIDS performance can be poor when trained on imbalanced datasets. To address the challenge of AIDS performance caused by these unbalanced training datasets, generative adversarial models are proposed to obtain adversarial attacks from one side and analyse their quality from another. According to extensive usage and reliability criteria for generative adversarial models in different disciplines, Generative Adversarial Networks (GANs), Bidirectional GAN (BiGAN), and Wasserstein GAN (WGAN) are employed to serve AIDS. The authors have extensively assessed their abilities and robustness to deliver high‐quality attacks for AIDS. AIDSs are constructed, trained, and tuned based on these models to measure their impacts. The authors have employed two datasets: NSL‐KDD and CICIDS‐2017 for generalisation purposes, where ML and DL approaches are utilised to implement AIDSs. Their results show that the WGAN model outperformed GANs and BiGAN models in binary and multiclass classifications for both datasets.

A histogram-driven generative adversarial network for brain MRI to CT synthesis

Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) are commonly used tools for medical diagnostic assessment. Considering the ionizing radiation of CT imaging, estimating CT images from radiation-free MRI would be beneficial for medical diagnosis. Although state-of-art generative adversarial networks or end-to-end architecture models can generate realistic natural images, it is challenging to generate medical CT images with high signal-to-noise ratios and are paired with MRI. We propose a histogram-driven generative adversarial network (HisGAN) to address this issue, estimate CT images paired with MR, and develop a histogram-based dynamic scaling factor to facilitate learning different image styles. By employing an adversarial learning strategy to train the end-to-end generator, the generator better simulate the nonlinear mapping from source to target. For the generator, the proposed method applies multiple learnable parameters to adjust the overall weights of the dilated convolution layers to ensure sufficient expansive receptive fields for improved performance. Additionally, the method utilizes deep residual networks to train randomly smoothed generated images and employs adversarial loss to enhance the generation of the discriminator, achieving a balance between the generator and the discriminator. Our approach can synthesize image details at the pixel level in the target domain and has been evaluated using two datasets for MR to CT, T1 to T2, and Flair to T1ce modality synthesis tasks. The proposed method outperforms existing generative adversarial models applied to medical image synthesis.

Deep learning techniques for biomedical data processing

The interest in Deep Learning (DL) has seen an exponential growth in the last ten years, producing a significant increase in both theoretical and applicative studies. On the one hand, the versatility and the ability to tackle complex tasks have led to the rapid and widespread diffusion of DL technologies. On the other hand, the dizzying increase in the availability of biomedical data has made classical analyses, carried out by human experts, progressively more unlikely. Contextually, the need for efficient and reliable automatic tools to support clinicians, at least in the most demanding tasks, has become increasingly pressing. In this survey, we will introduce a broad overview of DL models and their applications to biomedical data processing, specifically to medical image analysis, sequence processing (RNA and proteins) and graph modeling of molecular data interactions. First, the fundamental key concepts of DL architectures will be introduced, with particular reference to neural networks for structured data, convolutional neural networks, generative adversarial models, and siamese architectures. Subsequently, their applicability for the analysis of different types of biomedical data will be shown, in areas ranging from diagnostics to the understanding of the characteristics underlying the process of transcription and translation of our genetic code, up to the discovery of new drugs. Finally, the prospects and future expectations of DL applications to biomedical data will be discussed.

Deep Learning Methods Used in Remote Sensing Images: A Review

Undeniably, Deep Learning (DL) has rapidly eroded traditional machine learning in Remote Sensing (RS) and geoscience domains with applications such as scene understanding, material identification, extreme weather detection, oil spill identification, among many others. Traditional machine learning algorithms are given less and less attention in the era of big data. Recently, a substantial amount of work aimed at developing image classification approaches based on the DL model's success in computer vision. The number of relevant articles has nearly doubled every year since 2015. Advances in remote sensing technology, as well as the rapidly expanding volume of publicly available satellite imagery on a worldwide scale, have opened up the possibilities for a wide range of modern applications. However, there are some challenges related to the availability of annotated data, the complex nature of data, and model parameterization, which strongly impact performance. In this article, a comprehensive review of the literature encompassing a broad spectrum of pioneer work in remote sensing image classification is presented including network architectures (vintage Convolutional Neural Network, CNN; Fully Convolutional Networks, FCN; encoder-decoder, recurrent networks; attention models, and generative adversarial models). The characteristics, capabilities, and limitations of current DL models were examined, and potential research directions were discussed.

Generative models for sound field reconstruction.

This work examines the use of generative adversarial networks for reconstructing sound fields from experimental data. It is investigated whether generative models, which learn the underlying statistics of a given signal or process, can improve the spatio-temporal reconstruction of a sound field by extending its bandwidth. The problem is significant as acoustic array processing is naturally band limited by the spatial sampling of the sound field (due to the difficulty to satisfy the Nyquist criterion in space domain at high frequencies). In this study, the reconstruction of spatial room impulse responses in a conventional room is tested based on three different generative adversarial models. The results indicate that the models can improve the reconstruction, mostly by recovering some of the sound field energy that would otherwise be lost at high frequencies. There is an encouraging outlook in the use of statistical learning models to overcome the bandwidth limitations of acoustic sensor arrays. The approach can be of interest in other areas, such as computational acoustics, to alleviate the classical computational burden at high frequencies.

A new generative approach for optical coherence tomography data scarcity: unpaired mutual conversion between scanning presets

In optical coherence tomography (OCT), there is a trade-off between the scanning time and image quality, leading to a scarcity of high quality data. OCT platforms provide different scanning presets, producing visually distinct images, limiting their compatibility. In this work, a fully automatic methodology for the unpaired visual conversion of the two most prevalent scanning presets is proposed. Using contrastive unpaired translation generative adversarial architectures, low quality images acquired with the faster Macular Cube preset can be converted to the visual style of high visibility Seven Lines scans and vice-versa. This modifies the visual appearance of the OCT images generated by each preset while preserving natural tissue structure. The quality of original and synthetic generated images was compared using brisque. The synthetic generated images achieved very similar scores to original images of their target preset. The generative models were validated in automatic and expert separability tests. These models demonstrated they were able to replicate the genuine look of the original images. This methodology has the potential to create multi-preset datasets with which to train robust computer-aided diagnosis systems by exposing them to the visual features of different presets they may encounter in real clinical scenarios without having to obtain additional data.Graphical Unpaired mutual conversion between scanning presets. Two generative adversarial models are trained for the conversion of OCT images into images of another scanning preset, replicating the visual features that characterise said preset.

Differentiable and Scalable Generative Adversarial Models for Data Imputation

Differentiable and Scalable Generative Adversarial Models for Data Imputation

ResViT: Residual Vision Transformers for Multimodal Medical Image Synthesis.

Generative adversarial models with convolutional neural network (CNN) backbones have recently been established as state-of-the-art in numerous medical image synthesis tasks. However, CNNs are designed to perform local processing with compact filters, and this inductive bias compromises learning of contextual features. Here, we propose a novel generative adversarial approach for medical image synthesis, ResViT, that leverages the contextual sensitivity of vision transformers along with the precision of convolution operators and realism of adversarial learning. ResViT's generator employs a central bottleneck comprising novel aggregated residual transformer (ART) blocks that synergistically combine residual convolutional and transformer modules. Residual connections in ART blocks promote diversity in captured representations, while a channel compression module distills task-relevant information. A weight sharing strategy is introduced among ART blocks to mitigate computational burden. A unified implementation is introduced to avoid the need to rebuild separate synthesis models for varying source-target modality configurations. Comprehensive demonstrations are performed for synthesizing missing sequences in multi-contrast MRI, and CT images from MRI. Our results indicate superiority of ResViT against competing CNN- and transformer-based methods in terms of qualitative observations and quantitative metrics.

Disentangled Representation Learning and Generation With Manifold Optimization.

Disentanglement is a useful property in representation learning, which increases the interpretability of generative models such as variational autoencoders (VAE), generative adversarial models, and their many variants. Typically in such models, an increase in disentanglement performance is traded off with generation quality. In the context of latent space models, this work presents a representation learning framework that explicitly promotes disentanglement by encouraging orthogonal directions of variations. The proposed objective is the sum of an autoencoder error term along with a principal component analysis reconstruction error in the feature space. This has an interpretation of a restricted kernel machine with the eigenvector matrix valued on the Stiefel manifold. Our analysis shows that such a construction promotes disentanglement by matching the principal directions in the latent space with the directions of orthogonal variation in data space. In an alternating minimization scheme, we use the Cayley ADAM algorithm, a stochastic optimization method on the Stiefel manifold along with the Adam optimizer. Our theoretical discussion and various experiments show that the proposed model is an improvement over many VAE variants in terms of both generation quality and disentangled representation learning.

Uncertainty-Guided Optimal Scheduling of Automated Energy Internet

This article tries to propose a novel uncertainty-guided day-ahead scheduling framework for the microgrid energy Internet incorporating the renewable energy sources, battery storage, and electric vehicles. The proposed model makes the use of the prediction interval concept to capture the uncertainty effects in the bandwidth and create an optimal interval for the output cost function. To this end, a deep learning approach based on the generative adversarial models (GAMs) is designed to not only have a prediction interval with high confidence level but also make the least bandwidth with the maximum possible information. Moreover, an evolving approach based on bat algorithm is proposed to help to train the GAM optimally. Finally, the model performance and quality are assessed using the IEEE standard test system.

Next-generation deep learning based on simulators and synthetic data.

Deep learning (DL) is being successfully applied across multiple domains, yet these models learn in a most artificial way: they require large quantities of labeled data to grasp even simple concepts. Thus, the main bottleneck is often access to supervised data. Here, we highlight a trend in a potential solution to this challenge: synthetic data. Synthetic data are becoming accessible due to progress in rendering pipelines, generative adversarial models, and fusion models. Moreover, advancements in domain adaptation techniques help close the statistical gap between synthetic and real data. Paradoxically, this artificial solution is also likely to enable more natural learning, as seen in biological systems, including continual, multimodal, and embodied learning. Complementary to this, simulators and deep neural networks (DNNs) will also have a critical role in providing insight into the cognitive and neural functioning of biological systems. We also review the strengths of, and opportunities and novel challenges associated with, synthetic data.

Super-Resolution Reconstruction of Remote Sensing Images Using Generative Adversarial Network With Shallow Information Enhancement

The super-resolution reconstruction method based on deep learning can significantly improve the spatial super-resolution of remote sensing images. However, the current methods make insufficient use of the remote context information and channel information in shallow feature extraction, resulting in the limited effect of super-resolution reconstruction. This paper proposed a new super-resolution reconstruction model, SIEGAN, which uses generative adversarial network with shallow information enhancement to improve the effect of super-resolution reconstruction of remote sensing images. Similar to other generative adversarial models, SIEGAN is composed of generator and discriminator. But SIEGAN enhances the generator's ability to extract shallow information by using three different scale convolution operations. Specifically, a depth-wise convolution is used to extract the local context information of each band of the image. A depth-wise dilation convolution is used to capture the remote context information in the image. Finally, a 1×1 convolution is used to extract the correlation features between different channels in remote sensing images. In addition, SIEGAN uses U-Net network as its discriminator to provide detailed feedback per pixel to the generator, to improve the model's ability to identify image details. And the spectral-spatial total variation loss function is introduced to ensure the spectral-spatial reliability of the reconstructed images. The experimental results on Gaofen-1 data proved that compared with the state-of-the-art models, SIEGAN has achieved better super-resolution reconstruction performance. Furthermore, the reconstructed images by SIEGAN demonstrate better performance in land cover classification.

End-to-End Domain Adaptive Attention Network for Cross-Domain Person Re-Identification

Person re-identification (re-ID) remains challenging in a real-world scenario, as it requires a trained network to generalise to totally unseen target data in the presence of variations across domains. Recently, generative adversarial models have been widely adopted to enhance the diversity of training data. These approaches, however, often fail to generalise to other domains, as existing generative person re-identification models have a disconnect between the generative component and the discriminative feature learning stage. To address the on-going challenges regarding model generalisation, we propose an end-to-end domain adaptive attention network to jointly translate images between domains and learn discriminative re-id features in a single framework. To address the domain gap challenge, we introduce an attention module for image translation from source to target domains without affecting the identity of a person. More specifically, attention is directed to the background instead of the entire image of the person, ensuring identifying characteristics of the subject are preserved. The proposed joint learning network results in a significant performance improvement over state-of-the-art methods on several challenging benchmark datasets.

A Framework of Composite Functional Gradient Methods for Generative Adversarial Models.

Generative adversarial networks (GAN) are trained through a minimax game between a generator and a discriminator to generate data that mimics observations. While being widely used, GAN training is known to be empirically unstable. This paper presents a new theory for generative adversarial methods that does not rely on the traditional minimax formulation. Our theory shows that with a strong discriminator, a good generator can be obtained by composite functional gradient learning, so that several distance measures (including the KL divergence and the JS divergence) between the probability distributions of real data and generated data are simultaneously improved after each functional gradient step until converging to zero. This new point of view leads to stable procedures for training generative models. It also gives a new theoretical insight into the original GAN. Empirical results on image generation show the effectiveness of our new method.

Multimodal deep generative adversarial models for scalable doubly semi-supervised learning

The comprehensive utilization of incomplete multi-modality data is a difficult problem with strong practical value. Most of the previous multimodal learning algorithms require massive training data with complete modalities and annotated labels, which greatly limits their practicality. Although some existing algorithms can be used to complete the data imputation task, they still have two disadvantages: (1) they cannot control the semantics of the imputed modalities accurately; and (2) they need to establish multiple independent converters between any two modalities when extended to multimodal cases. To overcome these limitations, we propose a novel doubly semi-supervised multimodal learning (DSML) framework. Specifically, DSML uses a modality-shared latent space and multiple modality-specific generators to associate multiple modalities together. Here we divided the shared latent space into two independent parts, the semantic labels and the semantic-free styles, which allows us to easily control the semantics of generated samples. In addition, each modality has its own separate encoder and classifier to infer the corresponding semantic and semantic-free latent variables. The above DSML framework can be adversarially trained by using our specially designed softmax-based discriminators. Large amounts of experimental results show that the DSML obtains better performance than the baselines on three tasks, including semi-supervised classification, missing modality imputation and cross-modality retrieval.

Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation

Training generative models that can generate high-quality text with sufficient diversity is an important open problem for Natural Language Generation (NLG) community. Recently, generative adversarial models have been applied extensively on text generation tasks, where the adversarially trained generators alleviate the exposure bias experienced by conventional maximum likelihood approaches and result in promising generation quality. However, due to the notorious defect of mode collapse for adversarial training, the adversarially trained generators face a quality-diversity trade-off, i.e., the generator models tend to sacrifice generation diversity severely for increasing generation quality. In this paper, we propose a novel approach which aims to improve the performance of adversarial text generation via efficiently decelerating mode collapse of the adversarial training. To this end, we introduce a cooperative training paradigm, where a language model is cooperatively trained with the generator and we utilize the language model to efficiently shape the data distribution of the generator against mode collapse. Moreover, instead of engaging the cooperative update for the generator in a principled way, we formulate a meta learning mechanism, where the cooperative update to the generator serves as a high level meta task, with an intuition of ensuring the parameters of the generator after the adversarial update would stay resistant against mode collapse. In the experiment, we demonstrate our proposed approach can efficiently slow down the pace of mode collapse for the adversarial text generators. Overall, our proposed method is able to outperform the baseline approaches with significant margins in terms of both generation quality and diversity in the testified domains.