Learning Generative Adversarial Networks Research Articles

DLGAN: Undersampled MRI reconstruction using Deep Learning based Generative Adversarial Network

Magnetic Resonance Imaging (MRI) is a crucial tool for quantitative image analysis and clinical diagnosis, providing detailed anatomical images to assist in the detection of various abnormalities. However, the widespread use of MRI is hindered by the challenges associated with long sampling periods and low-resolution image processing, prompting the development of various traditional methods to address these limitations. In recent times, advanced Deep Learning (DL) techniques have been applied to tackle the inverse problem of reconstructing MRI images from undersampled data. These DL models have demonstrated substantial improvements in terms of image reconstruction performance, cost-effectiveness, and reduced acquisition time, offering significant potential for further enhancements in this domain. This study introduces a novel DLGAN (Deep Learning Generative Adversarial Network) model comprising two sub-GAN modules tailored for specific datasets. Each module incorporates DLGAN (Generator Block, GB) and DLGAN (Discriminator Block, DB) blocks, strategically designed to regenerate MR images from K Space data. These blocks effectively leverage information from both the ground truth and KSpace characteristics, resulting in enhanced image reconstruction performance while also reducing complexity and improving overall efficiency. The DLGAN model effectively addresses common issues such as artefact removal and vanishing-gradient problems through the extraction of hierarchical features. To evaluate the model’s effectiveness, comprehensive experiments were conducted, utilizing metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). The experimental findings demonstrate that the proposed DLGAN model outperforms the most recent designs in MRI image reconstruction, showcasing its potential for significant advancements in clinical diagnosis and quantitative image analysis.

Design and Parameter Optimization of Zero Position Code Considering Diffraction Based on Deep Learning Generative Adversarial Networks

Absolute measurement has consistently been the primary focus in the development of precision linear and angular displacement measurements. The scheme design of binary zero position codes is an important factor for absolute measurement. Designing and optimizing high-bit zero position codes with over 100 bits face considerable challenges. Simultaneously, the working parameters of zero position codes [unit code width (b), distance (d), and yaw angle (α)] remarkably affect their post-installation performance, particularly in absolute positioning and limit code application in multi-degree-of-freedom measurement schemes. This study addresses these challenges by proposing a design method for zero position codes that considers diffraction based on generative adversarial networks and aims to explore a design with increased efficiency and accuracy as well as optimization for high-bit zero position codes. Additionally, the tolerance range of zero positioning performance for each working parameter is examined. By leveraging the adversarial network structure, this study generates the optimization of a 150-bit code and processes the tests of the zero position code by using simulation results. The following working parameter ranges for code design are recommended on the basis of theoretical and experimental results: b greater than 10 μm, d and α within 1000 μm and 3490 μrad, and avoidance of intervals with sharp changes in the full width at half maximum. The proposed code design and parameter optimization lay a solid foundation for research and engineering applications in absolute measurement field and have considerable potential for generalization and wide applicability.

IFL-GAN: Improved Federated Learning Generative Adversarial Network With Maximum Mean Discrepancy Model Aggregation.

The generative adversarial network (GAN) is usually built from the centralized, independent identically distributed (i.i.d.) training data to generate realistic-like instances. In real-world applications, however, the data may be distributed over multiple clients and hard to be gathered due to bandwidth, departmental coordination, or storage concerns. Although existing works, such as federated learning GAN (FL-GAN), adopt different distributed strategies to train GAN models, there are still limitations when data are distributed in a non-i.i.d. manner. These studies suffer from convergence difficulty, producing generated data with low quality. Fortunately, we found that these challenges are often due to the use of a federated averaging strategy to aggregate local GAN models' updates. In this article, we propose an alternative approach to tackling this problem, which learns a globally shared GAN model by aggregating locally trained generators' updates with maximum mean discrepancy (MMD). In this way, we term our approach improved FL-GAN (IFL-GAN). The MMD score helps each local GAN hold different weights, making the global GAN in IFL-GAN getting converged more rapidly than federated averaging. Extensive experiments on MNIST, CIFAR10, and SVHN datasets demonstrate the significant improvement of our IFL-GAN in both achieving the highest inception score and producing high-quality instances.

A joint learning method for incomplete and imbalanced data in electronic health record based on generative adversarial networks

Electronic health records (EHR), present challenges of incomplete and imbalanced data in clinical predictions. Previous studies addressed these two issues with two-step separately, which caused the decrease in the performance of prediction tasks. In this paper, we propose a unified framework to simultaneously addresses the challenges of incomplete and imbalanced data in EHR. Based on the framework, we develop a model called Missing Value Imputation and Imbalanced Learning Generative Adversarial Network (MVIIL-GAN). We use MVIIL-GAN to perform joint learning on the imputation process of high missing rate data and the conditional generation process of EHR data. The joint learning is achieved by introducing two discriminators to distinguish the fake data from the generated data at sample-level and variable-level. MVIIL-GAN integrate the missing values imputation and data generation in one step, improving the consistency of parameter optimization and the performance of prediction tasks. We evaluate our framework using the public dataset MIMIC-IV with high missing rates data and imbalanced data. Experimental results show that MVIIL-GAN outperforms existing methods in prediction performance. The implementation of MVIIL-GAN can be found at https://github.com/Peroxidess/MVIIL-GAN.

Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module.

Recently, contrastive learning has gained popularity in the field of unsupervised image-to-image (I2I) translation. In a previous study, a query-selected attention (QS-Attn) module, which employed an attention matrix with a probability distribution, was used to maximize the mutual information between the source and translated images. This module selected significant queries using an entropy metric computed from the attention matrix. However, it often selected many queries with equal significance measures, leading to an excessive focus on the background. In this study, we proposed a dual-learning framework with QS-Attn and convolutional block attention module (CBAM) called object-stable dual contrastive learning generative adversarial network (OS-DCLGAN). In this paper, we utilize a CBAM, which learns what and where to emphasize or suppress, thereby refining intermediate features effectively. This CBAM was integrated before the QS-Attn module to capture significant domain information for I2I translation tasks. The proposed framework outperformed recently introduced approaches in various I2I translation tasks, showing its effectiveness and versatility. The code is available at https://github.com/RedPotatoChip/OSUDL.

Data Augmentation Techniques for Machine Learning Applied to Optical Spectroscopy Datasets in Agrifood Applications: A Comprehensive Review.

Machine learning (ML) and deep learning (DL) have achieved great success in different tasks. These include computer vision, image segmentation, natural language processing, predicting classification, evaluating time series, and predicting values based on a series of variables. As artificial intelligence progresses, new techniques are being applied to areas like optical spectroscopy and its uses in specific fields, such as the agrifood industry. The performance of ML and DL techniques generally improves with the amount of data available. However, it is not always possible to obtain all the necessary data for creating a robust dataset. In the particular case of agrifood applications, dataset collection is generally constrained to specific periods. Weather conditions can also reduce the possibility to cover the entire range of classifications with the consequent generation of imbalanced datasets. To address this issue, data augmentation (DA) techniques are employed to expand the dataset by adding slightly modified copies of existing data. This leads to a dataset that includes values from laboratory tests, as well as a collection of synthetic data based on the real data. This review work will present the application of DA techniques to optical spectroscopy datasets obtained from real agrifood industry applications. The reviewed methods will describe the use of simple DA techniques, such as duplicating samples with slight changes, as well as the utilization of more complex algorithms based on deep learning generative adversarial networks (GANs), and semi-supervised generative adversarial networks (SGANs).

Deep learning-based ultrasound transducer induced CT metal artifact reduction using generative adversarial networks for ultrasound-guided cardiac radioablation

In US-guided cardiac radioablation, a possible workflow includes simultaneous US and planning CT acquisitions, which can result in US transducer-induced metal artifacts on the planning CT scans. To reduce the impact of these artifacts, a metal artifact reduction (MAR) algorithm has been developed based on a deep learning Generative Adversarial Network called Cycle-MAR, and compared with iMAR (Siemens), O-MAR (Philips) and MDT (ReVision Radiology), and CCS-MAR (Combined Clustered Scan-based MAR). Cycle-MAR was trained with a supervised learning scheme using sets of paired clinical CT scans with and without simulated artifacts. It was then evaluated on CT scans with real artifacts of an anthropomorphic phantom, and on sets of clinical CT scans with simulated artifacts which were not used for Cycle-MAR training. Image quality metrics and HU value-based analysis were used to evaluate the performance of Cycle-MAR compared to the other algorithms. The proposed Cycle-MAR network effectively reduces the negative impact of the metal artifacts. For example, the calculated HU value improvement percentage for the cardiac structures in the clinical CT scans was 59.58%, 62.22%, and 72.84% after MDT, CCS-MAR, and Cycle-MAR application, respectively. The application of MAR algorithms reduces the impact of US transducer-induced metal artifacts on CT scans. In comparison to iMAR, O-MAR, MDT, and CCS-MAR, the application of developed Cycle-MAR network on CT scans performs better in reducing these metal artifacts.

Mutual learning generative adversarial network

Mutual learning generative adversarial network

Supervised Anomaly Detection via Conditional Generative Adversarial Network and Ensemble Active Learning.

Anomaly detection has wide applications in machine intelligence but is still a difficult unsolved problem. Major challenges include the rarity of labeled anomalies and it is a class highly imbalanced problem. Traditional unsupervised anomaly detectors are suboptimal while supervised models can easily make biased predictions towards normal data. In this paper, we present a new supervised anomaly detector through introducing the novel Ensemble Active Learning Generative Adversarial Network (EAL-GAN). EAL-GAN is a conditional GAN having a unique one generator vs. multiple discriminators architecture where anomaly detection is implemented by an auxiliary classifier of the discriminator. In addition to using the conditional GAN to generate class balanced supplementary training data, an innovative ensemble learning loss function ensuring each discriminator makes up for the deficiencies of the others is designed to overcome the class imbalanced problem, and an active learning algorithm is introduced to significantly reduce the cost of labeling real-world data. We present extensive experimental results to demonstrate that the new anomaly detector consistently outperforms a variety of SOTA methods by significant margins.

Effects of Sim2Real Image Translation via DCLGAN on Lane Keeping Assist System in CARLA Simulator

Autonomous vehicle (AV) simulation using a virtual environment has the advantage of being able to test algorithms in various scenarios with reduced resources. However, there may exist a visual gap between the virtual environment and the real-world. In this paper, in order to mitigate this gap, we trained Dual Contrastive Learning Generative Adversarial Networks (DCLGAN) to realistically convert the image of the CARLA simulator and then evaluated the effect of the Sim2Real conversion focusing on the lane keeping assist system (LKAS). Moreover, in order to avoid the case where the lane is translated distortedly by DCLGAN, we found the optimal training hyperparameters using feature similarity (FSIM). After training, we built a system that connected the CARLA simulator with DCLGAN and AV in real-time. As for the result, we collected data and analyzed them using the following four methods. First, image reality was measured with Fréchet Inception Distance (FID), which we quantitatively verified to reflect the lane characteristics. The CARLA images that passed through DCLGAN had smaller FID values than the original images. Second, lane segmentation accuracy through ENet-SAD was improved by DCLGAN. Third, in the curved route, the case of using DCLGAN drove closer to the center of the lane and had a high success rate. Lastly, in the straight route, DCLGAN improved lane restoring ability after deviating from the center of the lane as much as in reality. Consequently, it convinced that the proposed method could be applicable to mitigate the gap of simulation toward real-world.

Application of Local Color Simulation Method of Landscape Painting Based on Deep Learning Generative Adversarial Networks

The traditional computer simulation landscape painting has the defect that it cannot completely present the painting, the application research of local color simulation method of landscape painting based on deep learning adversarial networks is proposed. Based on the improved generative adversarial networks design, the semantic label hierarchical classification design of landscape paintings is carried out. Through the training and design of generative adversarial networks, the generator and discriminator of landscape painting are designed, and the semantic segmentation algorithm of landscape painting of generative adversarial networks is proposed. Finally, a simulation experiment test is carried out on the local color simulation of landscape painting. The results show that this application method is better than the traditional computer simulation method, it can fully reflect the realism of landscape art painting and the integrity of the picture itself. The texture detail clarity of the generated map is stronger than that of the traditional one, and the semantic content is more accurate. It has important practical reference value.

Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory

Two Rényi-type generalizations of the Shannon cross-entropy, the Rényi cross-entropy and the Natural Rényi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we derive the Rényi and Natural Rényi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family, and we tabulate the results for ease of reference. We also summarise the Rényi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources.

RareGAN: Generating Samples for Rare Classes

We study the problem of learning generative adversarial networks (GANs) for a rare class of an unlabeled dataset subject to a labeling budget. This problem is motivated from practical applications in domains including security (e.g., synthesizing packets for DNS amplification attacks), systems and networking (e.g., synthesizing workloads that trigger high resource usage), and machine learning (e.g., generating images from a rare class). Existing approaches are unsuitable, either requiring fully-labeled datasets or sacrificing the fidelity of the rare class for that of the common classes. We propose RareGAN, a novel synthesis of three key ideas: (1) extending conditional GANs to use labelled and unlabelled data for better generalization; (2) an active learning approach that requests the most useful labels; and (3) a weighted loss function to favor learning the rare class. We show that RareGAN achieves a better fidelity-diversity tradeoff on the rare class than prior work across different applications, budgets, rare class fractions, GAN losses, and architectures.

PO-1641 Virtual brain MRI missing sequences creation using machine learning generative adversarial networks

PO-1641 Virtual brain MRI missing sequences creation using machine learning generative adversarial networks

A deep learning pipeline to simulate fluorodeoxyglucose (FDG) uptake in head and neck cancers using non-contrast CT images without the administration of radioactive tracer

ObjectivesPositron emission tomography (PET) imaging is a costly tracer-based imaging modality used to visualise abnormal metabolic activity for the management of malignancies. The objective of this study is to demonstrate that non-contrast CTs alone can be used to differentiate regions with different Fluorodeoxyglucose (FDG) uptake and simulate PET images to guide clinical management.MethodsPaired FDG-PET and CT images (n = 298 patients) with diagnosed head and neck squamous cell carcinoma (HNSCC) were obtained from The cancer imaging archive. Random forest (RF) classification of CT-derived radiomic features was used to differentiate metabolically active (tumour) and inactive tissues (ex. thyroid tissue). Subsequently, a deep learning generative adversarial network (GAN) was trained for this CT to PET transformation task without tracer injection. The simulated PET images were evaluated for technical accuracy (PERCIST v.1 criteria) and their ability to predict clinical outcome [(1) locoregional recurrence, (2) distant metastasis and (3) patient survival].ResultsFrom 298 patients, 683 hot spots of elevated FDG uptake (elevated SUV, 6.03 ± 1.71) were identified. RF models of intensity-based CT-derived radiomic features were able to differentiate regions of negligible, low and elevated FDG uptake within and surrounding the tumour. Using the GAN-simulated PET image alone, we were able to predict clinical outcome to the same accuracy as that achieved using FDG-PET images.ConclusionThis pipeline demonstrates a deep learning methodology to simulate PET images from CT images in HNSCC without the use of radioactive tracer. The same pipeline can be applied to other pathologies that require PET imaging.

Lung cancer classification using exponential mean saturation linear unit activation function in various generative adversarial network models

AbstractNowadays, the mortality rate due to lung cancer increases rapidly worldwide as it can be classified only at the later stages. Early classification of lung cancer will help patients to take treatment and decrease the death rate. The limited dataset and diversity of data samples are the bottlenecks for early classification. In this paper, robust deep learning generative adversarial network (GAN) models are employed to enhance the dataset and to increase classification accuracy. The activation function plays an important feature‐learning role in neural networks. Since the existing activation functions suffer from various drawbacks such as vanishing gradient, dead neurons, output offset, etc., this paper proposes a novel activation function exponential mean saturation linear unit (EMSLU), which aims to speed up training, reduce network running time, and improve classification accuracy. The experiments were conducted using vanilla GAN, Wasserstein generative adversarial network, Wasserstein generative adversarial network with gradient penalty, conditional generative adversarial network, and deep convolutional generative adversarial network. Each GAN is tested with rectified linear unit, exponential linear unit, and proposed EMSLU activation functions. The results show that all the GAN's with EMSLU yields improved precision, recall, F1‐score, and accuracy.

Oncological Applications of Deep Learning Generative Adversarial Networks

Oncological Applications of Deep Learning Generative Adversarial Networks

StyleGANs and Transfer Learning for Generating Synthetic Images in Industrial Applications

Deep learning applications on computer vision involve the use of large-volume and representative data to obtain state-of-the-art results due to the massive number of parameters to optimise in deep models. However, data are limited with asymmetric distributions in industrial applications due to rare cases, legal restrictions, and high image-acquisition costs. Data augmentation based on deep learning generative adversarial networks, such as StyleGAN, has arisen as a way to create training data with symmetric distributions that may improve the generalisation capability of built models. StyleGAN generates highly realistic images in a variety of domains as a data augmentation strategy but requires a large amount of data to build image generators. Thus, transfer learning in conjunction with generative models are used to build models with small datasets. However, there are no reports on the impact of pre-trained generative models, using transfer learning. In this paper, we evaluate a StyleGAN generative model with transfer learning on different application domains—training with paintings, portraits, Pokémon, bedrooms, and cats—to generate target images with different levels of content variability: bean seeds (low variability), faces of subjects between 5 and 19 years old (medium variability), and charcoal (high variability). We used the first version of StyleGAN due to the large number of publicly available pre-trained models. The Fréchet Inception Distance was used for evaluating the quality of synthetic images. We found that StyleGAN with transfer learning produced good quality images, being an alternative for generating realistic synthetic images in the evaluated domains.

Generative adversarial network with object detector discriminator for enhanced defect detection on ultrasonic B-scans

Non-destructive testing is a set of techniques for defect detection in materials. While the set of imaging techniques is manifold, ultrasonic imaging is the one used the most. The analysis is mainly performed by human inspectors manually analyzing the acquired images. A low number of defects in real ultrasonic inspections and legal issues concerning data from such inspections make it difficult to obtain proper results from automatic ultrasonic image (B-scan) analysis. The goal of presented research is to obtain an improvement of the detection results by expanding the training data set with realistic synthetic samples. In this paper, we present a novel deep learning Generative Adversarial Network model for generating realistic ultrasonic B-scans with defects in distinct locations. Furthermore, we show that generated B-scans can be used for synthetic data augmentation, and can improve the performances of deep convolutional neural object detection networks. Our novel method was developed on a dataset with almost 4000 images and more than 6000 annotated defects. When trained only on real data, detector can achieve an average precision of 70%. By training only on generated data the results increased to 72%, and by mixing generated and real data we achieve almost 76% average precision. We believe that synthetic data generation can generalize to other tasks with limited data. It could also be used for training human personnel.

Improving the image quality and quantitative accuracy of arterial spin labeling based on generative adversarial network

Objective: To investigate the use of deep learning approaches to improve the image quality of arterial spin labeling (ASL) and optimize its quantitative accuracy for cerebral blood flow (CBF). Methods: The clinical and imaging data of 101 patients with cerebrovascular disease in Tianjin Huanhu Hospital from May 2018 to August 2019 were retrospectively collected. Patients were divided into a training set (71 cases) and a validation set (30 cases). In training set, there were 53 cases of male,18 cases of female,with age of 55.0 (41.3, 64.5), and in validation set, there were 23 cases of male,7 cases of female,with age of57.5 (49.0, 65.0). Quantitative perfusion weighted imaging was used as the reference standard, and the original ASL-CBF images were reconstructed by training a deep learning generative adversarial network (GAN). The image quality of original ASL-CBF and GAN-CBF was compared by the structural similarity index and the normalized root mean square error. Pearson correlation coefficient was used to analyze the correlation between ASL-CBF, GAN-CBF and quantitative perfusion in different cerebral vascular blood supply areas and stroke areas, and to verify the improvement effect of GAN on the image quality and quantitative accuracy of ASL. Results: There were significant differences in gender, age, disease type, site and size between training set and validation set (all P>0.05). Compared with ASL-CBF, GAN-CBF had a higher structural similarity index (0.888 vs 0.801, P<0.001), and a lower normalized root mean square error (0.628 vs 0.775, P<0.001). The correlation between GAN-CBF and quantitative perfusion in different blood vessel supply areas and stroke lesion areas was improved compared with the original ASL-CBF, in which the perforating branch of the middle artery (r=0.853) and stroke lesion areas (r=0.765) were the most obvious (all P<0.001). Conclusion: The generative adversarial network could improve the image quality and quantization accuracy of ASL without increasing the scanning time, and expand the clinical application value of ASL.