Generative Adversarial Networks Training Research Articles

Adaptive Weighted Discriminator for Training Generative Adversarial Networks.

Generative adversarial network (GAN) has become one of the most important neural network models for classical unsupervised machine learning. A variety of discriminator loss functions have been developed to train GAN's discriminators and they all have a common structure: a sum of real and fake losses that only depends on the actual and generated data respectively. One challenge associated with an equally weighted sum of two losses is that the training may benefit one loss but harm the other, which we show causes instability and mode collapse. In this paper, we introduce a new family of discriminator loss functions that adopts a weighted sum of real and fake parts, which we call adaptive weighted loss functions or aw-loss functions. Using the gradients of the real and fake parts of the loss, we can adaptively choose weights to train a discriminator in the direction that benefits the GAN's stability. Our method can be potentially applied to any discriminator model with a loss that is a sum of the real and fake parts. For our experiments, SN-GAN, AutoGAN, and BigGAN are used. Experiments validated the effectiveness of our loss functions on unconditional and conditional image generation tasks, improving the baseline results by a significant margin on CIFAR-10, STL-10, and CIFAR-100 datasets in Inception Scores (IS) and Fréchet Inception Distance (FID) metrics.

Deep Learning Combined with Surface-Enhanced Raman Spectroscopy for Chemical Sensing and Recognition

Surface-enhanced Raman spectroscopy (SERS) combines the unique advantages of Raman spectroscopy – namely detection of specific molecular fingerprints without requiring prior labelling – with enhanced sensitivity. SERS takes advantage of the amplification of the incident electric field by localized surface plasmons on noble metal nanoparticle surfaces, which allows the rapid, non-destructive detection of a wide variety of analytes ranging from small molecules to entire microorganisms.The data obtained from SERS experiments, however, can be difficult to interpret. Spectra are dominated by the chemical species in closest contact with the plasmonic field, and consequently spectra recorded at different points in time from the same sample can exhibit considerable differences. The rich chemical information contained in the spectral data therefore is often not utilized to its full potential or lost altogether.Here, we combine deep convolutional neural networks (CNNs) with SERS to classify series of spectra obtained from mixtures of different sample species (1). We find that CNNs outperform standard chemometric methods, such as principal component analysis, but also support-vector machine classifiers, shallow artificial neural networks, or even trained human experts while requiring minimal data preprocessing, such as background subtraction or smoothing. We discuss optimizing the classifier with regards to network topology, learning strategy, and data augmentation.In addition, we show that we can extract useful information from trained networks. In simple cases, activation maximization spectra, which represent the spectral features that are most relevant to discriminate between different sample classes, can provide a good approximation of the spectra in the original sample. We also train generative adversarial networks (GANs) (2) to generate / extract characteristic spectra from large, more heterogeneous datasets. These spectra can provide additional insight into the composition of the sample and might thus be relevant to guide further experiments.Deep learning and SERS are two powerful techniques that complement each other. The high sensitivity of SERS quickly generates large, but often highly complex spectral datasets, which are in turn well-suited to the analysis by CNNs. Most importantly, deep learning not only excels at classification but can be used as a more general tool to facilitate the interpretation of heterogeneous, seemingly intractable datasets. Given the tremendous attention deep learning has attracted over the past decade, we anticipate that it will play an increasingly prominent role in spectral analysis as well. Yang, J. et al., 2019. Deep learning for vibrational spectral analysis: Recent progress and a practical guide. Analytica chimica acta, 1081, pp.6–17.Odena, A., Olah, C. & Shlens, J., 2016. Conditional Image Synthesis With Auxiliary Classifier GANs. arXiv:1610.09585 [stat.ML] Figure 1

Local and non-local dependency learning and emergence of rule-like representations in speech data by deep convolutional generative adversarial networks

This paper argues that training Generative Adversarial Networks (GANs) on local and non-local dependencies in speech data offers insights into how deep neural networks discretize continuous data and how symbolic-like rule-based morphophonological processes emerge in a deep convolutional architecture. Acquisition of speech has recently been modeled as a dependency between latent space and data generated by GANs in Beguš (2020b), who models learning of a simple local allophonic distribution. We extend this approach to test learning of local and non-local phonological processes that include approximations of morphological processes. We further parallel outputs of the model to results of a behavioral experiment where human subjects are trained on the data used for training the GAN network. Four main conclusions emerge: (i) the networks provide useful information for computational models of speech acquisition even if trained on a comparatively small dataset of an artificial grammar learning experiment; (ii) local processes are easier to learn than non-local processes, which matches both behavioral data in human subjects and typology in the world’s languages. This paper also proposes (iii) how we can actively observe the network’s progress in learning and explore the effect of training steps on learning representations by keeping latent space constant across different training steps. Finally, this paper shows that (iv) the network learns to encode the presence of a prefix with a single latent variable; by interpolating this variable, we can actively observe the operation of a non-local phonological process. The proposed technique for retrieving learning representations has general implications for our understanding of how GANs discretize continuous speech data and suggests that rule-like generalizations in the training data are represented as an interaction between variables in the network’s latent space.

Sparsity Aware Normalization for GANs

Generative adversarial networks (GANs) are known to benefit from regularization or normalization of their critic (discriminator) network during training. In this paper, we analyze the popular spectral normalization scheme, find a significant drawback and introduce sparsity aware normalization (SAN), a new alternative approach for stabilizing GAN training. As opposed to other normalization methods, our approach explicitly accounts for the sparse nature of the feature maps in convolutional networks with ReLU activations. We illustrate the effectiveness of our method through extensive experiments with a variety of network architectures. As we show, sparsity is particularly dominant in critics used for image-to-image translation settings. In these cases our approach improves upon existing methods, in less training epochs and with smaller capacity networks, while requiring practically no computational overhead.

Generative Adversarial Networks (GANs)

Generative Adversarial Networks (GANs) is a novel class of deep generative models that has recently gained significant attention. GANs learn complex and high-dimensional distributions implicitly over images, audio, and data. However, there exist major challenges in training of GANs, i.e., mode collapse, non-convergence, and instability, due to inappropriate design of network architectre, use of objective function, and selection of optimization algorithm. Recently, to address these challenges, several solutions for better design and optimization of GANs have been investigated based on techniques of re-engineered network architectures, new objective functions, and alternative optimization algorithms. To the best of our knowledge, there is no existing survey that has particularly focused on the broad and systematic developments of these solutions. In this study, we perform a comprehensive survey of the advancements in GANs design and optimization solutions proposed to handle GANs challenges. We first identify key research issues within each design and optimization technique and then propose a new taxonomy to structure solutions by key research issues. In accordance with the taxonomy, we provide a detailed discussion on different GANs variants proposed within each solution and their relationships. Finally, based on the insights gained, we present promising research directions in this rapidly growing field.

Synthesis of glioma histopathology images using generative adversarial networks

Deep learning, a subset of artificial intelligence, has shown great potential in several recent applications to pathology. These have mainly involved the use of classifiers to diagnose disease, while generative modelling techniques have been less frequently used. Generative adversarial networks (GANs) are a type of deep learning model that has been used to synthesize realistic images in a range of domains, both general purpose and medical. In the GAN framework, a generator network is trained to synthesize fake images, while a dueling discriminator network aims to distinguish between the fake images and a set of real training images. As GAN training progresses, the generator network ideally learns the important features of a dataset, allowing it to create images that the discriminator cannot distinguish from the real ones. We report on our use of GANs to synthesize high resolution, realistic histopathology images of gliomas. The well- known Progressive GAN framework was trained on a set of image patches extracted from digital slides in the Cancer Genome Atlas repository, and was able to generate fake images that were visually indistinguishable from the real training images. Generative modelling in pathology has numerous potential applications, including dataset augmentation for training deep learning classifiers, image processing, and expanding educational material.LEARNING OBJECTIVESThis presentation will enable the learner to: 1.Explain basic principles of generative modelling in deep learning.2.Discuss applications of deep learning to neuropathology image synthesis.

Assessment of Generative Adversarial Networks for Synthetic Anterior Segment Optical Coherence Tomography Images in Closed-Angle Detection

PurposeTo develop generative adversarial networks (GANs) that synthesize realistic anterior segment optical coherence tomography (AS-OCT) images and evaluate deep learning (DL) models that are trained on real and synthetic datasets for detecting angle closure.MethodsThe GAN architecture was adopted and trained on the dataset with AS-OCT images collected from the Joint Shantou International Eye Center of Shantou University and the Chinese University of Hong Kong, synthesizing open- and closed-angle AS-OCT images. A visual Turing test with two glaucoma specialists was performed to assess the image quality of real and synthetic images. DL models, trained on either real or synthetic datasets, were developed. Using the clinicians’ grading of the AS-OCT images as the reference standard, we compared the diagnostic performance of open-angle vs. closed-angle detection of DL models and the AS-OCT parameter, defined as a trabecular-iris space area 750 µm anterior to the scleral spur (TISA750), in a small independent validation dataset.ResultsThe GAN training included 28,643 AS-OCT anterior chamber angle (ACA) images. The real and synthetic datasets for DL model training have an equal distribution of open- and closed-angle images (all with 10,000 images each). The independent validation dataset included 238 open-angle and 243 closed-angle AS-OCT ACA images. The image quality of real versus synthetic AS-OCT images was similar, as assessed by the two glaucoma specialists, except for the scleral spur visibility. For the independent validation dataset, both DL models achieved higher areas under the curve compared with TISA750. Two DL models had areas under the curve of 0.97 (95% confidence interval, 0.96–0.99) and 0.94 (95% confidence interval, 0.92–0.96).ConclusionsThe GAN synthetic AS-OCT images appeared to be of good quality, according to the glaucoma specialists. The DL models, trained on all-synthetic AS-OCT images, can achieve high diagnostic performance.Translational RelevanceThe GANs can generate realistic AS-OCT images, which can also be used to train DL models.

EMOCGAN: a novel evolutionary multiobjective cyclic generative adversarial network and its application to unpaired image translation

Generative adversarial networks (GANs) have been accepted as powerful models in the field of computer vision, speech and language processing, etc. However, a major concern regarding GANs is the requirement of paired images for image-to-image translation, which is not always possible in the case of real-world applications. Moreover, they also suffer from training instability as well as mode collapse problem. These concerns remain open challenging issues for GANs and become more complex in the case of cyclic GAN. Motivated by evolutionary GAN, we hereby propose a novel evolutionary multiobjective cyclic GAN (EMOCGAN) to address the above stated challenges related to cyclic GAN training for the image-to-image translation. In this work, we have also introduced a new approach for model training by integrating the concept of evolutionary computation, multiobjective optimization, cyclic GAN along with different selection mechanisms. To overcome local optima stagnation, metropolis acceptance criteria and Pareto-based selection on two scores (objective functions) are utilized. Evolutionary concepts in training helped to address the instability and mode collapse problems. Extensive experiments on real-world image datasets show that the EMOCGAN outperforms state-of-the-art method in terms of visually realistic appearance and retaining background information as well as salient objects. Quantitative comparisons of our EMOCGAN with the cyclic GAN also show significantly better scores obtained by our model, as indicated by structural similarity index (SSIM) and universal quality index (UQI). The model demonstrated best efficacy in terms of SSIM on Apple $$\leftrightarrow$$ Orange, while it shows higher UQI values for Monet $$\leftrightarrow$$ Picture and Summer $$\leftrightarrow$$ Winter datasets.

High-end equipment data desensitization method based on improved Stackelberg GAN

High-end equipment refers to a type of technical equipment with high technical content, large capital investment, and long development cycle. Therefore, high-end equipment data has extraordinary significance and its desensitization is an urgent problem in data analysis. Traditional data desensitization principles are processing original data such as substitution and adding noise. These methods may not only damage data correlation information, but also result in data disclosure and high computing cost. Given the aforementioned reasons, the study proposes a high-end equipment data desensitization method based on improved Stackelberg Generative Adversarial Networks (GAN). When compared with the normal GAN, the structure proposed in the study includes more generators and discriminators. By inputting the original data, the trained GAN can output indistinguishable data from the original data which helps data mining and also ensures the privacy of data. We experimented on two datasets: optimal improvement was determined by Gaussian dataset experiments, i.e. Stackelberg GAN with eight discriminators. The second experiment results on real-world datasets proved that the 8-discriminator Stackelberg GAN better fits the original data and significantly aids data desensitization.

Geological Facies modeling based on progressive growing of generative adversarial networks (GANs)

Geological facies modeling has long been studied to predict subsurface resources. In recent years, generative adversarial networks (GANs) have been used as a new method for geological facies modeling with surprisingly good results. However, in conventional GANs, all layers are trained concurrently, and the scales of the geological features are not considered. In this study, we propose to train GANs for facies modeling based on a new training process, namely progressive growing of GANs or a progressive training process. In the progressive training process, GANs are trained layer by layer, and geological features are learned from coarse scales to fine scales. We also train a GAN in the conventional training process, and compare the conventionally trained generator with the progressively trained generator based on visual inspection, multi-scale sliced Wasserstein distance (MS-SWD), multi-dimensional scaling (MDS) plot visualization, facies proportion, variogram, and channel sinuosity, width, and length metrics. The MS-SWD reveals realism and diversity of the generated facies models, and is combined with MDS to visualize the relationship between the distributions of the generated and training facies models. The conventionally and progressively trained generators both have very good performances on all metrics. The progressively trained generator behaves especially better than the conventionally trained generator on the MS-SWD, MDS plots, and the necessary training time. The training time for the progressively trained generator can be as small as 39% of that for the conventionally trained generator. This study demonstrates the superiority of the progressive training process over the conventional one in geological facies modeling, and provides a better option for future GAN-related researches.

Supervised deep convolutional generative adversarial networks

Generative adversarial networks (GANs) are one of the most important generative network models. Using real samples, the GAN generates fake samples from the noise given as input to the network. This popular network model, which has recently emerged and consists of several variants, has different applications in many areas. Some of the studies have been implemented by applying GANs to real-world problems. Another part is aimed at improving the performance of GANs or eliminating the disadvantages observed over time. One of these studies is DCGAN. The importance of DCGAN is that it contributes significantly to balancing GAN training with its convolutional architecture. GAN and naturally DCGAN have an unsupervised network structure. While the network is informed that the samples given as input are real or fake, the category label information is not given to the network. In the present study, a method is proposed, which enables creating a supervised network structure when using multi-categories data set with DCGAN structure. The proposed method ensures that noise can be given a category label and this generated category label information can be used in the output layer. This method, which is easily applicable and effective, is named as Supervised DCGAN (SDCGAN).

Crowd jumping load simulation with generative adversarial networks

To mathematically represent crowd jumping loads, the features of the jumping load of each person, including pulse curve patterns, pulse interval sequences, and pulse energy sequences are considered. These features are essentially highdimensional random variables. However, they have to be represented in a practically simplified model due to the lack of mathematical tools. The recently emerged generative adversarial networks (GANs) can model high-dimensional random variables well, as demonstrated in image synthesis and text generation. Therefore, this study adopts GANs as a new method for modelling crowd jumping loads. Conditional GANs (CGANs) combined with Wasserstein GANs with gradient penalty (WGANs*—GP) are used in pulse curve pattern modelling, where a multi-layer perceptron and convolutional neural network are selected as the discriminator and generator, respectively. For the pulse energy sequence and pulse interval sequence modelling, similar GANs are used, where recurrent neural networks are selected as both the generator and discriminator. Finally, crowd jumping loads can be simulated by connected the pulse samples based on the pulse energy sequence samples and interval sequence samples, generated by the three proposed GANs. The experimental individual and crowd jumping load records are utilized in training GANs to ensure their output can simulate real load records well. Finally, the feasibility of the proposed GANs was verified by comparing the measured structural responses of an existing floor to the predicted structural responses.

Generative Adversarial Networks for Data Augmentation in Structural Adhesive Inspection

The technological advances brought forth by the Industry 4.0 paradigm have renewed the disruptive potential of artificial intelligence in the manufacturing sector, building the data-driven era on top of concepts such as Cyber–Physical Systems and the Internet of Things. However, data availability remains a major challenge for the success of these solutions, particularly concerning those based on deep learning approaches. Specifically in the quality inspection of structural adhesive applications, found commonly in the automotive domain, defect data with sufficient variety, volume and quality is generally costly, time-consuming and inefficient to obtain, jeopardizing the viability of such approaches due to data scarcity. To mitigate this, we propose a novel approach to generate synthetic training data for this application, leveraging recent breakthroughs in training generative adversarial networks with limited data to improve the performance of automated inspection methods based on deep learning, especially for imbalanced datasets. Preliminary results in a real automotive pilot cell show promise in this direction, with the approach being able to generate realistic adhesive bead images and consequently object detection models showing improved mean average precision at different thresholds when trained on the augmented dataset. For reproducibility purposes, the model weights, configurations and data encompassed in this study are made publicly available.

A Stopping Criterion for the Training Process of the Specific Signal Generator

Mathematical description of a complex signal is very important in engineering but nearly impossible in many occasions. The emergence of the Generative Adversarial Network (GAN) shows the possibility to train a single neural network to be a Specific Signal Generator (SSG), which is only controlled by a random vector with several elements. However, there is no explicit criterion for the GAN training process to stop, and in real applications the training always stops after a certain big iteration. In this paper, a serious issue was discussed during the process to use GAN as a SSG. And, an explicit criterion for the GAN as a SSG to stop the training process were proposed. Several experiments were carried out to illustrate the issues mentioned above and the effectiveness of the stopping criterion proposed in this paper.

Robotic Musicianship Based on Least Squares and Sequence Generative Adversarial Networks

Robotic musicianship research aims to the configuration of robots can analyze, reason, and generate music autonomous. The goal of this research is to achieve the inspiring and meaningful musical interactions between humans and artificially creative robots. This research presents a new model of automatic music generation which is based on least squares and generative adversarial networks (GANs). This research specifies classical piano as the music source and uses the sensors as the context-awareness technology to sense and receive the input audio in human-robot interaction. Therefore, this research applies sequence generation adversarial network (SeqGAN) techniques that are better able to address discrete issues in generating samples of classical piano melodies and datasets. Modifying the SeqGAN approach, this research presented the Least Squares SeqGAN (LS-SeqGAN) method to create melody units on different chords and generates a set of music pieces as testing dataset. In this research, we implement the original method and use the least squares method to stabilize the training of GANs. The performance evaluation shows that proposed LS-SeqGAN method can fulfill the need both of music quality and creativity. It offers a robust infrastructure for the human-robotic interaction that can be used to promote the related robotic applications.

Semantic Segmentation of High Resolution Satellite Imagery using Generative Adversarial Networks with Progressive Growing

ABSTRACT With increase in urbanization and Earth Sciences research into urban areas, the need to quickly and accurately segment urban rooftop maps has never been greater. Current machine learning techniques struggle to produce high accuracy maps in dense urban zones where there is high image noise and foot print overlap. In this paper, we evaluate a training methodology for pixel-wise segmentation for high-resolution satellite imagery using progressive growing of generative adversarial networks as a solution. We apply our model to segmenting building rooftops and compare these results to conventional methods for rooftop segmentation. We evaluate our approach using the SpaceNet version 2 and xView datasets. Our experiments show that for SpaceNet, progressive Generative Adversarial Network (GAN) training achieved a test accuracy of 93% compared to 89% for traditional GAN training and 87% for U-Net architecture, while for xView, we achieved 71% accuracy using progressive GAN training compared to 69% through traditional GAN training and 65% using U-Net.

Challenges and Corresponding Solutions of Generative Adversarial Networks (GANs): A Survey Study

Generative Adversarial Networks (GANs) are an innovative class of deep learning generative model that has been popular among academics recently. GANs are able to learn distributions on complex high-dimensional data which made it efficient in images and audio processing. Nevertheless, in the training of GANs, some major challenges exist namely mode collapse, non-convergence, and instability. In recent years, in order to overcome these challenges, researchers have proposed many variants of GANs by redesigning network architecture, changing the form of objective functions, and altering optimization algorithms. In this research, we conducted a comprehensive investigation on the progress of GANs design and optimization solutions. Finally, according to the classification method, we provided a problem-solving structure to solve conquer the GANs training challenges.

Automatic detection of annual rings and pith location along Norway spruce timber boards using conditional adversarial networks

In the woodworking industry, detection of annual rings and location of pith in relation to timber board cross sections, and how these properties vary in the longitudinal direction of boards, is relevant for many purposes such as assessment of shape stability and prediction of mechanical properties of timber. The current work aims at developing a fast, accurate and operationally simple deep learning-based algorithm for automatic detection of surface growth rings and pith location along knot-free clear wood sections of Norway spruce boards. First, individual surface growth rings that are visible along the four longitudinal sides of the scanned boards are detected using trained conditional generative adversarial networks (cGANs). Then, pith locations are determined, on the basis of the detected growth rings, by using a trained multilayer perceptron (MLP) artificial neural network. The proposed algorithm was solely based on raw images of board surfaces obtained from optical scanning and applied to a total of 104 Norway spruce boards with nominal dimensions of 45times 145times 4500,hbox {mm}^{3}. The results show that optical scanners and the proposed automatic method allow for accurate and fast detection of individual surface growth rings and pith location along boards. For boards with the pith located within the cross section, median errors of 1.4 mm and 2.9 mm, in the x- and y-direction, respectively, were obtained. For a sample of boards with the pith located outside the board cross section in most positions along the board, the median discrepancy between automatically estimated and manually determined pith locations was 3.9 mm and 5.4 mm in the x- and y-direction, respectively.

Using GANs with adaptive training data to search for new molecules

The process of drug discovery involves a search over the space of all possible chemical compounds. Generative Adversarial Networks (GANs) provide a valuable tool towards exploring chemical space and optimizing known compounds for a desired functionality. Standard approaches to training GANs, however, can result in mode collapse, in which the generator primarily produces samples closely related to a small subset of the training data. In contrast, the search for novel compounds necessitates exploration beyond the original data. Here, we present an approach to training GANs that promotes incremental exploration and limits the impacts of mode collapse using concepts from Genetic Algorithms. In our approach, valid samples from the generator are used to replace samples from the training data. We consider both random and guided selection along with recombination during replacement. By tracking the number of novel compounds produced during training, we show that updates to the training data drastically outperform the traditional approach, increasing potential applications for GANs in drug discovery.

Improvement of Technology of Passenger Intermodal Transportation with Involvement of Railway Transport in the Conditions of Tourism Development

Purpose. The main purpose of the authors is to define and methodically substantiate the ways to increase the efficiency of intermodal passenger transportations with the involvement of high-speed trains as an auxiliary mode of transport in terms of sea and river tourism. Methodology. In the process of research the following was used: the method of factor analysis – to determine the factors influencing the attractiveness of tourist travel using high-speed trains as ancillary transport; method of skipping stops – to increase the efficiency of using high-speed trains as an auxiliary mode of transport when making tourist trips; methods of construction and training of generative-adversarial networks for the formation of model of passenger flows forecasting, on the basis of historical data of multivariate time series; method of genetic algorithms – to optimize the model of mixed-integer programming, which allows obtaining the optimal scheme of high-speed trains on the line. Findings. In order to preserve the attractiveness of tourist travels and increase the route speed of trains, it is proposed to improve the technology of planning their work based on the method of skipping stops. A mathematical model of mixed-integer programming has been formed, which simultaneously provides the attractiveness of tourist travel and profitability for railway operators. To prepare the initial data, a method for forecasting passenger flows based on multivariate time series has been developed. The optimization procedure of the generated model was implemented in the form of software in the Matlab language. Originality. The method of skipping stops, which was first used to improve the technology of intermodal passenger traffic, was further developed in the work. An original method for predicting passenger flows based on multivariate time series using a modern model of generative-competitive neural networks is proposed. Practical value. The obtained results are aimed at improving the methodological approaches to the formation of modern technologies of intermodal passenger transportation and the realization of the potential of high-speed rail transportations as a basis for the comprehensive development of tourism.