Boundary Equilibrium Generative Adversarial Network Research Articles

Limited agricultural spectral dataset expansion based on generative adversarial networks

With the rise of deep learning, the combination of spectroscopy analysis techniques and deep learning methods has been extensively utilized in the field of agriculture, such as the detection of crop diseases, soil analysis, and crop quality assessment. Unfortunately, it is often difficult to obtain enough spectral samples that can be used for deep learning because of environmental constraints, equipment limitations, and labour costs. To address this issue, we propose a spectral sample augmentation technique based on the K-condition boundary equilibrium generative adversarial networks (KC-BEGAN). First, a stable boundary equilibrium generative adversarial network (BEGAN) model is constructed, and the KC-BEGAN model is built by incorporating chemical property labels, multiscale gradient information, and the k-nearest neighbour algorithm. The goal is to enrich complete spectral samples with chemical properties. Second, we compare the differences between generated samples and real samples using methods such as t-distributed stochastic neighbour embedding, Mahalanobis distance, F test, and maximum mean discrepancy. Following data augmentation by the KC-BEGAN model, the R2 on the test set for traditional regression models (PLSR, SVR, RR, PCR) and deep regression models (Inception-ResNet, Inception, 1D-CNN) improved by 4.9%, 0.4%, 2.5%, 4.4%, 3.6%, 4.4%, and 6.8% respectively. Furthermore, this study replaces the GAN module in the KC-BEGAN model with Diffusion-GAN and conducts experiments following the same procedure to evaluate the feasibility of other GAN models in augmenting labelled near-infrared spectral samples. The research results indicate that the spectral samples generated by the KC-BEGAN model are reliable and can meet the expansion requirements of small-scale spectral sample sets. Simultaneously, replacing the GAN module in the KC-BEGAN model with other GAN models is also feasible, suggesting that the continuous label marking method proposed in this study is indeed effective.

AE-BEGAN based Synthetic Data Augmentation for Sample-Limited High-Dimensional Problems with Application to NIR Spectral Data

Synthetic data augmentation holds substantial research and application value in scenarios characterized by limited samples and high dimensions. It enhances the analytical ability and efficiency of spectral analysis models. This paper proposes Autoencoder-Combined Boundary Equilibrium Generative Adversarial Networks (AE-BEGAN) as a new method for augmenting synthetic data in scenarios with limited samples and high dimensions, with a specific emphasis on near-infrared (NIR) spectral data. The spectral data first undergoes preprocessing procedures that encompass advanced noise reduction algorithms and techniques for removing abnormal samples, guaranteeing elimination of unwanted disturbances and outliers. Then, the pre-processed data is utilized to train the AE-BEGAN model, which generates augmented synthetic samples. Finally, real NIR spectral data obtained from lubricant samples exhibiting different water contents were employed to validate and test the performance of the model. The experimental results demonstrate that the AE-BEGAN model outperforms other GANs in generating synthetic data of high quality and diversity, as quantified by two evaluation metrics, α-Precision and β-Recall with scores of approximately 0.86 and 0.28, respectively. The application case study confirms that the AE-BEGAN model exhibits the capability to generate derived NIR spectra and expand the number of spectra in scenarios with limited samples and high dimensions.

From Beginning to BEGANing: Role of Adversarial Learning in Reshaping Generative Models

Deep generative models, such as deep Boltzmann machines, focused on models that provided parametric specification of probability distribution functions. Such models are trained by maximizing intractable likelihood functions, and therefore require numerous approximations to the likelihood gradient. This underlying difficulty led to the development of generative machines such as generative stochastic networks, which do not represent the likelihood functions explicitly, like the earlier models, but are trained with exact backpropagation rather than the numerous approximations. These models use piecewise linear units that are having well behaved gradients. Generative machines were further extended with the introduction of an associative adversarial network leading to the generative adversarial nets (GANs) model by Goodfellow in 2014. The estimations in GANs process two multilayer perceptrons, called the generative model and the discriminative model. These are learned jointly by alternating the training of the two models, using game theory principles. However, GAN has many difficulties, including: the difficulty of training the models; criticality in the selection of hyper-parameters; difficulty in the control of generated samples; balancing the convergence of the discriminator and generator; and the problem of modal collapse. Since its inception, efforts have been made to tackle these issues one at a time or in multiples at several stages by many researchers. However, most of these have been handled efficiently in the boundary equilibrium generative adversarial networks (BEGAN) model introduced by Berthelot et al. in 2017. In this work we presented the advent of adversarial networks, starting with the history behind the models and c developments done on GANs until the BEGAN model was introduced. Since some time has elapsed since the proposal of BEGAN, we provided an up-to-date study, as well as future directions for various aspects of adversarial learning.

Controllable Wireless Spoofing Attack Based on Conditional BEGAN and Auxiliary Channel Sensing

This paper investigates how to build a controllable wireless spoofing attack launch framework that is driven by fundamental channel modeling and practical wireless datasets. First, we propose a wireless spoofing attack scheme against the defense mechanism with adversarial deep learning. To obtain channel characteristics and facilitate offline training of the attack model, auxiliary channel sensing is proposed with fundamental channel modeling. Based on these, a conditional boundary equilibrium generative adversarial network (CBEGAN) is designed with adversarial autoencoder (AAE), which takes true labels of signals and channel characteristics as conditions and enables the generation of controllable spoofing signals to fool the protected legitimate classifier. We verify the performance of the proposed spoofing attack scheme with CBEGAN and channel sensing by using wireless datasets, which contain signal data of multiple emitters and modulation types. Results show that the proposed scheme outperforms random attack, replay attack, and the recent attack scheme based on generative adversarial network (GAN) when a single legitimate emitter sends a fixed modulation type. It is also shown that the average attack success probability of the proposed CBEGAN attack model can reach more than 80% while mimicking multiple emitters and modulation types. The performance of the proposed scheme on different channel conditions including signal-to-noise ratio (SNR) and K-factor of the Rician fading channel is evaluated.

Realistic image synthesis of COVID-19 chest X-rays using depthwise boundary equilibrium generative adversarial networks

<span>Researchers in various related fields research preventing and controlling the spread of the coronavirus disease (COVID-19) virus. The spread of the COVID-19 is increasing exponentially and infecting humans massively. Preliminary detection can be observed by looking at abnormal conditions in the airways, thus allowing the entry of the virus into the patient's respiratory tract, which can be represented using computer tomography (CT) scan and chest X-ray (CXR) imaging. Particular deep learning approaches have been developed to classify COVID-19 CT or CXR images such as convolutional neural network (CNN), and deep convolutional neural network (DCNN). However, COVID-19 CXR dataset was measly opened and accessed. Particular deep learning method performance can be improved by augmenting the dataset amount. Therefore, the COVID-19 CXR dataset was possibly augmented by generating the synthetic image. This study discusses a fast and real-like image synthesis approach, namely depthwise boundary equilibrium generative adversarial network (DepthwiseBEGAN). DepthwiseBEGAN was reduced memory load 70.11% in training processes compared to the conventional BEGAN. DepthwiseBEGAN synthetic images were inspected by measuring the Fréchet inception distance (FID) score with the real-to-real score equal to 4.3866 and real-to-fake score equal to 4.4674. Moreover, generated DepthwiseBEGAN synthetic images improve 22.59% accuracy of conventional CNN models.</span>

Face Image Generation and Enhancement Using Conditional Generative Adversarial Network

The accuracy and speed of a single image super-resolution using a convolutional neural network is often a problem in improving finer texture details when using large enhancement factors. Some recent studies have focused on minimal mean square error, resulting in a high peak signal to noise ratio. Generally, although the peak signal to noise ratio has a high value, the output image is less detailed. This shows that the determination of super-resolution is not optimal. Conditional Generative Adversarial Network based on Boundary Equilibrium Generative Adversarial Network, by combining Mean Square Error Loss and GAN Loss as a loss function to optimize the super-resolution model and produce super-resolution images. Also, the generator network is designed with skip connection architecture to increase convergence speed and strengthen feature distribution. Image quality value parameters used in this study are Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM). The results showed the highest image quality values using dataset validation were 26.55 for PSNR values and 0.93 for SSIM values. The highest image quality values using the testing dataset are 24.56 for the PSNR value and 0.91 for the SSIM value.

Parallel data free singing voice conversion with cycle-consistent BEGAN

Singing voice conversion (SVC) is the method to modify the timbre of the source singer with the target singer while retaining the linguistic content. Recent studies are mainly focused with non-parallel training data, since it is difficult to obtain parallel training data in real life applications. In this paper, a parallel data-free SVC technique is proposed using Cycle-consistent Boundary Equilibrium Generative Adversarial Networks (CycleBEGAN) with gated convolutional neural networks (CNNs) and an identity-mapping loss. CycleBEGAN allows the learning of data distribution using both adversarial loss and cycle-consistency loss. Gated CNN and identity mapping loss ensures the sequential and hierarchical structures of information and preservation of linguistic information. This technique produces high quality converted singing voice without any time-alignment procedures and requires only a small amount of training data.

Intelligent Generation Method of Innovative Structures Based on Topology Optimization and Deep Learning.

Computer-aided design has been widely used in structural calculation and analysis, but there are still challenges in generating innovative structures intelligently. Aiming at this issue, a new method was proposed to realize the intelligent generation of innovative structures based on topology optimization and deep learning. Firstly, a large number of structural models obtained from topology optimization under different optimization parameters were extracted to produce the training set images, and the training set labels were defined as the corresponding load cases. Then, the boundary equilibrium generative adversarial networks (BEGAN) deep learning algorithm was applied to generate numerous innovative structures. Finally, the generated structures were evaluated by a series of evaluation indexes, including innovation, aesthetics, machinability, and mechanical performance. Combined with two engineering cases, the application process of the above method is described here in detail. Furthermore, the 3D reconstruction and additive manufacturing techniques were applied to manufacture the structural models. The research results showed that the proposed approach of structural generation based on topology optimization and deep learning is feasible, and can not only generate innovative structures but also optimize the material consumption and mechanical performance further.

Generative Adversarial Networks in Medical Image Processing.

The emergence of generative adversarial networks (GANs) has provided new technology and framework for the application of medical images. Specifically, a GAN requires little to no labeled data to obtain high-quality data that can be generated through competition between the generator and discriminator networks. Therefore, GANs are rapidly proving to be a state-of-the-art foundation, achieving enhanced performances in various medical applications. In this article, we introduce the principles of GANs and their various variants, deep convolutional GAN, conditional GAN, Wasserstein GAN, Info-GAN, boundary equilibrium GAN, and cycle-GAN. All various GANs have found success in medical imaging tasks, including medical image enhancement, segmentation, classification, reconstruction, and synthesis. Furthermore, we summarize the data processing methods and evaluation indicators. Finally, we note the limitations of existing methods and the existing challenges that need to be addressed in this field. Although GANs are in the initial stage of development in medical image processing, it will have a great prospect in the future.

Generating adversarial examples with elastic-net regularized boundary equilibrium generative adversarial network

To improve the attack success rate and image perceptual quality of adversarial examples against deep neural networks(DNNs), we propose a new Generative Adversarial Network (GAN) based attacker, named Elastic-net Regularized Boundary Equilibrium Generative Adversarial Network(ERBEGAN). Recent studies have shown that DNNs are easy to attack by adversarial examples(AEs) where benign images with small-magnitude perturbations mislead DNNs to incorrect results. A number of methods are proposed to generate AEs, but how to generate them with high attack success rate and perceptual quality needs more effort. Most attackers generate AEs by restricting L2-norm and L∞-norm of adversarial perturbations. However, very few works have been developed on L1 distortion matrix which encourages sparsity in the perturbation. In this paper, we penalize both L2-norm and L1-norm of perturbation as Elastic-Net regularization to improve the diversity and robustness of AEs. We further improve GAN by minimizing the additional pixel-wise loss derived from the Wasserstein distance between benign and adversarial auto-encoder loss distributions. Extensive experiments and visualizations on several datasets show that the proposed ERBEGAN can yield higher attack success rates than the state-of-the-art GAN-based attacker AdvGAN under the semi-whitebox and black-box attack settings. Besides, our method efficiently generates diverse adversarial examples that are more perceptually realistic.

Pine Cone Detection Using Boundary Equilibrium Generative Adversarial Networks and Improved YOLOv3 Model.

The real-time detection of pine cones in Korean pine forests is not only the data basis for the mechanized picking of pine cones, but also one of the important methods for evaluating the yield of Korean pine forests. In recent years, there has been a certain number of detection accuracy for image processing of fruits in trees using deep-learning methods, but the overall performance of these methods has not been satisfactory, and they have never been used in the detection of pine cones. In this paper, a pine cone detection method based on Boundary Equilibrium Generative Adversarial Networks (BEGAN) and You Only Look Once (YOLO) v3 mode is proposed to solve the problems of insufficient data set, inaccurate detection result and slow detection speed. First, we use traditional image augmentation technology and generative adversarial network BEGAN to implement data augmentation. Second, we introduced a densely connected network (DenseNet) structure in the backbone network of YOLOv3. Third, we expanded the detection scale of YOLOv3, and optimized the loss function of YOLOv3 using the Distance-IoU (DIoU) algorithm. Finally, we conducted a comparative experiment. The experimental results show that the performance of the model can be effectively improved by using BEGAN for data augmentation. Under same conditions, the improved YOLOv3 model is better than the Single Shot MultiBox Detector (SSD), the faster-regions with convolutional neural network (Faster R-CNN) and the original YOLOv3 model. The detection accuracy reaches 95.3%, and the detection efficiency is 37.8% higher than that of the original YOLOv3.

Generation of High-resolution Lung Computed Tomography Images using Generative Adversarial Networks.

To deal with the limiting data in training for new deep learning modules, we purpose a method to generate high-resolution medical images by implementing generative adversarial networks (GAN) models. Firstly, the boundary equilibrium generative adversarial networks model was used to generate the whole lung computed tomography images. Image inpainting was then integrated to generate the delicate details of the lung part by dividing into a coarse network and a refinement network to inpaint more completed and intricate details. With this method, we aim to increase the amount of high-resolution medical images for future applications in deep learning.

Synthetic spectra generated by boundary equilibrium generative adversarial networks and their applications with consensus algorithms.

One of the major restrictions in spectroscopic analysis is the limited number of calibrations, especially for biological samples. Meanwhile, there is a lack of effective algorithms to simulate synthetic spectra from the real spectra of limited samples. Thus in this work, a boundary equilibrium generative adversarial network (BEGAN) was proposed to automatically generate synthetic spectra and successfully produce spectra from two datasets. Then, the impact of the diversity ratio was estimated in the aspect of the quality and diversity of the generated spectra by BEGAN, and a negative correlation was found between quality and diversity. Finally, these synthetic spectra are applied in a consensus algorithm named creating diversity partial least squares (CDPLS) to replenish virtual samples in every iteration. Results show that the synthetic spectra generated by BEGAN are of high quality and improve the predictive performance of CDPLS. It can concluded that BEGAN has the potential to generate derived homologous spectra and expand the number of spectra in some small sample sets.

Research on Spinal Canal Generation Method based on Vertebral Foramina Inpainting of Spinal CT Images by using BEGAN

Abstract Spinal surgery is of high risk due to the possibility of neurologic damage, which may cause life-threatening sequelae. Although the emerging robotic-assisted spinal surgery provides better accuracy compared with traditional surgery, the construction of boundary constraints around the spinal canal for safety in surgery is still required. The establishment of a three-dimensional (3D) model of the spinal canal during preoperative preparation can facilitate the generation of surgical boundary constraints. This article presents a novel framework for spinal canal generation based on spinal CT image inpainting by using the boundary equilibrium generative adversarial network (BEGAN). First, U-net is used to simplify the image features and then ResNet50 is applied to classify the vertebral foramen features and mark the area to be restored. Finally, BEGAN generates the target features to complete the vertebral foramina inpainting for the generation of the spinal canal. The experimental results show that the average accuracies (Mean Intersection over Union) of the vertebral foramina and spine inpainting are 0.9396 and 0.9332, respectively, and the accuracy of image inpainting decreases with increase in the inpainting area. The proposed method can accurately generate the vertebral contours and complete the 3D reconstruction of the spinal canal.

BEGAN v3: Avoiding Mode Collapse in GANs Using Variational Inference

In the field of deep learning, the generative model did not attract much attention until GANs (generative adversarial networks) appeared. In 2014, Google’s Ian Goodfellow proposed a generative model called GANs. GANs use different structures and objective functions from the existing generative model. For example, GANs use two neural networks: a generator that creates a realistic image, and a discriminator that distinguishes whether the input is real or synthetic. If there are no problems in the training process, GANs can generate images that are difficult even for experts to distinguish in terms of authenticity. Currently, GANs are the most researched subject in the field of computer vision, which deals with the technology of image style translation, synthesis, and generation, and various models have been unveiled. The issues raised are also improving one by one. In image synthesis, BEGAN (Boundary Equilibrium Generative Adversarial Network), which outperforms the previously announced GANs, learns the latent space of the image, while balancing the generator and discriminator. Nonetheless, BEGAN also has a mode collapse wherein the generator generates only a few images or a single one. Although BEGAN-CS (Boundary Equilibrium Generative Adversarial Network with Constrained Space), which was improved in terms of loss function, was introduced, it did not solve the mode collapse. The discriminator structure of BEGAN-CS is AE (AutoEncoder), which cannot create a particularly useful or structured latent space. Compression performance is not good either. In this paper, this characteristic of AE is considered to be related to the occurrence of mode collapse. Thus, we used VAE (Variational AutoEncoder), which added statistical techniques to AE. As a result of the experiment, the proposed model did not cause mode collapse but converged to a better state than BEGAN-CS.

Simplified Fréchet Distance for Generative Adversarial Nets.

We introduce a distance metric between two distributions and propose a Generative Adversarial Network (GAN) model: the Simplified Fréchet distance (SFD) and the Simplified Fréchet GAN (SFGAN). Although the data generated through GANs are similar to real data, GAN often undergoes unstable training due to its adversarial structure. A possible solution to this problem is considering Fréchet distance (FD). However, FD is unfeasible to realize due to its covariance term. SFD overcomes the complexity so that it enables us to realize in networks. The structure of SFGAN is based on the Boundary Equilibrium GAN (BEGAN) while using SFD in loss functions. Experiments are conducted with several datasets, including CelebA and CIFAR-10. The losses and generated samples of SFGAN and BEGAN are compared with several distance metrics. The evidence of mode collapse and/or mode drop does not occur until 3000k steps for SFGAN, while it occurs between 457k and 968k steps for BEGAN. Experimental results show that SFD makes GANs more stable than other distance metrics used in GANs, and SFD compensates for the weakness of models based on BEGAN-based network structure. Based on the experimental results, we can conclude that SFD is more suitable for GAN than other metrics.

Slice-selective learning for Alzheimer's disease classification using a generative adversarial network: a feasibility study of external validation.

The aim of this feasibility study was to use slice selective learning using a Generative Adversarial Network for external validation. We aimed to build a model less sensitive to PET imaging acquisition environment, since differences in environments negatively influence network performance. To investigate the slice performance, each slice evaluation was performed. We trained our model using a 18F-fluorodeoxyglucose ([18F]FDG) PET/CT dataset obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database and tested the model with a Severance Hospital dataset. We applied slice selective learning to reduce computational cost and to extract unbiased features. We extracted features of Alzheimer's disease (AD) and normal cognitive (NC) condition using a Boundary Equilibrium Generative Adversarial Network (BEGAN) for stable convergence. Then, we utilized these features to train a support vector machine (SVM) classifier to distinguish AD from NC. The slice range that covered the posterior cingulate cortex (PCC) using double slices showed the best performance. The accuracy, sensitivity, and specificity of our proposed network was 94.33%, 91.78%, and 97.06% using the Severance dataset and 94.82%, 92.11%, and 97.45% using the ADNI dataset. The performance on the two independent datasets showed no statistical difference (p > 0.05). Moreover, there was a statistical difference in the performance between using two slices and one slice as input (p < 0.05). Our model learned the generalized features of AD and NC for external validation when appropriate slices were selected. This study showed the feasibility of this model with consistent performance when tested using datasets acquired from a variety of image-acquisition environments.

Deep UL2DL: Data-Driven Channel Knowledge Transfer From Uplink to Downlink

To remove the need for signaling overhead of feedback channels for transmitter channel state information (CSI) in Frequency Division Duplexing (FDD), we propose using convolutional neural networks and generative adversarial networks (GANs) to infer the downlink (DL) CSI by observing the uplink (UL) CSI. Our data-driven scheme exploits the fact that both DL and UL channels share the same propagation environment. As such, we extracted the environment information from UL channel response to a latent domain and then transferred the derived environment information from the latent domain to predict the DL channel. To prevent incorrect latent domain and the problem of oversimplistic assumptions, we did not use any specific parametric model and, instead, used data-driven approaches to discover the underlying structure of data without any prior model assumptions. To overcome the challenge of capturing the UL-DL joint distribution, we used a mean square error-based variant of the GAN structure with improved convergence properties called boundary equilibrium GAN. For training and testing we used simulated data of Extended Vehicular-A (EVA) and Extended Typical Urban (ETU) models. Simulation results verified that our methods can accurately infer and predict the downlink CSI from the uplink CSI for different multipath environments.

Unpaired Speech Enhancement by Acoustic and Adversarial Supervision for Speech Recognition

Many speech enhancement methods try to learn the relationship between noisy and clean speech, obtained using an acoustic room simulator. We point out several limitations of enhancement methods relying on clean speech targets; the goal of this work is proposing an alternative learning algorithm, called acoustic and adversarial supervision (AAS). AAS makes the enhanced output both maximizing the likelihood of transcription on the pre-trained acoustic model and having general characteristics of clean speech, which improve generalization on unseen noisy speeches. We employ the connectionist temporal classification and the unpaired conditional boundary equilibrium generative adversarial network as the loss function of AAS. AAS is tested on two datasets including additive noise without and with reverberation, Librispeech + DEMAND and CHiME-4. By visualizing the enhanced speech with different loss combinations, we demonstrate the role of each supervision. AAS achieves a lower word error rate than other state-of-the-art methods using the clean speech target in both datasets.

Information-Based Boundary Equilibrium Generative Adversarial Networks with Interpretable Representation Learning

This paper describes a new image generation algorithm based on generative adversarial network. With an information-theoretic extension to the autoencoder-based discriminator, this new algorithm is able to learn interpretable representations from the input images. Our model not only adversarially minimizes the Wasserstein distance-based losses of the discriminator and generator but also maximizes the mutual information between small subset of the latent variables and the observation. We also train our model with proportional control theory to keep the equilibrium between the discriminator and the generator balanced, and as a result, our generative adversarial network can mitigate the convergence problem. Through the experiments on real images, we validate our proposed method, which can manipulate the generated images as desired by controlling the latent codes of input variables. In addition, the visual qualities of produced images are effectively maintained, and the model can stably converge to the equilibrium. However, our model has a difficulty in learning disentangling factors because our model does not regularize the independence between the interpretable factors. Therefore, in the future, we will develop a generative model that can learn disentangling factors.