STL-10 Datasets Research Articles

Multi-branch evolutionary generative adversarial networks based on covariance crossover operators

Generative Adversarial Networks (GANs) have brought surprises in image generation and processing, particularly excelling in alleviating the problem of sparse samples in few-shot learning. Evolutionary Generative Adversarial Networks (EGANs) aim to convert the generation task into an optimization problem by integrating multiple loss functions to minimize the gap between the generated distribution and the data distribution. However, EGANs’ heavy reliance on mutation operations introduces excessive randomness, resulting in unstable generator updates and affecting the diversity of generated samples. This leads to challenges such as model collapse and gradient vanishing. In this paper, inspired by human grafting mechanism, we propose a novel Multi-branch Evolutionary Generative Adversarial Network (ME-GAN) based covariance crossover operator, which includes two distinct branches – Evolutionary GAN (E-GAN) and Conditional Evolutionary GAN (CE-GAN). Specifically, the E-GAN branch involves a mutation process to introduce randomness to the generated samples, while the CE-GAN branch aims to provide more realistic and diversify generated samples through conditional enhancement and relevant mutation processes. We propose a crossover operation that utilizes the covariance similarity metric to transfer different feature attributes between offspring of different generations, thereby generating a diverse sample. Extensive experiments on CIFAR-10, STL-10, and CelebA datasets demonstrate the effectiveness and superiority of the proposed framework.

ECCT: Efficient Contrastive Clustering via Pseudo-Siamese Vision Transformer and Multi-view Augmentation

Image clustering aims to divide a set of unlabeled images into multiple clusters. Recently, clustering methods based on contrastive learning have attracted much attention due to their ability to learn discriminative feature representations. Nevertheless, existing clustering algorithms face challenges in capturing global information and preserving semantic continuity. Additionally, these methods often exhibit relatively singular feature distributions, limiting the full potential of contrastive learning in clustering. These problems can have a negative impact on the performance of image clustering. To address the above problems, we propose a deep clustering framework termed Efficient Contrastive Clustering via Pseudo-Siamese Vision Transformer and Multi-view Augmentation (ECCT). The core idea is to introduce Vision Transformer (ViT) to provide the global view, and improve it with Hilbert Patch Embedding (HPE) module to construct a new ViT branch. Finally, we fuse the features extracted from the two ViT branches to obtain both global view and semantic coherence. In addition, we employ multi-view random aggressive augmentation to broaden the feature distribution, enabling the model to learn more comprehensive and richer contrastive features. Our results on five datasets demonstrate that ECCT outperforms previous clustering methods. In particular, the ARI metric of ECCT on the STL-10 (ImageNet-Dogs) dataset is 0.852 (0.424), which is 10.3% (4.8%) higher than the best baseline.

Computer-Vision-Oriented Adaptive Sampling in Compressive Sensing.

Compressive sensing (CS) is recognized for its adeptness at compressing signals, making it a pivotal technology in the context of sensor data acquisition. With the proliferation of image data in Internet of Things (IoT) systems, CS is expected to reduce the transmission cost of signals captured by various sensor devices. However, the quality of CS-reconstructed signals inevitably degrades as the sampling rate decreases, which poses a challenge in terms of the inference accuracy in downstream computer vision (CV) tasks. This limitation imposes an obstacle to the real-world application of existing CS techniques, especially for reducing transmission costs in sensor-rich environments. In response to this challenge, this paper contributes a CV-oriented adaptive CS framework based on saliency detection to the field of sensing technology that enables sensor systems to intelligently prioritize and transmit the most relevant data. Unlike existing CS techniques, the proposal prioritizes the accuracy of reconstructed images for CV purposes, not only for visual quality. The primary objective of this proposal is to enhance the preservation of information critical for CV tasks while optimizing the utilization of sensor data. This work conducts experiments on various realistic scenario datasets collected by real sensor devices. Experimental results demonstrate superior performance compared to existing CS sampling techniques across the STL10, Intel, and Imagenette datasets for classification and KITTI for object detection. Compared with the baseline uniform sampling technique, the average classification accuracy shows a maximum improvement of 26.23%, 11.69%, and 18.25%, respectively, at specific sampling rates. In addition, even at very low sampling rates, the proposal is demonstrated to be robust in terms of classification and detection as compared to state-of-the-art CS techniques. This ensures essential information for CV tasks is retained, improving the efficacy of sensor-based data acquisition systems.

MV-MR: Multi-Views and Multi-Representations for Self-Supervised Learning and Knowledge Distillation.

We present a new method of self-supervised learning and knowledge distillation based on multi-views and multi-representations (MV-MR). MV-MR is based on the maximization of dependence between learnable embeddings from augmented and non-augmented views, jointly with the maximization of dependence between learnable embeddings from the augmented view and multiple non-learnable representations from the non-augmented view. We show that the proposed method can be used for efficient self-supervised classification and model-agnostic knowledge distillation. Unlike other self-supervised techniques, our approach does not use any contrastive learning, clustering, or stop gradients. MV-MR is a generic framework allowing the incorporation of constraints on the learnable embeddings via the usage of image multi-representations as regularizers. The proposed method is used for knowledge distillation. MV-MR provides state-of-the-art self-supervised performance on the STL10 and CIFAR20 datasets in a linear evaluation setup. We show that a low-complexity ResNet50 model pretrained using proposed knowledge distillation based on the CLIP ViT model achieves state-of-the-art performance on STL10 and CIFAR100 datasets.

An attention mechanism module with spatial perception and channel information interaction

In the field of deep learning, the attention mechanism, as a technology that mimics human perception and attention processes, has made remarkable achievements. The current methods combine a channel attention mechanism and a spatial attention mechanism in a parallel or cascaded manner to enhance the model representational competence, but they do not fully consider the interaction between spatial and channel information. This paper proposes a method in which a space embedded channel module and a channel embedded space module are cascaded to enhance the model’s representational competence. First, in the space embedded channel module, to enhance the representational competence of the region of interest in different spatial dimensions, the input tensor is split into horizontal and vertical branches according to spatial dimensions to alleviate the loss of position information when performing 2D pooling. To smoothly process the features and highlight the local features, four branches are obtained through global maximum and average pooling, and the features are aggregated by different pooling methods to obtain two feature tensors with different pooling methods. To enable the output horizontal and vertical feature tensors to focus on different pooling features simultaneously, the two feature tensors are segmented and dimensionally transposed according to spatial dimensions, and the features are later aggregated along the spatial direction. Then, in the channel embedded space module, for the problem of no cross-channel connection between groups in grouped convolution and for which the parameters are large, this paper uses adaptive grouped banded matrices. Based on the banded matrices utilizing the mapping relationship that exists between the number of channels and the size of the convolution kernels, the convolution kernel size is adaptively computed to achieve adaptive cross-channel interaction, enhancing the correlation between the channel dimensions while ensuring that the spatial dimensions remain unchanged. Finally, the output horizontal and vertical weights are used as attention weights. In the experiment, the attention mechanism module proposed in this paper is embedded into the MobileNetV2 and ResNet networks at different depths, and extensive experiments are conducted on the CIFAR-10, CIFAR-100 and STL-10 datasets. The results show that the method in this paper captures and utilizes the features of the input data more effectively than the other methods, significantly improving the classification accuracy. Despite the introduction of an additional computational burden (0.5 M), however, the overall performance of the model still achieves the best results when the computational overhead is comprehensively considered.

Stable and Fast Deep Mutual Information Maximization Based on Wasserstein Distance.

Deep learning is one of the most exciting and promising techniques in the field of artificial intelligence (AI), which drives AI applications to be more intelligent and comprehensive. However, existing deep learning techniques usually require a large amount of expensive labeled data, which limit the application and development of deep learning techniques, and thus it is imperative to study unsupervised machine learning. The learning of deep representations by mutual information estimation and maximization (Deep InfoMax or DIM) method has achieved unprecedented results in the field of unsupervised learning. However, in the DIM method, to restrict the encoder to learn more normalized feature representations, an adversarial network learning method is used to make the encoder output consistent with a priori positively distributed data. As we know, the model training of the adversarial network learning method is difficult to converge, because there is a logarithmic function in the loss function of the cross-entropy measure, and the gradient of the model parameters is susceptible to the "gradient explosion" or "gradient disappearance" phenomena, which makes the training of the DIM method extremely unstable. In this regard, we propose a Wasserstein distance-based DIM method to solve the stability problem of model training, and our method is called the WDIM. Subsequently, the training stability of the WDIM method and the classification ability of unsupervised learning are verified on the CIFAR10, CIFAR100, and STL10 datasets. The experiments show that our proposed WDIM method is more stable to parameter updates, has faster model convergence, and at the same time, has almost the same accuracy as the DIM method on the classification task of unsupervised learning. Finally, we also propose a reflection of future research for the WDIM method, aiming to provide a research idea and direction for solving the image classification task with unsupervised learning.

Convolutional Neural Network (CNN) Layer Development for Effectiveness of Classification Tasks

This paper presents a novel 2D convolutional layer motivated by the principles of Partial Differential Equation (PDE) of Neural Interaction. Our objective is to leverage this layer to enhance the classification accuracy of Deep Convolutional Neural Networks (DCNN) for various classification tasks. We place a particular emphasis on its integration within the ResNet architecture, and we conduct experimental evaluations on the CIFAR10 and STL10 datasets to validate its efficacy.

Face Sketch to Image Generation using Generative Adversarial Network

Numerous studies have been conducted in the area of sketch to picture conversion and they got the good outcomes, but sometimes it is not accurate that they observed the blurry boundaries, the mixing of two colors that is the color of hair and face or mixing of both. These results are of the convolution neural networks that are basic of GAN. So to overcome their drawbacks we proposed a novel generative adversarial network using conditional GAN. For that we converted the original image in sketch and both the sketch and original image as reference is applied as input. We got more realistic and sharp colored images as compared to other. We focused on the feature detection, and the results are good. For the experimentation we used the STL-10 dataset. We overcome the problem of mixing of colors and got the different colors for hair, lips, and skin using conditional GAN as compared to CNN modern with increased performance and precision.

SpinalNet: Deep Neural Network With Gradual Input

Deep neural networks (DNNs) have achieved the state of the art performance in numerous fields. However, DNNs need high computation times, and people always expect better performance in a lower computation. Therefore, we study the human somatosensory system and design a neural network (SpinalNet) to achieve higher accuracy with fewer computations. Hidden layers in traditional NNs receive inputs in the previous layer, apply activation function, and then transfer the outcomes to the next layer. In the proposed SpinalNet, each layer is split into three splits: 1) input split, 2) intermediate split, and 3) output split. Input split of each layer receives a part of the inputs. The intermediate split of each layer receives outputs of the intermediate split of the previous layer and outputs of the input split of the current layer. The number of incoming weights becomes significantly lower than traditional DNNs. The SpinalNet can also be used as the fully connected or classification layer of DNN and supports both traditional learning and transfer learning. We observe significant error reductions with lower computational costs in most of the DNNs. Traditional learning on the VGG-5 network with SpinalNet classification layers provided the state-of-the-art (SOTA) performance on QMNIST, Kuzushiji-MNIST, and EMNIST (Letters, Digits, and Balanced) datasets. Traditional learning with ImageNet pre-trained initial weights and SpinalNet classification layers provided the SOTA performance on STL-10, Fruits 360, Bird225, and Caltech-101 datasets. The scripts of the proposed SpinalNet training are available at the following link: <uri>https://github.com/dipuk0506/SpinalNet</uri> <i>Impact Statement</i>&#x2014;Research in deep neural networks (DNNs) has gained significant attention from industries and academia due to their eye-catching performance. DNNs have enabled machines to perform myriad tasks with high accuracy that once only humans could do. Several researchers have recently proposed different types of NNs and have achieved high accuracy. The recent success of biologically inspired convolutional neural networks and the miraculous spinal architecture of humans has motivated us to develop a neural network with gradual inputs. We have achieved superior performance in several datasets. After the first online appearance of the initial version of this paper, several researchers have applied the proposed neural network in several new datasets and reported promising results. We may observe numerous novel applications of SpinalNet in upcoming years.

Mathematical Analysis and Performance Evaluation of the GELU Activation Function in Deep Learning

Selecting the most suitable activation function is a critical factor in the effectiveness of deep learning models, as it influences their learning capacity, stability, and computational efficiency. In recent years, the Gaussian error linear unit (GELU) activation function has emerged as a dominant method, surpassing traditional functions such as the rectified linear unit (ReLU) in various applications. This study presents a rigorous mathematical investigation of the GELU activation function, exploring its differentiability, boundedness, stationarity, and smoothness properties in detail. In addition, we conduct an extensive experimental comparison of the GELU function against a broad range of alternative activation functions, utilizing a residual convolutional network trained on the CIFAR-10, CIFAR-100, and STL-10 datasets as the empirical testbed. Our results demonstrate the superior performance of GELU compared to other activation functions, establishing its suitability for a wide range of deep learning applications. This comprehensive study contributes to a more profound understanding of the underlying mathematical properties of GELU and provides valuable insights for practitioners aiming to select activation functions that optimally align with their specific objectives and constraints in deep learning.

Identification of untrained class data using neuron clusters

Convolutional neural networks (CNNs), a representative type of deep neural networks, are used in various fields. There are problems that should be solved to operate CNN in the real-world. In real-world operating environments, the CNN’s performance may be degraded due to data of untrained types, which limits its operability. In this study, we propose a method for identifying data of a type that the model has not trained on based on the neuron cluster, a set of neurons activated based on the type of input data. In experiments performed on the ResNet model with the MNIST, CIFAR-10, and STL-10 datasets, the proposed method identifies data of untrained and trained types with an accuracy of 85% or higher. The more data used for neuron cluster identification, the higher the accuracy; conversely, the more complex the dataset's characteristics, the lower the accuracy. The proposed method uses only the information of activated neurons without any addition or modification of the model’s structure; hence, the computational cost is low without affecting the classification performance of the model.

Mixed and constrained input mutation for effective fuzzing of deep learning systems

Adversarial examples cause misclassifications of deep learning (DL) systems. It isn’t easy to debug misclassifications due to the intrinsic complexity of DL architecture. Thus, applying coverage-guided fuzzing, a widely used technique to find crashes in complex software, is promising. However, mutation strategies of DL fuzzers, such as DeepHunter, have a limitation. They restrict multiple transformations and so penalize generating diverse inputs. Otherwise, they cause significant distortion, rendering invalid inputs, such as unrecognizable or ambiguous to humans. However, multiple transformations are critical in mutation-based fuzzing. To address this problem, we propose the mixed and constrained mutation (MCM) for DL fuzzers. Human perception-based constraints of MCM avoid significant distortion in a single transformation and the aggregation of multiple transformations. We verify transformation parameters through a survey with 15 participants on each MNIST, STL-10, and ImageNet dataset to implement such constraints, followed by statistical tests. MCM returns valid inputs in almost every fuzzing iteration. Furthermore, MCM improved the fuzzing performance on various DL architectures on MNIST, STL-10, and ImageNet compared to DeepHunter: MCM discovered 17.6% more seeds showing new coverage and 132% more adversarial examples on average. These adversarial examples correspond to more than double the incorrect classes for each original image than DeepHunter.

AlphaGAN: Fully Differentiable Architecture Search for Generative Adversarial Networks.

Generative Adversarial Networks (GANs) are formulated as minimax game problems that generative networks attempt to approach real data distributions by adversarial learning against discriminators which learn to distinguish generated samples from real ones, of which the intrinsic problem complexity poses challenges to performance and robustness. In this work, we aim to boost model learning from the perspective of network architectures, by incorporating recent progress on automated architecture search into GANs. Specially we propose a fully differentiable search framework, dubbed alphaGAN, where the searching process is formalized as solving a bi-level minimax optimization problem. The outer-level objective aims for seeking an optimal network architecture towards pure Nash Equilibrium conditioned on the network parameters of generators and discriminators optimized with a traditional adversarial loss within inner level. The entire optimization performs a first-order approach by alternately minimizing the two-level objective in a fully differentiable manner that enables obtaining a suitable architecture efficiently from an enormous search space. Extensive experiments on CIFAR-10 and STL-10 datasets show that our algorithm can obtain high-performing architectures only with 3-GPU hours on a single GPU in the search space comprised of approximate 2×1011 possible configurations. We further validate the method on the state-of-the-art network StyleGAN2, and push the score of Fréchet Inception Distance (FID) further, i.e., achieving 1.94 on CelebA, 2.86 on LSUN-church and 2.75 on FFHQ, with relative improvements 3%∼ 26% over the baseline architecture. We also provide a comprehensive analysis of the behavior of the searching process and the properties of searched architectures, which would benefit further research on architectures for generative models. Codes and models are available at https://github.com/yuesongtian/AlphaGAN.

Self-Sparse Generative Adversarial Networks

Generative Adversarial Networks (GANs) are an unsupervised generative model that learns data distribution through adversarial training. However, recent experiments indicated that GANs are difficult to train due to the requirement of optimization in the high dimensional parameter space and the zero gradient problem. In this work, we propose a Self Sparse Generative Adversarial Network (Self-Sparse GAN) that reduces the parameter space and alleviates the zero gradient problem. In the Self-Sparse GAN, we design a Self-Adaptive Sparse Transform Module (SASTM) comprising the sparsity decomposition and feature-map recombination, which can be applied on multi-channel feature maps to obtain sparse feature maps. The key idea of Self-Sparse GAN is to add the SASTM following every deconvolution layer in the generator, which can adaptively reduce the parameter space by utilizing the sparsity in multi-channel feature maps. We theoretically prove that the SASTM can not only reduce the search space of the convolution kernel weight of the generator but also alleviate the zero gradient problem by maintaining meaningful features in the Batch Normalization layer and driving the weight of deconvolution layers away from being negative. The experimental results show that our method achieves the best FID scores for image generation compared with WGAN-GP on MNIST, Fashion-MNIST, CIFAR-10, STL-10, mini-ImageNet, CELEBA-HQ, and LSUN bedrooms, and the relative decrease of FID is 4.76% ~ 21.84%.

Evolutionary Architectural Search for Generative Adversarial Networks

The Generative Adversarial Network (GAN) has shown powerfulness in various real-world artificial intelligence applications. However, its network architecture is generally designed through a manual trial-and-error process, which is relatively tedious, slow, and sub-optimal. This paper hence develops an evolutionary architectural search (EAS) technique to automate the entire design process of the GAN. In particular, different objective functions are used in the generator of a GAN as variation operators. This helps train the generator with various candidate architectures and their associated weights to play adversarial training against the discriminator of the GAN. Following evaluations by the discriminator, superior candidate generators survive to the next generation and evolve for potentially better architectures and their weights simultaneously, leading to more stabilized GANs with improved performance. The GAN designed through EAS is termed an EAS-GAN in this paper and is tested against existing evolutionary and other state-of-the-art GANs. The test results show that the EAS-GAN offers better generative performance overall, with the Fréchet inception distance scoring 22.1, 38.8, and 8.3 on the CIFAR-10, STL-10, and LSUN bedroom data sets, respectively.

HAM: Hybrid attention module in deep convolutional neural networks for image classification

Recently, many researches have demonstrated that the attention mechanism has great potential in improving the performance of deep convolutional neural networks (CNNs). However, the existing methods either ignore the importance of using channel attention and spatial attention mechanisms simultaneously or bring much additional model complexity. In order to achieve a balance between performance and model complexity, we propose the Hybrid Attention Module (HAM), a really lightweight yet efficient attention module. Given an intermediate feature map as the input feature, HAM firstly produces one channel attention map and one channel refined feature through the channel submodule, and then based on the channel attention map, the spatial submodule divides the channel refined feature into two groups along the channel axis to generate a pair of spatial attention descriptors. By applying saptial attention descriptors, the spatial submodule generates the final refined feature which can adaptively emphasize the important regions. Besides, HAM is a simple and general module, it can be embedded into various mainstream deep CNN architectures seamlessly and can be trained with base CNNs in the end-to-end way. We evaluate HAM through abundant of experiments on CIFAR-10, CIFAR-100 and STL-10 datasets. The experimental results show that HAM-integrated networks achieve accuracy improvements and further reduce the negative impact of less training data on deeper networks performance than its counterparts, which proves the effectiveness of HAM.

SoFTNet: A concept-controlled deep learning architecture for interpretable image classification

Interpreting deep learning (DL)-based computer vision models is challenging due to the complexity of internal representations. Most recent techniques for rendering DL learning outcomes interpretable operate on low-level features rather than high-level concepts. Methods that explicitly incorporate high-level concepts do so through a determination of the relevancy of user-defined concepts or else concepts extracted directly from the data. However, they do not leverage the potential of concepts to explain model predictions. To overcome this challenge, we introduce a novel DL architecture – the Slow/Fast Thinking Network (SoFTNet) – enabling users to define/control high-level features and utilize them to perform image classification predicatively. We draw inspiration from the dual-process theory of human thought processes, decoupling low-level, fast & non-transparent processing from high-level, slow & transparent processing. SoFTNet hence uses a shallow convolutional neural network for low-level processing in conjunction with a memory network for high-level concept-based reasoning.We conduct experiments on the CUB-200-2011 and STL-10 datasets and also present a novel concept-based deep K-nearest neighbor approach for baseline comparisons. Our experiments show that SoFTNet achieves comparable performance to state-of-art non-interpretable models and outperforms comparable interpretative methods.

Maximizing bi-mutual information of features for self-supervised deep clustering

Self-supervised learning based on mutual information makes good use of classification models and label information produced by clustering tasks to train networks parameters, and then updates the downstream clustering assignment with respect to maximizing mutual information between label information. This kind of methods have attracted more and more attention and obtained better progress, but there is still a larger improvement space compared with the methods of supervised learning, especially on the challenge image datasets. To this end, a self-supervised deep clustering method by maximizing mutual information is proposed (bi-MIM-SSC), where deep convolutional network is employed as a feature encoder. The first term is to maximize mutual information between output-feature pairs for importing more semantic meaning to the output features. The second term is to maximize mutual information between an input image and its feature generated by the encoder for keeping the useful information of an original image in latent space as possible. Furthermore, pre-training is carried out to further enhance the representation ability of the encoder, and the auxiliary over-clustering is added in clustering network. The performance of the proposed method bi-MIM-SSC is compared with other clustering methods on the CIFAR10, CIFAR100 and STL10 datasets. Experimental results demonstrate that the proposed bi-MIM-SSC method has better feature representation ability and provide better clustering results.

Adaptive Weighted Discriminator for Training Generative Adversarial Networks.

Generative adversarial network (GAN) has become one of the most important neural network models for classical unsupervised machine learning. A variety of discriminator loss functions have been developed to train GAN's discriminators and they all have a common structure: a sum of real and fake losses that only depends on the actual and generated data respectively. One challenge associated with an equally weighted sum of two losses is that the training may benefit one loss but harm the other, which we show causes instability and mode collapse. In this paper, we introduce a new family of discriminator loss functions that adopts a weighted sum of real and fake parts, which we call adaptive weighted loss functions or aw-loss functions. Using the gradients of the real and fake parts of the loss, we can adaptively choose weights to train a discriminator in the direction that benefits the GAN's stability. Our method can be potentially applied to any discriminator model with a loss that is a sum of the real and fake parts. For our experiments, SN-GAN, AutoGAN, and BigGAN are used. Experiments validated the effectiveness of our loss functions on unconditional and conditional image generation tasks, improving the baseline results by a significant margin on CIFAR-10, STL-10, and CIFAR-100 datasets in Inception Scores (IS) and Fréchet Inception Distance (FID) metrics.

Improving generative adversarial networks with simple latent distributions

Generative Adversarial Networks (GANs) have drawn great attention recently since they are the powerful models to generate high-quality images. Although GANs have achieved great success, they usually suffer from unstable training and consequently may lead to the poor generations in some cases. Such drawback is argued mainly due to the difficulties in measuring the divergence between the highly complicated the real and fake data distributions, which are normally in the high-dimensional space. To tackle this problem, previous researchers attempt to search a proper divergence capable of measuring the departure of the complex distributions. In contrast, we attempt to alleviate this problem from a different perspective: while retaining the information as much as possible of the original high dimensional distributions, we learn and leverage an additional latent space where simple distributions are defined in a low-dimensional space; as a result, we can readily compute the distance between two simple distributions with an available divergence measurement. Concretely, to retain the data information, the mutual information is maximized between the variables for the high dimensional complex distributions and the low dimensional simple distributions. The departure of the resulting simple distributions are then measured in the original way of GANs. Additionally, for simplifying the optimization further, we optimize directly the lower bound for mutual information. Termed as SimpleGAN, we conduct the proposed approach over the several different baseline models, i.e., conventional GANs, DCGAN, WGAN-GP, WGAN-GP-res, and LSWGAN-GP on the benchmark CIFAR-10 and STL-10 datasets. SimpleGAN shows the obvious superiority on these baseline models. Furthermore, in comparison with the existing methods measuring directly the distribution departure in the high-dimensional space, our method clearly demonstrates its superiority. Finally, a series of experiments show the advantages of the proposed SimpleGAN.