Bi-EvoGAN: Bi-level Evolutionary Approach for Generative Adversarial Networks

Abstract

Generative Adversarial Networks (GANs) are an adversarial approach to generative modeling using deep learning methods, which have become one of the most relevant topics in the last years. Despite producing interesting outputs, recent studies have demonstrated that GANs remain difficult to control and suffer from training problems, such as instability and mode collapse, as well as several aspects of their models are designed by hand. Neuroevolution is a recent evolution strategy that uses Evolutionary Algorithms (EAs) in order to provide an automatic design of deep neural networks architectures. Many recent efforts have been dedicated to contributing on GANs by developing different single-level EAs approaches, however they could not solve their common issues completely which are still open problems. In this paper, we propose a new design of evolutionary algorithms in the context of GANs, consisting of dividing the evolution search space into two complementary levels, modeling a Bi-level Evolutionary Approach called Bi-EvoGAN searching to surpass their adversarial optimization difficulties and also to demonstrate the performance of considering two levels of optimization in comparison to a single optimization level. To the best of our knowledge, this work presents the first application of the bi-level optimization in the context of generative adversarial networks. The upper level uses an Evolutionary Algorithm-based Topology Optimization method named “EA-TO” to evolve a population of GAN networks as a set of individuals in order to find the best network topology. The lower level applies an Evolutionary Algorithm-based Decomposition for Hyperparameter Optimization termed “EA/D-HO” in order to select the best hyperparameters of each GAN individual. The performance of Bi-EvoGAN is explained by the double optimization of the GAN topology and their hyperparameters through the new bilevel optimization design allowing to evolve a GAN population in its higher fitness which will improve the quality of the search method and consequently to accelerate the convergence. Experimental studies are carried out based on four well-known benchmark datasets, MNIST, Fashion-MNIST, CelebA and LSUN-Bedroom allowing to show the efficiency of our approach in comparison to the other state-of-the-art approaches.