An efficient evolutionary architecture search for variational autoencoder with alternating optimization and adaptive crossover

Abstract

Variational autoencoder is a commonly unsupervised learning model. However, its complex structure hinders the utilization of the network architecture search algorithm to release researchers from tedious manual design. To design excellent architectures automatically, this paper proposes an efficient evolutionary architecture search for variational autoencoder with alternating optimization and adaptive crossover(AOC-VAE). Firstly, to alleviate the problem of large search space when automatically designing variational autoencoders, AOC-VAE designs an alternating optimized search mechanism based on the specific coupling of encoder and decoder in variational autoencoders, which reduces the original huge search space almost to half. Then, AOC-VAE can find quickly the optimal individual in the solution space by designing an adaptive crossover mechanism. In early evolutionary period, the structural differences between individuals are relatively significant, making crossover operations more inclined to exchange structural information between individuals. As evolution progresses, the individual structures in the population tend to be similar, and the exchange of information concentrates on the parameter. Finally, in the optimization process, a fitness evaluation mechanism based on dynamic weights is designed to accurately find out the outstanding individuals under the current optimization goal. Individual fitness in the population is more inclined to be affected by the current optimization goal, thus guiding the population to evolve according to the optimization goal at different stages. AOC-VAE is verified on MNIST, SVHN, CIFAR-10, and CIFAR-100 benchmark datasets and compared with 14 algorithms. The experimental results show that the VAE network structure designed by the AOC-VAE performs well in the image classification task.