Neural Architecture Search based on Genetic Algorithm and Deployed in a Bare-Metal Kubernetes Cluster

Abstract

The interest in Deep Neural Networks has dramatically increased, especially e. g. in Computer Vision or Neural Language Processing tasks. Due to the heavy influence of the Neural Networks architecture on its predictive accuracy, Neural Architecture Search has gained much attention in recent years. Neural Architecture Search typically comes along with a high computational demand and thus, requires scalability as well as high availability to ensure no data loss or waste of computational power. Hence, we developed a scalable and highly available multi-objective Neural Architecture Search and adopted it to the modern thinking of developing applications by subdividing an already existing, monolithic approach – based on a Genetic Algorithm – into microservices. Moreover, we adjusted the initial population creation by mutating each individual 1,000 times, extended the approach by inception layers, implemented it as island model and achieved on MNIST, Fashion-MNIST and CIFAR-10 dataset 99.75%, 94.35% and 89.90% test accuracy, respectively. Furthermore, we analyzed nine different configurations of the Genetic Algorithm – with only one subpopulation – to identify well performing settings. Besides, our model is strongly focused on high availability empowered by the deployment in our bare-metal Kubernetes cluster. Our results show that the introduced Neural Architecture Search can easily handle and recover – without the necessity of human interaction – from the exceptional loss of Kubernetes pods within seconds and no loss of results or the algorithms state.