GLNAS: Greedy Layer-wise Network Architecture Search for low cost and fast network generation

Abstract

The process of applying machine learning algorithms to practical problems can be a challenging and tedious task for non-experts. Previous research has sought to alleviate this burden by introducing automated machine learning techniques, including Network Architecture Search (NAS) and Differentiable Architecture Search (DARTS). However, these methods use a fixed number of layers and predefined skip connections which impose limitations on the generation of an optimal network architecture. In this paper, we propose a novel approach called Greedy Layer-wise Network Architecture Search (GLNAS), which trains network layers one after another and evaluates the network’s performance after each layer is added. GLNAS also assesses the effectiveness of skip connections between layers by testing various outputs of previous layers as an input to the current layer. Our experiment results demonstrate that the network generated by GLNAS requires fewer parameters (i.e., 3.5 millions in both CIFAR-10 and CIFAR-100 datasets) and GPU resources during the searching phase (i.e., 0.17 and 0.24 GPU days in CIFAR-10 and CIFAR-100 datasets respectively) than many existing methods.