ASMEvoNAS: Adaptive segmented multi-objective evolutionary network architecture search

Abstract

Network architecture search (NAS) has attracted much attention as an automatic design technique of network architecture. In particular, multi-objective evolutionary algorithms (MOEAs) have been a popular kind of optimizer in NAS due to their global optimization capability. However, as a population-based iterative search method, MOEAs are subject to the unbearable computational cost of individual evaluation on multiple objectives at each generation, which affects their generalization ability and transferability of MOEA-based NAS. Therefore, an adaptive segmented multi-objective evolutionary network architecture search (ASMEvoNAS) method is proposed in this paper. Firstly, an adaptive segmented evaluation strategy is designed to adaptively select different but more suitable objectives to efficiently assess the candidate architectures at different evolutionary stages, instead of evaluating them by all the considered objectives simultaneously. Thus, the computational cost and complexity of the search process can be controlled and reduced to some extent. Secondly, a preference-based pre-selection strategy is designed to filter out the initialized architectures with high parameter quantities to reduce the total parameter scale of the whole population and memory consumption. Last, a novel desirable gene reservation-based crossover and a directed connection-based mutation are proposed to produce offspring. Experimental results show that ASMEvoNAS shows promising performance on CIFAR-10, CIFAR-100, and ImageNet with error rates of 2.21%, 15.57%, and 24.43% top-1, respectively. The proposed method reduces the search cost to 0.36 GPU-Days on CIFAR-10 while maintaining competitive classification performance compared to state-of-the-art networks. In addition, ASMEvoNAS presents superior performance when dealing with the considered transfer tasks, as well as the benchmark dataset of NAS.