Neural architecture search via reference point based multi‐objective evolutionary algorithm

Abstract

For neural architecture search, NSGA-Net has searched a representative neural architecture set of Pareto-optimal solutions to consider both accuracy and computation complexity simultaneously. However, some decision-makers only concentrate on such neural architectures in the subpart regions of Pareto-optimal Frontier that they have interests in. Under the above circumstances, certain uninterested neural architectures may cost many computing resources. In order to consider the preference of decision-makers, we propose the reference point based NSGA-Net (RNSGA-Net) for neural architecture search. The core of RNSGA-Net adopts the reference point approach to guarantee the Pareto-optimal region close to the reference points and also combines the advantage of NSGAII with the fast nondominated sorting approach to split the Pareto front. Moreover, we augment an extra bit value of the original encoding to represent two types of residual block and one type of dense block for residual connection and dense connection in the RNSGA-Net. In order to satisfy the decision-maker preference, the multi-objective is measured to search competitive neural architecture by minimizing an error metric and FLOPs of computational complexity. Experiment results on the CIFAR-10 dataset demonstrate that RNSGA-Net can improve NSGA-Net in terms of the more structured representation space and the preference of decision-makers.