Regularized Differentiable Architecture Search

Abstract

Differentiable architecture search (DARTS) transforms architectural optimization into a super-network optimization by stacking two cells (2 c.). However, repeatedly stacking two cells is a sub-optimal operation since cells in different depths should be various. Besides, we find that the performance is slightly improved by increasing the number of searched cells (e.g., from 2 c. to 5 c.), but it leads to uneven resource allocation. This letter proposes a regularized DARTS (RDARTS) to adjust the architectural differences and balance degrees of freedom and resource allocation. Specifically, we use separate architectural parameters for two reduction cells and three normal cells, and then propose a Reg distance to calculate the difference between cells. We design a new validation loss which is the weighting of cross-entropy and Reg loss, and introduce an adaptive adjustment method. Results show that RDARTS achieves the top-1 accuracy of 97.64% and 75.8% on CIFAR and ImageNet.