MR-DARTS: Restricted connectivity differentiable architecture search in multi-path search space

Abstract

Differentiable search methods can be used to find effective network architectures fast. However, these approaches are accompanied by low accuracy when evaluating a searched architecture, especially evaluating a searched architecture after transferring it to a different dataset. Two reasons can explain this phenomenon. The one is that the networks composed of cells have the depth gap in their structures between the search and evaluate stage. Another is that cells have insufficient ability to extract diverse features. This paper presents the Multi-path Restricted DARTS method to address these critical problems, using a multi-path search space and a restricted connectivity algorithm to perform a more exact search with limited resources. Restricted connectivity algorithm deepens the cells’ structure and makes cells more suitable for deep networks to bridge the depth gap. Multi-path search space enables cells to extract and fuse different-scales features to improve the representation capacity of a network. Our approach achieves state-of-the-art performance on CIFAR10 and CIFAR100 with the smallest parameters (only 2.5 M), demonstrating strong transfer learning ability in complex datasets.