D-DARTS: Distributed Differentiable Architecture Search

Abstract

Differentiable ARchiTecture Search (DARTS) is one of the most trending Neural Architecture Search (NAS) methods. It drastically reduces search cost by resorting to weight-sharing. However, this approach also dramatically reduces the search space, thus excluding potential promising architectures. In this article, we propose D-DARTS, a solution that addresses this problem by nesting neural networks at the cell level instead of using weight-sharing to produce more diversified and specialized architectures. Moreover, we introduce a novel algorithm that can derive deeper architectures from a few trained cells, increasing performance and saving computation time. In addition, we introduce DARTOpti, an alternative search space in which we optimize existing handcrafted architectures instead of searching from scratch. Our solution reaches competitive performance on image classification tasks.