LightNAS: On Lightweight and Scalable Neural Architecture Search for Embedded Platforms

Abstract

Neural architecture search (NAS) is an emerging paradigm to automate the design of competitive deep neural networks (DNNs). In practice, DNNs are subject to strict latency constraints and any violation may lead to catastrophic consequences (e.g., autonomous vehicles). However, to obtain the architecture that strictly satisfies the required latency constraint, previous hardware-aware differentiable NAS methods have to repeat a plethora of search runs to tune relevant hyper-parameters by trial and error, and as a result, the total design cost increases proportionally (empirically by 10 times). To tackle this, we, in this paper, introduce a lightweight and scalable hardware-aware NAS framework named LightNAS, which consists of two separate stages. In the first stage, we strive to search for the architecture that strictly satisfies the required latency constraint at the macro level in a differentiable manner, and more importantly, through a one-time search (i.e., you only search once). The architectures searched in the first stage are denoted as LightNets. After that, in the second stage, we introduce an efficient evolutionary scheme to further explore the micro-level channel configuration of each LightNet at low cost. To achieve this, we propose an effective yet computationally cheap proxy, namely batchwise training estimation (BTE), as a plug-in complement to enable the channel-level exploration of LightNets on the fly such that the accuracy of LightNets can be improved without degrading the runtime latency on target hardware. Finally, extensive experiments are conducted on one popular embedded platform (i.e., Nvidia Jetson AGX Xavier) to demonstrate the efficacy of the proposed approach over previous state-of-the-art counterparts.