Abstract
The progress devoted to improving the performance of neural networks has come at a high price in terms of cost and experience. Fortunately, the emergence of Neural Architecture Search improves the speed of network design, but most excellent works only optimize for high accuracy without penalizing the model complexity. In this paper, we propose an efficient CNN architecture search framework, MOO-DNAS, with multi-objective optimization based on differentiable neural architecture search. The main goal is to trade off two competing objectives, classification accuracy and network latency, so that the search algorithm is able to discover an efficient model while maintaining high accuracy. In order to achieve a better implementation, we construct a novel factorized hierarchical search space to support layer variety and hardware friendliness. Furthermore, a robust sampling strategy named “hard-sampling” is proposed to obtain final structures with higher average performance by keeping the highest scoring operator. Experimental results on the benchmark datasets MINST, CIFAR10 and CIFAR100 demonstrate the effectiveness of the proposed method. The searched architectures, MOO-DNAS-Nets, achieve advanced accuracy with fewer parameters and FLOPs, and the search cost is less than one GPU-day.
Highlights
Convolutional neural networks (CNNs) have made great achievements in a variety of computer vision applications, including image recognition [1]-[5], object detection [6]-[8] and semantic segmentation [9], etc
To explore light-weight neural networks, we present an efficient CNN architecture search framework based on differentiable neural architecture search
In order to better understand network design automation, this study proposes MOO-Differentiable Neural Architecture Search (DNAS), a Neural Architecture Search (NAS) method based on differentiable neural architecture search and multi-objective optimization
Summary
Convolutional neural networks (CNNs) have made great achievements in a variety of computer vision applications, including image recognition [1]-[5], object detection [6]-[8] and semantic segmentation [9], etc. Most recent research focuses on developing efficient CNN models with improved performance for intelligent devices. Existing model compression techniques such as network pruning [10][11], quantization [12][13], and knowledge distillation [14] can be used to study compact deep neural networks by trading accuracy for efficiency. These techniques inevitably result in performance degradation due to information loss and the compressed models are usually the upper bound by pre-trained models. It is hard to design light-weight neural networks [15]-[19], because researchers need to consider many influencing
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
