System Architecting Approach for Designing Deep Learning Models

Abstract

Deep Learning (DL) models have proven to be very effective in solving many challenging problems, especially, those related to computer vision, text, and speech. However, the design of such models is challenging because of the vast search space and computational complexity that needs to be explored. Our goal in this paper is to reduce the human effort required to design architectures by using a system architecture development process that allows the exploration of large design space by automating certain model construction, alternative generation, and assessment. The proposed framework is generic and targeted at all deep learning architectures that can be expressed by logical models with certain numeric properties. The implementation of the proposed approach is presented, along with the test results achieved on CIFAR-10 dataset using a convolutional neural network (CNN). We show that the architecture generated by our approach achieves 5.23% error rate with only 1.2M parameters, which shows the capability to design high performing architectures.