AdaDeep: A Usage-Driven, Automated Deep Model Compression Framework for Enabling Ubiquitous Intelligent Mobiles

Abstract

Recent breakthroughs in deep neural networks (DNNs) have fueled a tremendously growing demand for bringing DNN-powered intelligence into mobile platforms. While the potential of deploying DNNs on resource-constrained platforms has been demonstrated by DNN compression techniques, the current practice suffers from two limitations: 1) merely stand-alone compression schemes are investigated even though each compression technique only suit for certain types of DNN layers; and 2) mostly compression techniques are optimized for DNNs’ inference accuracy, without explicitly considering other application-driven system performance (e.g., latency and energy cost) and the varying resource availability across platforms (e.g., storage and processing capability). To this end, we propose AdaDeep, a usage-driven, automated DNN compression framework for systematically exploring the desired trade-off between performance and resource constraints, from a holistic system level. Specifically, in a layer-wise manner, AdaDeep automatically selects the most suitable combination of compression techniques and the corresponding compression hyperparameters for a given DNN. Thorough evaluations on six datasets and across twelve devices demonstrate that <inline-formula><tex-math notation="LaTeX">${\sf AdaDeep}$</tex-math></inline-formula> can achieve up to <inline-formula><tex-math notation="LaTeX">$18.6\times$</tex-math></inline-formula> latency reduction, <inline-formula><tex-math notation="LaTeX">$9.8\times$</tex-math></inline-formula> energy-efficiency improvement, and <inline-formula><tex-math notation="LaTeX">$37.3\times$</tex-math></inline-formula> storage reduction in DNNs while incurring negligible accuracy loss. Furthermore, <inline-formula><tex-math notation="LaTeX">${\sf AdaDeep}$</tex-math></inline-formula> also uncovers multiple novel combinations of compression techniques.