Abstract
Deep learning frameworks have progressed beyond human recognition capabilities and, now it’s the perfect opportunity to optimize them for implementation on the embedded platforms. The present deep learning architectures support learning capabilities, but they lack flexibility for applying learned knowledge on the tasks in other unfamiliar domains. This work tries to fill this gap with the deep neural network-based solution for object detection in unrelated domains with a focus on the reduced footprint of the developed model. Knowledge distillation provides efficient and effective teacher-student learning for a variety of different visual recognition tasks. A lightweight student network can be easily trained under the guidance of the high-capacity teacher networks. The teacher-student architecture implementation on binary classes shows a 20% improvement in accuracy within the same training iterations using the transfer learning approach. The scalability of the student model is tested with binary, ternary and multiclass and their performance is compared on basis of inference speed. The results show that the inference speed does not depend on the number of classes. For similar recognition accuracy, the inference speed of about 50 frames per second or 20ms per image. Thus, this approach can be generalized as per the application requirement with minimal changes, provided the dataset format compatibility.
Highlights
Deep neural networks are thriving, due to vast data availability, newer complex models, and heterogeneous compute capacity
Convolution Neural Networks models have evolved to surpass the human capabilities in image classification task but when it comes to their deployment on the edge devices, there use is limited due to various resource constraints as described below: A
The factors which effect the MAC operations are batch size, image dimensions, filter type, no. of channels, kernel size and activation size. These combined for every neuron to neuron connections make the millions of hyper-parameters of the DNN. To reduce these transformation functions parameterized by learnable weights, researchers worldwide have developed their own various model compression techniques, but only some of the well-researched approaches are covered here for brief overview
Summary
Deep neural networks are thriving, due to vast data availability, newer complex models, and heterogeneous compute capacity. In [6], it is shown that having a minimum depth to the network is vital for the model performance All these approaches assumed that the distributions of the labelled and unlabelled data were the same. Squeezenet [8] performs 2.17 billion of operation but with a smaller footprint of 4.8MB, while Darknet [9], an open source for the Yolo [10], does less than 1 billion operations with 28MB of the footprint Note that this comparison is assuming a baseline accuracy of 80 per cent in recognizing the labelled visuals. This work explores the knowledge distillation approach in deep neural networks for IoT edge devices for real-time applications.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: International Journal of Advanced Computer Science and Applications
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
