Compressed lightweight deep learning models for resource‐constrained Internet of things devices in the healthcare sector

Abstract

AbstractThe performance of convolutional neural networks (CNNs) in image classification and object detection has been remarkable, even though they contain millions and billions of parameters. This over‐parameterization of CNN makes them both memory‐intensive and computationally complex and exhaustive. This greatly hinders the application of CNNs in resource‐constrained environments such as Internet of things (IoT) and edge devices. This poses a critical challenge for CNNs in deploying these powerful computer vision tools to mobile devices, which needs immediate attention. In this study, we have proposed a novel technique based on non‐convex optimization, max‐norm regularization. The max‐norm will structurally prune the number of parameters without compromising the model's performance. The proximal gradient descent algorithm is used for network optimization while using this non‐convex regularizer. The max‐norm is combined with the channel pruning to achieve more sparse CNN networks. Later, the pruned network can be easily deployed in the resource‐constrained application environment. The proposed technique is tested on several benchmark datasets for validation. In addition, in this study, the sparsified CNNs are used for biomedical image analysis using the BRAIN MRI dataset. This sparsely trained CNN model can later serve as the best lightweight model applicable in the IoT healthcare sector for detecting and classifying three types of brain tumours, one of the most life‐threatening diseases whose early detection can save the costly lives of human beings. This is the first paper to propose the novel max‐norm regularizer to enforce sparse learning through CNNs. The paper provides a detailed analysis of convex and non‐convex regularizers before presenting the proposed novel max‐norm regularizer. Finally, the paper compares the proposed max‐norm regularizer with existing regularization methods using state‐of‐the‐art CNN models.