A novel weight pruning strategy for light weight neural networks with application to the diagnosis of skin disease

Abstract

In recent years, deep learning-based models have achieved significant advances in manifold computer vision problems, but tedious parameter tuning has complicated their application to computer-aided diagnostic (CAD) systems. As such, this study introduces a novel pruning strategy to improve the accuracy of five lightweight deep convolutional neural network (DCNN) architectures applied to the classification of skin disease. Unlike conventional pruning methods (such as optimal brain surgeon), the proposed technique does not change the model size yet improves performance after fine tuning. This training approach, intended to improve accuracy without increasing model complexity, is experimentally verified using 1167 pathological images. The clinical data included 11 different skin disease types collected over the past ten years, with varying image quantities in each category. A novel hierarchical pruning method, based on standard deviation, is then developed and used to prune parameters in each convolution layer according to the different weight distributions. This training strategy achieves an 83.5% Top-1 accuracy using a pruned MnasNet (12.5 MB), which is 1.8% higher than that of unpruned InceptionV3 (256 MB). Comparative experiments using other networks (MobileNetV2, SqueezeNet, ShuffleNetV2, Xception, ResNet50, DenseNet121) and dataset (HAM10000) also demonstrate consistent improvements when adopting the proposed model training technique. This distinctive robustness across various network types and simple deployment demonstrates the potential of this methodology for generalization to other computer vision tasks.