A novel lightweight deep learning framework with knowledge distillation for efficient diabetic foot ulcer detection

Abstract

Diabetic Foot Ulcers (DFUs) are a critical healthcare issue requiring early detection. Although deep learning models show promise in DFU diagnosis, their large parameter sizes and high computational demands limit their practical use. This work addresses these challenges by introducing DFU-LWNet, a lightweight, knowledge-distilled model that maintains or even surpasses the performance of existing models while drastically reducing computing costs. The DFU-LWNet model synergistically integrates the MBConv Block, leveraging its capabilities in feature expansion, depthwise convolution for parameter efficiency, and adaptive feature recalibration through the Squeeze-and-Excitation (SE) mechanism. The methodology commences with the fine-tuning of seven pre-trained models, ultimately selecting InceptionV3 as the optimal teacher model based on experimental results. Subsequently, knowledge distillation (KD) is applied, leveraging the insights from this robust teacher model to enhance the performance of the student model, DFU-LWNet. The primary objective is to reduce the student model's parameter size and computational complexity while maintaining diagnostic accuracy. Through comprehensive evaluations, DFU-LWNet achieves an impressive classification accuracy of 96.23 % on a publicly available dataset comprising 1055 foot images. Notably, this achievement is accomplished with a remarkably low parameter size of only 0.49 million, consuming 2MB of disk space. This reduction in parameter size not only improves inference speed but also significantly reduces computational costs compared to pre-trained models. Moreover, the study employs Grad-CAM visualizations to demonstrate the model's enhanced saliency and interpretability in predictions post-KD, thereby offering valuable insights to clinicians. Additionally, the efficacy of DFU-LWNet extends to thermal imagery domains, as validated by experiments on a diverse dataset of thermal images of diabetic foot cases. The proposed approach achieves remarkable accuracy of 93.13 % on a separate thermal image dataset, demonstrating cross-domain success and reinforcing the model's practical utility in real-world scenarios. Compared to existing state-of-the-art methods, this work demonstrates promising practical viability. DFU-LWNet’s minimal parameter count and low prediction times make it well-suited for deployment in resource-limited healthcare settings.