504 Deep learning empowered computer-assisted diagnosis on smartphone skin images

Abstract

Nowadays deep learning has been widely applied in identifying skin disease using clinical skin images. Along with the development of smartphone and communication technology, telemedicine becomes more accessible for patients to consult with medical experts. However, rare research was done to prove the feasibility of using deep learning to predict skin conditions based on smartphone captured skin images, especially taken by non-professional patients. We collected 46946 skin images from the online patient-doctor telemedicine platform in mainland China. Each image was annotated by at least 2 dermatologists mutual blindly. We trained an assembly 6 different deep neural networks to provide predictions. The model is able to predict the probability among 30 common skin conditions, including infectious conditions, inflammatory conditions, pigmentary conditions, vascular conditions, and even tumors. The model achieves the top 1 and top 3 accuracies of 0.7708 and 0.9194. The coverage error of our model is 1.4171, which indicates the model has a short depth of ranked predictions to cover all the true labels. And label ranking average precision is 0.9392, which means the model has a high fraction of higher-ranked predictions mapping with ground truth labels. Furthermore, the results show the ability to differentiate some skin conditions which have similar morphology by using a deep learning approach on patient captured skin images. In conclusion, a deep learning approach would potentially provide not only initial evaluation of skin lesions to patients but also clues for differential diagnosis to general doctors through smartphone skin photography.