Clinical Skin Images Research Articles

Image Quality Assessment Using Convolutional Neural Network in Clinical Skin Images

The image quality received for clinical evaluation is often suboptimal. The goal is to develop an image quality analysis (IQA) tool to assess patient and primary care physicians (PCP) derived images using deep learning model. Dataset included patient and PCP derived images from August 21, 2018 - June 30, 2022 with 4 unique quality labels. VGG16 model was fine-tuned with input data and optimal threshold was determined by Youden’s index. Ordinal labels were transformed to binary labels by using a majority vote as model distinguishes between 2 categories (good vs. bad). At a threshold of 0.587, AUC for the test set was 0.885 (95%CI 0.838 - 0.933); sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were 0.829, 0.784, 0.906 and 0.645 respectively. Independent validation of 300 additional images (from patients and PCPs) demonstrated AUC 0.864 (95%CI: 0.818 - 0.909) and AUC 0.902 (95%CI: 0.85 - 0.95) respectively. The sensitivity, specificity, PPV and NPV for the 300 images were 0.827, 0.800, 0.959 and 0.450 respectively. We demonstrate a practical approach improving the image quality for clinical workflow. While users may have to capture additional images, this is offset by the improved workload and efficiency for clinical teams.

Human-multimodal deep learning collaboration in 'precise' diagnosis of lupus erythematosus subtypes and similar skin diseases.

Lupus erythematosus (LE) is a spectrum of autoimmune diseases. Due to the complexity of cutaneous LE (CLE), clinical skin image-based artificial intelligence is still experiencing difficulties in distinguishing subtypes of LE. We aim to develop a multimodal deep learning system (MMDLS) for human-AI collaboration in diagnosis of LE subtypes. This is a multi-centre study based on 25 institutions across China to assist in diagnosis of LE subtypes, other eight similar skin diseases and healthy subjects. In total, 446 cases with 800 clinical skin images, 3786 multicolor-immunohistochemistry (multi-IHC) images and clinical data were collected, and EfficientNet-B3 and ResNet-18 were utilized in this study. In the multi-classification task, the overall performance of MMDLS on 13 skin conditions is much higher than single or dual modals (Sen = 0.8288, Spe = 0.9852, Pre = 0.8518, AUC = 0.9844). Further, the MMDLS-based diagnostic-support help improves the accuracy of dermatologists from 66.88% ± 6.94% to 81.25% ± 4.23% (p = 0.0004). These results highlight the benefit of human-MMDLS collaborated framework in telemedicine by assisting dermatologists and rheumatologists in the differential diagnosis of LE subtypes and similar skin diseases.

44149 Visible improvements in skin pigmentation and redness with topical bakuchiol-containing sunscreen using noninvasive clinical skin imaging technique in photodamaged facial skin

44149 Visible improvements in skin pigmentation and redness with topical bakuchiol-containing sunscreen using noninvasive clinical skin imaging technique in photodamaged facial skin

Image Quality Assessment Using a Convolutional Neural Network for Clinical Skin Images

Background The quality of the images received for teledermatology evaluation is often suboptimal, with up to 50% of patients providing images that are poorly lit, off-center, or blurry. To ensure a similar level of care to in-person consultations, high-quality images are essential. Objective The aim of this study is to develop an image quality analysis tool to assess patient- and primary care physician (PCP)–derived images using a deep learning model leveraging multiple instance learning and ordinal regression for model predictions. Methods The data set used for this study was acquired from patient-derived images submitted to the Department of Dermatology, Duke University, between August 21, 2018, and December 31, 2019, and PCP-derived images between March 1, 2021, and June 30, 2022. Seven dermatology faculty members with a designation of professor, associate professor, and assistant professor evaluated 400 images each, and 2 dermatology residents evaluated 400 images, assuring that each image had 4 different quality labels. We used a pretrained model VGG16 architecture, further fine-tuned by updating weights based on the input data. The images were taken with cell phones (patients) or cameras (PCPs) in RGB scale, with the resolution being 76 pixels per inch for both height and width, and the average pixel size of the image being 2840×2793 (SD 986×983; 1471 inch2, SD 707 inch2). The optimal threshold was determined using the Youden index, which represents the best trade-off between sensitivity and specificity and balance the number of true positives and true negatives in the classification results. Once the model predicts the rank, the ordinal labels are transformed to binary labels by using a majority vote as the goal is to distinguish between 2 distinct categories (good vs bad quality) and not predict quality as a continuous variable. Results Based on the Youden index, we achieved a positive predicted value of 0.906, implying that the model will predict 90% of the good-quality images as such, while 10% of the poor-quality images are predicted as being of good quality to enhance clinical utility, with an area under the receiver operating characteristic curve (AUC) for the test set at 0.885 (95% CI 0.838-0.933) and sensitivity, specificity, and negative predictive value (NPV) of 0.829, 0.784, and 0.645, respectively. Further evaluation on independent validation consisting of 300 images from patients and 150 images from PCPs revealed AUCs of 0.864 (95% CI 0.818-0.909) and 0.902 (95% CI 0.85-0.95), respectively. The sensitivity, specificity, positive predicted value, and NPV for the 300 images were 0.827, 0.800, 0.959, and 0.450, respectively. Conclusions This study shows a practical approach to improve image quality for clinical decision-making. While patients and PCPs may have to capture additional images (due to lower NPV), this is offset by the reduced workload and improved efficiency of clinical teams due to the receipt of higher-quality images. Additional images can also be useful if all images (good or poor) are transmitted to medical records. Future studies need to focus on real-time clinical validation of our results. Conflicts of Interest None declared.

A Novel Vision Transformer Model for Skin Cancer Classification

Skin cancer can be fatal if it is found to be malignant. Modern diagnosis of skin cancer heavily relies on visual inspection through clinical screening, dermoscopy, or histopathological examinations. However, due to similarity among cancer types, it is usually challenging to identify the type of skin cancer, especially at its early stages. Deep learning techniques have been developed over the last few years and have achieved success in helping to improve the accuracy of diagnosis and classification. However, the latest deep learning algorithms still do not provide ideal classification accuracy. To further improve the performance of classification accuracy, this paper presents a novel method of classifying skin cancer in clinical skin images. The method consists of four blocks. First, class rebalancing is applied to the images of seven skin cancer types for better classification performance. Second, an image is preprocessed by being split into patches of the same size and then flattened into a series of tokens. Third, a transformer encoder is used to process the flattened patches. The transformer encoder consists of N identical layers with each layer containing two sublayers. Sublayer one is a multihead self-attention unit, and sublayer two is a fully connected feed-forward network unit. For each of the two sublayers, a normalization operation is applied to its input, and a residual connection of its input and its output is calculated. Finally, a classification block is implemented after the transformer encoder. The block consists of a flattened layer and a dense layer with batch normalization. Transfer learning is implemented to build the whole network, where the ImageNet dataset is used to pretrain the network and the HAM10000 dataset is used to fine-tune the network. Experiments have shown that the method has achieved a classification accuracy of 94.1%, outperforming the current state-of-the-art model IRv2 with soft attention on the same training and testing datasets. On the Edinburgh DERMOFIT dataset also, the method has better performance compared with baseline models.

Skin cancer risk self-assessment using AI as a mass screening tool

BackgroundSkin cancer is one of the most common cancers on the planet. The lack of noticeable symptoms, inaccessibility of professional dermatology expertise, and low population awareness prevent early detection, which is vital for successful treatment. MethodsWe proposed the free AI-powered self-examination tool allowing everyone to perform regular skin checkups with a smartphone's photo coupled with the new set of risk factors suitable for self-assessment. While most of the existing AI solutions focus on dermoscopic images and assisting staff in medical facilities, we in contrast designed the ensemble of deep convolutional neural networks to be used by non-specialists for assessing self-taken skin clinical images via mobile app. We aimed to distinguish between malignant and benign neoplasms only, encouraging people to follow up with dermatologists for an in-depth physical examination in case of discovered skin cancer risk. Results400000+ clinical skin images coupled with risk factors and demographic data have been collected, processed by AI, and validated by dermatology experts, which made it one of the biggest mass-screening campaigns for skin cancer disease in the Russian Federation. 9321 malignant cases were discovered including 5230 melanomas.The comprehensive analysis of mass-screening results was conducted for the following regions in Russian Federation: Moscow and Moscow region, Nizhny Novgorod, St. Petersburg and Leningradsky region, and Krasnodar region. We discovered the significantly higher melanoma cancer prevalence rate (number of cases per 100 000 people is 5.43 cases for males and 11.88 cases for females while statistical bulletin reports 4.12 cases in 2021) and much younger average age of skin cancer development (melanoma: 46.78 years for males and 41.35 years for females vs official numbers of 61.5 years and 62.1 years correspondingly; BCC/SCC: 56.28 years for males and 55.98 years for females vs official numbers of 68.2 years and 70.2 years correspondingly) comparing to statistical bulletins. Our findings demonstrate that the female population is at risk almost 5 years earlier and reveal the most common locations for malignant tumors: face, upper body, and shoulders. Our new skin cancer risk self-assessment criteria (20+ spots on arm and solarium visits) associate with the earlier melanoma development in the female population and should be considered in further skin cancer studies as risk factors. ConclusionThe proposed approach proved to be a cost-efficient way of doing mass skin cancer screenings, which have not been considered a feasible option before. The presented self-assessment risk factors along with the revised average age of malignant tumors development and prevalence rates should be considered in skin cancer surveillance.

MDFNet: application of multimodal fusion method based on skin image and clinical data to skin cancer classification.

Skin cancer is one of the ten most common cancer types in the world. Early diagnosis and treatment can effectively reduce the mortality of patients. Therefore, it is of great significance to develop an intelligent diagnosis system for skin cancer. According to the survey, at present, most intelligent diagnosis systems of skin cancer only use skin image data, but the multi-modal cross-fusion analysis using image data and patient clinical data is limited. Therefore, to further explore the complementary relationship between image data and patient clinical data, we propose multimode data fusion diagnosis network (MDFNet), a framework for skin cancer based on data fusion strategy. MDFNet establishes an effective mapping among heterogeneous data features, effectively fuses clinical skin images and patient clinical data, and effectively solves the problems of feature paucity and insufficient feature richness that only use single-mode data. The experimental results present that our proposed smart skin cancer diagnosis model has an accuracy of 80.42%, which is an improvement of about 9% compared with the model accuracy using only medical images, thus effectively confirming the unique fusion advantages exhibited by MDFNet. This illustrates that MDFNet can not only be applied as an effective auxiliary diagnostic tool for skin cancer diagnosis, help physicians improve clinical decision-making ability and effectively improve the efficiency of clinical medicine diagnosis, but also its proposed data fusion method fully exerts the advantage of information convergence and has a certain reference value for the intelligent diagnosis of numerous clinical diseases.

In vivo imaging with a fast large-area multiphoton exoscope (FLAME) captures the melanin distribution heterogeneity in human skin

Melanin plays a significant role in the regulation of epidermal homeostasis and photoprotection of human skin. The assessment of its epidermal distribution and overall content is of great interest due to its involvement in a wide range of physiological and pathological skin processes. Among several spectroscopic and optical imaging methods that have been reported for non-invasive quantification of melanin in human skin, the approach based on the detection of two-photon excited fluorescence lifetime distinguishes itself by enabling selective detection of melanin with sub-cellular resolution, thus facilitating its quantification while also resolving its depth-profile. A key limitation of prior studies on the melanin assessment based on this approach is their inability to account for the skin heterogeneity due to the reduced field of view of the images, which results in high dispersion of the measurement values. Pigmentation in both normal and pathological human skin is highly heterogeneous and its macroscopic quantification is critical for reliable measurements of the epidermal melanin distribution and for capturing melanin-related sensitive dynamic changes as a response to treatment. In this work, we employ a fast large-area multiphoton exoscope (FLAME), recently developed by our group for clinical skin imaging, that has the ability to evaluate the 3D distribution of epidermal melanin content in vivo macroscopically (millimeter scale) with microscopic resolution (sub-micron) and rapid acquisition rates (minutes). We demonstrate significant enhancement in the reliability of the melanin density and distribution measurements across Fitzpatrick skin types I to V by capturing the intra-subject pigmentation heterogeneity enabled by the large volumetric sampling. We also demonstrate the potential of this approach to provide consistent measurement results when imaging the same skin area at different times. These advances are critical for clinical and research applications related to monitoring pigment modulation as a response to therapies against pigmentary skin disorders, skin aging, as well as skin cancers.

Research Techniques Made Simple: Emerging Imaging Technologies for Noninvasive Optical Biopsy of Human Skin

Over the past few years, high-resolution optical imaging technologies such as optical coherence tomography (OCT), reflectance confocal microscopy (RCM), and multiphoton microscopy (MPM) have advanced significantly as new methodologies for clinical research and for real-time detection, diagnosis, and therapy monitoring of skin diseases. Implementation of these technologies into clinical research and practice requires clinicians to have an understanding of their capabilities, benefits, and limitations. This concise review provides insights on the application of OCT, RCM, and MPM for clinical skin imaging through images acquired invivo from the same lesions. The presented data are limited to pigmented lesions and basal cell carcinoma.

A Study on Radiation Dermatitis Grading Support System Based on Deep Learning by Hybrid Generation Method

Radiation dermatitis is one of the most common adverse events in patients undergoing radiotherapy. However, the objective evaluation of this condition is difficult to provide because the clinical evaluation of radiation dermatitis is made by visual assessment based on Common Terminology Criteria for Adverse Events (CTCAE). Therefore, we created a radiation dermatitis grading support system (RDGS) using a deep convolutional neural network (DCNN) and then evaluated the effectiveness of the RDGS. The DCNN was trained with a dataset that comprised 647 clinical skin images graded with radiation dermatitis (Grades 1-4) at our center from April 2011 to May 2019. We created the datasets by mixing data augmentation images generated by image conversion and images generated by Poisson image editing using the hybrid generation method (Hyb) against lowvolume severe dermatitis (Grade 4). We then evaluated the classification accuracy of RDGS based on the hybrid generation method (Hyb-RDGS). The overall accuracy of the Hyb-RDGS was 85.1%, which was higher than that of the data augmentation method generally used for image generation. Effectiveness of the Hyb-RDGS using Poisson image editing was suggested. This result shows a possible supporting system for objective evaluation in grading radiation dermatitis.

504 Deep learning empowered computer-assisted diagnosis on smartphone skin images

Nowadays deep learning has been widely applied in identifying skin disease using clinical skin images. Along with the development of smartphone and communication technology, telemedicine becomes more accessible for patients to consult with medical experts. However, rare research was done to prove the feasibility of using deep learning to predict skin conditions based on smartphone captured skin images, especially taken by non-professional patients. We collected 46946 skin images from the online patient-doctor telemedicine platform in mainland China. Each image was annotated by at least 2 dermatologists mutual blindly. We trained an assembly 6 different deep neural networks to provide predictions. The model is able to predict the probability among 30 common skin conditions, including infectious conditions, inflammatory conditions, pigmentary conditions, vascular conditions, and even tumors. The model achieves the top 1 and top 3 accuracies of 0.7708 and 0.9194. The coverage error of our model is 1.4171, which indicates the model has a short depth of ranked predictions to cover all the true labels. And label ranking average precision is 0.9392, which means the model has a high fraction of higher-ranked predictions mapping with ground truth labels. Furthermore, the results show the ability to differentiate some skin conditions which have similar morphology by using a deep learning approach on patient captured skin images. In conclusion, a deep learning approach would potentially provide not only initial evaluation of skin lesions to patients but also clues for differential diagnosis to general doctors through smartphone skin photography.

Gradient-based 3D skin roughness rendering from an in-vivo skin image for dynamic haptic palpation.

Many studies have revealed the importance of palpation for dermatologists; however, palpation is not always possible due to the risk of secondary infections or the risk of damaging the affected area. Thus, haptic rendering for indirect palpation using in-vivo skin images, which will enable to examine a real three-dimensional (3D) skin sample by virtual touch without directly palpating the infected skin area, could be a useful technology in dermatology. We propose a new method of accurate 3D skin surface reconstruction using simple gradients from a single skin image for accurate 3D roughness rendering with a haptic device. Our approach takes advantage of bilateral filtering to preserve skin roughness and image gradients in order to generate a 3D skin surface (polygonal meshes) while preserving skin wrinkles and rough surface textures. Our method was evaluated using two experiments. The accuracy was tested with six 3D ground-truth surfaces and four clinical skin images (acne, miliaria, sweet syndrome, and herpes simplex). For objective evaluation, a well-known 3D roughness estimation method and the Hausdorff distance were adopted to compute errors. All results showed that the accuracy of our method is superior to that of the two existing methods. Haptic roughness rendering for skin palpation examination requires efficient and accurate 3D surface reconstruction. In this study, we developed a new method that can be used to reconstruct a 3D skin surface accurately while preserving roughness through the use of a single skin image.

Skin lesion classification using relative color features

Clinically, it is difficult to differentiate the early stage of malignant melanoma and certain benign skin lesions due to similarity in appearance. Our research uses image analysis of clinical skin images and relative color-based pattern recognition techniques to enhance the images and improve differentiation of these lesions. First, the relative color images were created from digitized photographic images. Then, they were segmented into objects and morphologically filtered. Next, the relative color features were extracted from the objects to form two different feature spaces; a lesion feature space and an object feature space. The two feature spaces serve as two data models to be analyzed to determine the best features. Finally, we used a statistical analysis model of relative color features, which better classifies the various types of skin lesions. The best features found for differentiation of melanoma and benign skin lesions from this study are area, mean in the red and blue bands, standard deviation in the red and green bands, skewness in the green band, and entropy in the red band. With the relative color feature algorithm developed, the best results were obtained with a multi-layer perceptron neural network model. This showed an overall classification success of 79%, with 70% of the benign lesions successfully classified, and 86% of malignant melanoma successfully classified.