Attention Residual Learning for Skin Lesion Classification.

Currently, convolutional neural networks (CNNs) have made remarkable achievements in skin lesion classification because of their end-to-end feature representation abilities. However, precise skin lesion classification is still challenging because of the following three issues: (1) insufficient training samples, (2) inter-class similarities and intra-class variations, and (3) lack of the ability to focus on discriminative skin lesion parts. To address these issues, we propose a deep metric attention learning CNN (DeMAL-CNN) for skin lesion classification. In DeMAL-CNN, a triplet-based network (TPN) is first designed based on deep metric learning, which consists of three weight-shared embedding extraction networks. TPN adopts a triplet of samples as input and uses the triplet loss to optimize the embeddings, which can not only increase the number of training samples, but also learn the embeddings robust to inter-class similarities and intra-class variations. In addition, a mixed attention mechanism considering both the spatial-wise and channel-wise attention information is designed and integrated into the construction of each embedding extraction network, which can further strengthen the skin lesion localization ability of DeMAL-CNN. After extracting the embeddings, three weight-shared classification layers are used to generate the final predictions. In the training procedure, we combine the triplet loss with the classification loss as a hybrid loss to train DeMAL-CNN. We compare DeMAL-CNN with the baseline method, attention methods, advanced challenge methods, and state-of-the-art skin lesion classification methods on the ISIC 2016 and ISIC 2017 datasets, and test its generalization ability on the PH2 dataset. The results demonstrate its effectiveness.

Background: Using artificial intelligence (AI) with the concept of a deep learning-based automated computer-aided diagnosis (CAD) system has shown improved performance for skin lesion classification. Although deep convolutional neural networks (DCNNs) have significantly improved many image classification tasks, it is still difficult to accurately classify skin lesions because of a lack of training data, inter-class similarity, intra-class variation, and the inability to concentrate on semantically significant lesion parts. Innovations: To address these issues, we proposed an automated deep learning and best feature selection framework for multiclass skin lesion classification in dermoscopy images. The proposed framework performs a preprocessing step at the initial step for contrast enhancement using a new technique that is based on dark channel haze and top–bottom filtering. Three pre-trained deep learning models are fine-tuned in the next step and trained using the transfer learning concept. In the fine-tuning process, we added and removed a few additional layers to lessen the parameters and later selected the hyperparameters using a genetic algorithm (GA) instead of manual assignment. The purpose of hyperparameter selection using GA is to improve the learning performance. After that, the deeper layer is selected for each network and deep features are extracted. The extracted deep features are fused using a novel serial correlation-based approach. This technique reduces the feature vector length to the serial-based approach, but there is little redundant information. We proposed an improved anti-Lion optimization algorithm for the best feature selection to address this issue. The selected features are finally classified using machine learning algorithms. Main Results: The experimental process was conducted using two publicly available datasets, ISIC2018 and ISIC2019. Employing these datasets, we obtained an accuracy of 96.1 and 99.9%, respectively. Comparison was also conducted with state-of-the-art techniques and shows the proposed framework improved accuracy. Conclusions: The proposed framework successfully enhances the contrast of the cancer region. Moreover, the selection of hyperparameters using the automated techniques improved the learning process of the proposed framework. The proposed fusion and improved version of the selection process maintains the best accuracy and shorten the computational time.

Background and objectiveEarly detection and classification of skin cancer are critical for improving patient outcomes. Dermoscopic image analysis using Computer-Aided Diagnostics (CAD) is a powerful tool to assist dermatologists in identifying and classifying skin lesions. Traditional machine learning models require extensive feature engineering, which is time-consuming and less effective in handling complex data like skin lesions. This study proposes a deep learning-based network-level fusion architecture that integrates multiple deep models to enhance the classification and localization of skin lesions in dermoscopic images. The goal is to address challenges like irregular lesion shapes, inter-class similarities, and class imbalances while providing explainability through artificial intelligence.MethodsA novel hybrid contrast enhancement technique was applied for pre-processing and dataset augmentation. Two deep learning models, a 5-block inverted residual network and a 6-block inverted bottleneck network, were designed and fused at the network level using a depth concatenation approach. The models were trained using Bayesian optimization for hyperparameter tuning. Feature extraction was performed with a global average pooling layer, and shallow neural networks were used for final classification. Explainable AI techniques, including LIME, were used to interpret model predictions and localize lesion regions. Experiments were conducted on two publicly available datasets, HAM10000 and ISIC2018, which were split into training and testing sets.ResultsThe proposed fused architecture achieved high classification accuracy, with results of 91.3% and 90.7% on the HAM10000 and ISIC2018 datasets, respectively. Sensitivity, precision, and F1-scores were significantly improved after data augmentation, with precision rates of up to 90.91%. The explainable AI component effectively localized lesion areas with high confidence, enhancing the model’s interpretability.ConclusionsThe network-level fusion architecture combined with explainable AI techniques significantly improved the classification and localization of skin lesions. The augmentation and contrast enhancement processes enhanced lesion visibility, while fusion of models optimized classification accuracy. This approach shows potential for implementation in CAD systems for skin cancer diagnosis, although future work is required to address the limitations of computational resource requirements and training time.Clinical trail numberNot applicable.

Precise skin lesion classification is still challenging due to two problems, i.e., (1) inter-class similarity and intra-class variation of skin lesion images, and (2) the weak generalization ability of single Deep Convolutional Neural Network trained with limited data. Therefore, we propose a Global-Part Convolutional Neural Network (GP-CNN) model, which treats the fine-grained local information and global context information with equal importance. The Global-Part model consists of a Global Convolutional Neural Network (G-CNN) and a Part Convolutional Neural Network (P-CNN). Specifically, the G-CNN is trained with downscaled dermoscopy images, and is used to extract the global-scale information of dermoscopy images and produce the Classification Activation Map (CAM). While the P-CNN is trained with the CAM guided cropped image patches and is used to capture local-scale information of skin lesion regions. Additionally, we present a data-transformed ensemble learning strategy, which can further boost the classification performance by integrating the different discriminant information from GP-CNNs that are trained with original images, color constancy transformed images, and feature saliency transformed images, respectively. The proposed method is evaluated on the ISIC 2016 and ISIC 2017 Skin Lesion Challenge (SLC) classification datasets. Experimental results indicate that the proposed method can achieve the state-of-the-art skin lesion classification performance (i.e., an AP value of 0.718 on the ISIC 2016 SLC dataset and an Average Auc value of 0.926 on the ISIC 2017 SLC dataset) without any external data, compared with other current methods which need to use external data.

Fusing fine-tuned deep features for skin lesion classification

Early and accurate classification of skin lesions is critical for effective skin cancer diagnosis and treatment. However, the visual similarity of lesions in their early stages often leads to misdiagnoses and delayed interventions. This lack of transparency makes it challenging for dermatologists to interpret with validate decisions made by such methods, reducing their trust in the system. To overcome these complications, Skin Lesions Classification in Dermoscopic Images using Optimized Dynamic Graph Convolutional Recurrent Imputation Network (SLCDI-DGCRIN-RBBMOA) is proposed. The input image is pre-processed utilizing Confidence Partitioning Sampling Filtering (CPSF) to remove noise, resize, and enhance image quality. By using the Hybrid Dual Attention-guided Efficient Transformer and UNet 3+ (HDAETUNet3+) it segment ROI region of the preprocessed dermoscopic images. Finally, segmented images are fed to Dynamic Graph Convolutional Recurrent Imputation Network (DGCRIN) for classifying skin lesion as actinic keratosis, dermatofibroma, basal cell carcinoma, squamous cell carcinoma, benign keratosis, vascular lesion, melanocytic nevus, and melanoma. Generally, DGCRIN does not express any adaption of optimization strategies for determining optimal parameters to exact skin lesion classification. Hence, Red Billed Blue Magpie Optimization Algorithm (RBBMOA) is proposed to enhance DGCRIN that can exactly classify type of skin lesion. The proposed SLCDI-DGCRIN-RBBMOA technique attains 26.36%, 20.69% and 30.29% higher accuracy, 19.12%, 28.32%, and 27.84% higher precision, 12.04%, 13.45% and 22.80% higher recall and 20.47%, 16.34%, and 20.50% higher specificity compared with existing methods such as a deep learning method dependent on explainable artificial intelligence for skin lesion classification (DNN-EAI-SLC), multiclass skin lesion classification utilizing deep learning networks optimal information fusion (MSLC-CNN-OIF), and classification of skin cancer from dermoscopic images utilizing deep neural network architectures (CSC-DI-DCNN) respectively.

A comprehensive analysis of dermoscopy images for melanoma detection via deep CNN features

Transfer learning using a multi-scale and multi-network ensemble for skin lesion classification

Automated multi-class classification of skin lesions through deep convolutional neural network with dermoscopic images.

Automated skin lesion classification in the dermoscopy images is an essential way to improve diagnostic performance and reduce melanoma deaths. Although deep learning has shown proven advantages over traditional methods, which rely on handcrafted features, in image classification, it remains challenging to classify skin lesions due to the significant intra-class variation and inter-class similarity. In this paper, we propose a synergic deep learning (SDL) model to address this issue, which not only uses dual deep convolutional neural networks (DCNNs) but also enables them to mutually learn from each other. Specifically, we concatenate the image representation learned by both DCNNs as the input of a synergic network, which has a fully connected structure and predicts whether the pair of input images belong to the same class. We train the SDL model in the end-to-end manner under the supervision of the classification error in each DCNN and the synergic error. We evaluated our SDL model on the ISIC 2016 Skin Lesion Classification dataset and achieved the state-of-the-art performance.

Skin cancer is among the most common cancer types. Dermoscopic image analysis improves the diagnostic accuracy for detection of malignant melanoma and other pigmented skin lesions when compared to unaided visual inspection. Hence, computer-based methods to support medical experts in the diagnostic procedure are of great interest. Fine-tuning pre-trained convolutional neural networks (CNNs) has been shown to work well for skin lesion classification. Pre-trained CNNs are typically trained with natural images of a fixed image size significantly smaller than captured skin lesion images and consequently dermoscopic images are downsampled for fine-tuning. However, useful medical information may be lost during this transformation. In this paper, we explore the effect of input image size on skin lesion classification performance of fine-tuned CNNs. For this, we resize dermoscopic images to different resolutions, ranging from 64 x 64 to 768 x 768 pixels and investigate the resulting classification performance of three well-established CNNs, namely DenseNet-121, ResNet-18, and ResNet-50. Our results show that using very small images (of size 64 x 64 pixels) degrades the classification performance, while images of size 128 x 128 pixels and above support good performance with larger image sizes leading to slightly improved classification. We further propose a novel fusion approach based on a three-level ensemble strategy that exploits multiple fine-tuned networks trained with dermoscopic images at various sizes. When applied on the ISIC 2017 skin lesion classification challenge, our fusion approach yields an area under the receiver operating characteristic curve of 89.2% and 96.6% for melanoma classification and seborrheic keratosis classification, respectively, outperforming state-of-the-art algorithms.

Dermoscopic image classification using CNN with Handcrafted features

BackgroundAdvances in biomedical artificial intelligence may introduce or perpetuate sex and gender discriminations. Convolutional neural networks (CNN) have proven a dermatologist-level performance in image classification tasks but have not been assessed for sex and gender biases that may affect training data and diagnostic performance. In this study, we investigated sex-related imbalances in training data and diagnostic performance of a market-approved CNN for skin cancer classification (Moleanalyzer Pro®, Fotofinder Systems GmbH, Bad Birnbach, Germany). MethodsWe screened open-access dermoscopic image repositories widely used for CNN training for distribution of sex. Moreover, the sex-related diagnostic performance of the market-approved CNN was tested in 1549 dermoscopic images stratified by sex (female n = 773; male n = 776). ResultsMost open-access repositories showed a marked under-representation of images originating from female (40%) versus male (60%) patients. Despite these imbalances and well-known sex-related differences in skin anatomy or skin-directed behaviour, the tested CNN achieved a comparable sensitivity of 87.0% [80.9%–91.3%] versus 87.1% [81.1%–91.4%], specificity of 98.7% [97.4%–99.3%] versus 96.9% [95.2%–98.0%] and ROC-AUC of 0.984 [0.975–0.993] versus 0.979 [0.969–0.988] in dermoscopic images of female versus male origin, respectively. In the sample at hand, sex-related differences in ROC-AUCs were not statistically significant in the per-image analysis nor in an additional per-individual analysis (p ≥ 0.59). ConclusionDesign and training of artificial intelligence algorithms for medical applications should generally acknowledge sex and gender dimensions. Despite sex-related imbalances in open-access training data, the diagnostic performance of the tested CNN showed no sex-related bias in the classification of skin lesions.

Attention learning models using local Zernike moments-based normalized images and convolutional neural networks for skin lesion classification

Automatic skin lesion classification based on mid-level feature learning