YoTransViT: A transformer and CNN method for predicting and classifying skin diseases using segmentation techniques

Abstract

Skin disease cases are becoming more common, and diagnosing these diseases in a clinic is never an easy task. A deep learning (DL) based model was previously used to diagnose skin-related disorders; this may help to address these problems. But lately, the transformer model has become more well-liked and effective for computer vision applications such as image identification and skin condition detection. Consequently, vision transformers (ViT), which have shown exceptional performance in conventional classification tasks, were used in the present study. The goal of self-attention is to enhance the significance of key elements while muting distracting ones. More specifically, YoTransViT, an enhanced transformer network, is suggested. The suggested ViT structures with image augmentation and segmentation methods were applied to the ISIC 2019 dataset. We also assessed the effectiveness of our suggested method using an alternative transformer model, the swin transformer (SwinViT). In addition to some of the most well-known conventional neural network (CNN) designs, such as MobileNetV2, InceptionResNetV2 and DenseNet201 hence creating a baseline for our suggested framework through comparison. Eight different types of dermatological issues are used in this study. Various metrics for performance evaluation are computed to confirm the effectiveness of our method. The results demonstrated the efficiency of our suggested methodology, as ViT used image segmentation and augmentation approaches to reach a classification accuracy of 99.97% and precision of 100%. The transformer network has shown remarkable performance in this system for predicting skin diseases and has the potential to be extremely important in the field of computer vision. Lastly, we established and put into practice a web-based architecture called YoTransViT for the real-time prediction of skin diseases.