Automatic Segmentation Algorithm of Dermoscopy Image Based on Transformer and Convolutional Neural Network

Abstract

The lesion area of the dermoscopic image is highly similar to the background pixels, and there are various shapes, blurred edges, artificial or hair occlusion, etc. In order to obtain higher-precision segmentation of skin lesions, a automatic segmentation algorithm for dermoscopic images is proposed in this paper. Firstly, ResNet 34 is used to extract multiple resolution features, and the Transformer module is used to model globally the input features in the context part. Secondly, the multi-scale information of context features is aggregated through the Mixed Pooling Module. In addition, an efficient convolution module is designed between jump links of the corresponding codec modules to improve the edge refinement and anti-interference ability of the jump path. Finally, the decoder is used to restore the image resolution and fuse other shallow resolution features information, and using Focal Loss function to improve the accuracy of segment targets. The Dice coefficient, accuracy, Jackard index and sensitivity scores obtained by our method on the ISIC 2017 and ISIC 2018 datasets are 88.83%, 94.77%, 81.43%, 88.49% and 89.46%, 94.50%, 82.56% and 94.62% respectively. Compared with other algorithms, our method has certain advantages, proving its effectiveness.