Inferring Skin Lesion Segmentation With Fully Connected CRFs Based on Multiple Deep Convolutional Neural Networks

Yuming Qiu,Ju Zhang,Xiaolin Qin,Jingyong Cai

doi:10.1109/access.2020.3014787

Abstract

This article presents a method to infer skin lesion segmentation based on multiple deep convolutional neural network (DCNN) models by employing fully connected conditional random fields (CRFs). This method is on the strength of the synergism between ensemble learning which is responsible for introducing diversity from multiple DCNN models and CRFs inference which is in charge of probabilistic inference based on random fields over dermoscopy images. Contrasting to single DCNN models, the proposed method can gain better segmentation by comprehensively utilizing the advances and performance preferences of multiple different DCNN models. In comparison with simple ensemble schemes, it can effectively and precisely refine the fuzzy lesion boundary by utilizing the information in test images to maximize label agreement between similar pixels. Further, an engineering bonus is the feasibility of parallelization for the heavy operation, predicting on multiple DCNN models. In experiments, we tested the effectiveness and robustness of the proposed method on the mainstream datasets ISIC 2017 and PH2, and the results were competitive with the state-of-art methods. we also confirmed that the proposed method can capture the local information in fuzzy dermoscopy images being able to find more accurate lesion borders with a good boost on Boundary Recall (BR) metric. Moreover, since the hyper-parameters in CRFs are explainable, it is possible to adjust them manually to reach better results case by case, being attractive in practice. This work is of value on integration between the deep learning technologies and probabilistic inference in resolving lesion segmentation, and has great potential to be applied in similar tasks.

Highlights

Pigmented skin lesion border structure provides valuable information and clinical features such as asymmetry and irregularity for accurate diagnosis, and lesion borders are helpful for extracting other clinical features such as atypical pigment networks, blue-white areas and dots [1]–[5]
The experimental results showed that the performance of proposed method on International Skin Imaging Collaboration (ISIC) 2017 and PH2 exceeded baselines on most of all metrics, especially being 5.57% higher than voting ensemble, 4.76% higher than the best score in single deep convolutional neural network (DCNN) models and 7.59% higher than single DCNN models plus conditional random fields (CRFs) on ISIC 2017 on mean thresholded Jaccard index (mTJI) metric, the newest metric used in ISIC 2018 [22]
We propose a feasible and effective method to infer skin lesion segmentation with fully connect CRFs based on multiple DCNNs

Summary

Introduction

Pigmented skin lesion border structure provides valuable information and clinical features such as asymmetry and irregularity for accurate diagnosis, and lesion borders are helpful for extracting other clinical features such as atypical pigment networks, blue-white areas and dots [1]–[5]. Many attentions have turned to employment of various deep convolutional neural network (DCNN) methods such as FCNs [8], DeepLab [9], [10] and Mask R-CNN [11], and have made enormous progress [12]–[16] in lesion segmentation. It is known that deep neural network architectures vary in characteristics, strengths and weakness, even different hyper-parameters or initialization parameters in a same DCNN architecture may lead to different segmentation, so their segmentation results from a model may have different emphasis on different aspects, called performance preferences here. Single DCNN models usually perform unstably caused by different model architectures and/or hyper-parameters. A promising direction is to integrate all segmentation from different DCNN models to improve the final segmentation

Objectives

Methods

Findings

Discussion

Conclusion