TriClick: Interactive Dermoscopic Image Segmentation with Triangle Map

Abstract

AbstractLesion segmentation is a fundamental task that has been widely studied in biomedicine. State‐of‐the‐art methods still struggle with the inherent challenges of dermoscopic images such as uncertain boundaries or severe occlusion. In this study, we argue that this task can be better resolved by introducing user interaction and proposing a deep interactive framework for lesion segmentation. Our method is designed to best reflect user expertise to iteratively refine the segmentation results. In each iteration, professional users are expected to click three points on the boundary of the lesion to encourage the network to update the previous round of the segmentation result. To best exploit the clues from the interaction, we explore a novel encoding strategy by producing a triangle map whose vertices are the three clicked points. Owing to the strong guidance within such triangle maps, our method can outperform state‐of‐the‐art methods with baseline segmentation backbones. Extensive experimental results and ablation studies on three commonly used lesion segmentation data sets demonstrated the effectiveness of our method in terms of segmentation accuracy and encoding effectiveness. © 2024 The Authors. IEEJ Transactions on Electrical and Electronic Engineering published by Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.