EH-former: Regional easy-hard-aware transformer for breast lesion segmentation in ultrasound images

Abstract

Breast lesion segmentation of ultrasound images plays a crucial role in early screening and diagnosis of breast lesions. However, accurately segmenting lesions in breast ultrasound (BUS) images is challenging due to prevalent issues such as low contrast, intense speckle noise, and blurred lesion boundaries. Although existing deep learning-based segmentation models have made significant progress, few have strategically addressed these complex and noisy regional features in BUS images. The easy-to-hard manner in Curriculum Learning (CL) appears promising, but it often remains at the sample level and does not adequately address regional complexities. To address this, we design a region-wise CL to dynamically adjust the focus on hard regional features in BUS images. Specifically, we propose a Regional Easy-Hard-Aware Transformer (EH-Former), structured in two stages for lesion segmentation in BUS images. The first stage incorporates uncertainty estimation for dividing regional difficulty. In the second stage, we propose a novel Adaptive Easy-Hard region Separator (AdaSep), a module employing uncertainty-aware regularization to separate features of varying difficulties, allowing the two streams within EH-Former to focus on learning regional features of different complexities. Additionally, we develop a Dynamic Easy-Hard Feature Fusion (D-Fusion) module, dynamically adjusting the fusion weight of easy and hard regional features based on the current training epoch to achieve progressive regional feature learning. Extensive experimental results on five public datasets show that the proposed EH-Former consistently outperforms state-of-the-art methods in most metrics and exhibits better domain generalization capabilities. Furthermore, our region-wise CL significantly enhances the performance of EH-Former in detecting complex tissue structures and noisy areas that are challenging to segment accurately. The source code is available at https://github.com/lele0109/EH-Former.