Breast Ultrasound Image Segmentation Research Articles

Dual-Stream CoAtNet models for accurate breast ultrasound image segmentation

Dual-Stream CoAtNet models for accurate breast ultrasound image segmentation

MFMSNet: A Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network for breast ultrasound image segmentation

Breast tumor segmentation in ultrasound images is fundamental for quantitative analysis and plays a crucial role in the diagnosis and treatment of breast cancer. Recently, existing methods have mainly focused on spatial domain implementations, with less attention to the frequency domain. In this paper, we propose a Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network (MFMSNet). Specifically, we utilize Octave convolutions instead of conventional convolutions to effectively separate high-frequency and low-frequency components while reducing computational complexity. Introducing the Multi-frequency Transformer block (MF-Trans) enables efficient interaction between high-frequency and low-frequency information, thereby capturing long-range dependencies. Additionally, we incorporate Multi-scale interactive fusion module (MSIF) to merge high-frequency feature maps of different sizes, enhancing the emphasis on tumor edges by integrating local contextual information. Experimental results demonstrate the superiority of our MFMSNet over seven state-of-the-art methods on two publicly available breast ultrasound datasets and one thyroid ultrasound dataset. In the evaluation of MFMSNet, tests were conducted on the BUSI, BUI, and DDTI datasets, comprising 130 images (BUSI), 47 images (BUI), and 128 images (DDTI) in the respective test sets. Employing a five-fold cross-validation approach, the obtained dice coefficients are as follows: 83.42 % (BUSI), 90.79 % (BUI), and 79.96 % (DDTI). The code is available at https://github.com/wrc990616/MFMSNet.

DBL-Net: A dual-branch learning network with information from spatial and frequency domains for tumor segmentation and classification in breast ultrasound image

Breast cancer is a common malignancy, ranking as one of the most prevalent cancers among women globally. Improving treatment success rates for breast cancer are crucially dependent on early detection and accurate diagnosis. Ultrasound imaging, as a non-invasive and low-radiation imaging technique, plays a significant role in early breast cancer detection. However, due to the characteristics of fuzzy tumor boundaries and irregular shapes of the tumor, accurate breast tumor classification and segmentation remain challenging tasks. This paper introduces a novel approach for breast ultrasound image segmentation and classification tasks. Firstly, we study the potential value of frequency domain information in breast image analysis and propose the DBL-Net, a multi-task learning network for simultaneous tumor segmentation and tumor classification. Secondly, we customize a Spatial-Frequency Domain Feature Encoding module (S-FEM) to better encode and fuse spatial and frequency domain features of breast ultrasound images. Furthermore, we design a Representational Perception Enhancer (RPE) to improve the feature representation capabilities and prediction accuracy of the model. Finally, We have carried out a comprehensive series of tests on the BUS dataset, and the findings clearly indicate that our method outperforms the current state-of-the-art methods.

SC-Unext: A Lightweight Image Segmentation Model with Cellular Mechanism for Breast Ultrasound Tumor Diagnosis.

Automatic breast ultrasound image segmentation plays an important role in medical image processing. However, current methods for breast ultrasound segmentation suffer from high computational complexity and large model parameters, particularly when dealing with complex images. In this paper, we take the Unext network as a basis and utilize its encoder-decoder features. And taking inspiration from the mechanisms of cellular apoptosis and division, we design apoptosis and division algorithms to improve model performance. We propose a novel segmentation model which integrates the division and apoptosis algorithms and introduces spatial and channel convolution blocks into the model. Our proposed model not only improves the segmentation performance of breast ultrasound tumors, but also reduces the model parameters and computational resource consumption time. The model was evaluated on the breast ultrasound image dataset and our collected dataset. The experiments show that the SC-Unext model achieved Dice scores of 75.29% and accuracy of 97.09% on the BUSI dataset, and on the collected dataset, it reached Dice scores of 90.62% and accuracy of 98.37%. Meanwhile, we conducted a comparison of the model's inference speed on CPUs to verify its efficiency in resource-constrained environments. The results indicated that the SC-Unext model achieved an inference speed of 92.72ms per instance on devices equipped only with CPUs. The model's number of parameters and computational resource consumption are 1.46M and 2.13 GFlops, respectively, which are lower compared to other network models. Due to its lightweight nature, the model holds significant value for various practical applications in the medical field.

Edge-guided adaptive multi-feature learning for breast ultrasound image segmentation

An Edge-guided Adaptive Multi-feature Learning Network (EAML-Net) for lesion segmentation is proposed in this paper to address the issues of blurred edges, high speckle noise, low contrast and serious ultrasound artifacts in breast ultrasound images. EAML-Net consists of an encoder, a decoder and an Edge Guidance Module (EGM). Firstly, in the encoder, an Adaptive Residual Multi-feature Learning (ARML) module and a Mixed Pooling (MP) layer are utilized to mitigate the problems of speckle noise and low contrast. The former is used to adaptively obtain multi-resolution features, while the latter can guarantee the efficient transfer of those features between different layers. In addition, they can improve the segmentation accuracy of small targets. Then, the decoder gradually restores features to the original image size by upsampling. Furthermore, the EGM is explored to alleviate the problem of blurred edges and refine the segmentation results. In EGM, an Information Fusion Block (IFB) is used to extract and fuse those features located in the encoder and then detect them by an edge detection method to guide the network to learn more abundant edge information. Finally, the learning ability of EAML-Net is further accelerated with the supervision of a hybrid loss function so as to obtain superior segmentation performance. Experiment results on two public breast ultrasound datasets demonstrate that EAML-Net achieves 86.82% and 84.58% of the Dice index, respectively. Moreover, comparison and ablation experiment results further demonstrate the effectiveness of the proposed algorithm.

Deep learning-based segmentation of breast masses using convolutional neural networks

Automatic breast tumor segmentation based on convolutional neural networks (CNNs) is significant for the diagnosis and monitoring of breast cancers. CNNs have become an important method for early diagnosis of breast cancer and, thus, can help decrease the mortality rate. In order to assist medical professionals in breast cancer investigation a computerized system based on two encoder-decoder architectures for breast tumor segmentation has been developed. Two pre-trained DeepLabV3+ and U-Net models are proposed. The encoder generates a high-dimensional feature vector while the decoder analyses the low-resolution feature vector provided by the encoder and generates a semantic segmentation mask. Semantic segmentation based on deep learning techniques can overcome the limitations of traditional algorithms. To assess the efficiency of breast ultrasound image segmentation, we compare the segmentation results provided by CNNs against the Local Graph Cut technique (a semi-automatic segmentation method) in the Image Segmenter application. The output segmentation results have been evaluated by using the Dice similarity coefficient that compares the ground truth images provided by the specialists against the predicted segmentation results provided by the CNNs and Local Graph Cut algorithm. The proposed approach is validated on 780 breast ultrasonographic images of the BUSI public database of which 437 are benign and 210 are malignant. The BUSI database provides classification (benign or malignant) labels for ground truth in binary mask images. The average Dice scores computed between the ground truth images against CNNs were as follows: 0.9360 (malignant) and 0.9325 (benign) for the DeepLabV3+ architecture and of 0.6251 (malignant) and 0.6252 (benign) for the U-Net, respectively. When the segmentation results provided by CNNs were compared with the Local Graph Cut segmented images, the Dice scores were 0.9377 (malignant) and 0.9204 (benign) for DeepLabV3+ architecture and 0.6115 (malignant) and 0.6119 (benign) for U-Net, respectively. The results show that the DeepLabV3+ has significantly better segmentation performance and outperforms the U-Net network.

Attention gate and dilation U-shaped network (GDUNet): an efficient breast ultrasound image segmentation network with multiscale information extraction.

In recent years, computer-aided diagnosis (CAD) systems have played an important role in breast cancer screening and diagnosis. The image segmentation task is the key step in a CAD system for the rapid identification of lesions. Therefore, an efficient breast image segmentation network is necessary for improving the diagnostic accuracy in breast cancer screening. However, due to the characteristics of blurred boundaries, low contrast, and speckle noise in breast ultrasound images, breast lesion segmentation is challenging. In addition, many of the proposed breast tumor segmentation networks are too complex to be applied in practice. We developed the attention gate and dilation U-shaped network (GDUNet), a lightweight, breast lesion segmentation model. This model improves the inverted bottleneck, integrating it with tokenized multilayer perceptron (MLP) to construct the encoder. Additionally, we introduce the lightweight attention gate (AG) within the skip connection, which effectively filters noise in low-level semantic information across spatial and channel dimensions, thus attenuating irrelevant features. To further improve performance, we innovated the AG dilation (AGDT) block and embedded it between the encoder and decoder in order to capture critical multiscale contextual information. We conducted experiments on two breast cancer datasets. The experiment's results show that compared to UNet, GDUNet could reduce the number of parameters by 10 times and the computational complexity by 58 times while providing a double of the inference speed. Moreover, the GDUNet achieved a better segmentation performance than did the state-of-the-art medical image segmentation architecture. Our proposed GDUNet method can achieve advanced segmentation performance on different breast ultrasound image datasets with high efficiency.

Swin-Net: A Swin-Transformer-Based Network Combing with Multi-Scale Features for Segmentation of Breast Tumor Ultrasound Images

Breast cancer is one of the most common cancers in the world, especially among women. Breast tumor segmentation is a key step in the identification and localization of the breast tumor region, which has important clinical significance. Inspired by the swin-transformer model with powerful global modeling ability, we propose a semantic segmentation framework named Swin-Net for breast ultrasound images, which combines Transformer and Convolutional Neural Networks (CNNs) to effectively improve the accuracy of breast ultrasound segmentation. Firstly, our model utilizes a swin-transformer encoder with stronger learning ability, which can extract features of images more precisely. In addition, two new modules are introduced in our method, including the feature refinement and enhancement module (RLM) and the hierarchical multi-scale feature fusion module (HFM), given that the influence of ultrasonic image acquisition methods and the characteristics of tumor lesions is difficult to capture. Among them, the RLM module is used to further refine and enhance the feature map learned by the transformer encoder. The HFM module is used to process multi-scale high-level semantic features and low-level details, so as to achieve effective cross-layer feature fusion, suppress noise, and improve model segmentation performance. Experimental results show that Swin-Net performs significantly better than the most advanced methods on the two public benchmark datasets. In particular, it achieves an absolute improvement of 1.4–1.8% on Dice. Additionally, we provide a new dataset of breast ultrasound images on which we test the effect of our model, further demonstrating the validity of our method. In summary, the proposed Swin-Net framework makes significant advancements in breast ultrasound image segmentation, providing valuable exploration for research and applications in this domain.

Multi-task approach based on combined CNN-transformer for efficient segmentation and classification of breast tumors in ultrasound images

Nowadays, inspired by the great success of Transformers in Natural Language Processing, many applications of Vision Transformers (ViTs) have been investigated in the field of medical image analysis including breast ultrasound (BUS) image segmentation and classification. In this paper, we propose an efficient multi-task framework to segment and classify tumors in BUS images using hybrid convolutional neural networks (CNNs)-ViTs architecture and Multi-Perceptron (MLP)-Mixer. The proposed method uses a two-encoder architecture with EfficientNetV2 backbone and an adapted ViT encoder to extract tumor regions in BUS images. The self-attention (SA) mechanism in the Transformer encoder allows capturing a wide range of high-level and complex features while the EfficientNetV2 encoder preserves local information in image. To fusion the extracted features, a Channel Attention Fusion (CAF) module is introduced. The CAF module selectively emphasizes important features from both encoders, improving the integration of high-level and local information. The resulting feature maps are reconstructed to obtain the segmentation maps using a decoder. Then, our method classifies the segmented tumor regions into benign and malignant using a simple and efficient classifier based on MLP-Mixer, that is applied for the first time, to the best of our knowledge, for the task of lesion classification in BUS images. Experimental results illustrate the outperformance of our framework compared to recent works for the task of segmentation by producing 83.42% in terms of Dice coefficient as well as for the classification with 86% in terms of accuracy.

Enhancing breast ultrasound segmentation through fine-tuning and optimization techniques: Sharp attention UNet.

Segmentation of breast ultrasound images is a crucial and challenging task in computer-aided diagnosis systems. Accurately segmenting masses in benign and malignant cases and identifying regions with no mass is a primary objective in breast ultrasound image segmentation. Deep learning (DL) has emerged as a powerful tool in medical image segmentation, revolutionizing how medical professionals analyze and interpret complex imaging data. The UNet architecture is a highly regarded and widely used DL model in medical image segmentation. Its distinctive architectural design and exceptional performance have made it popular among researchers. With the increase in data and model complexity, optimization and fine-tuning models play a vital and more challenging role than before. This paper presents a comparative study evaluating the effect of image preprocessing and different optimization techniques and the importance of fine-tuning different UNet segmentation models for breast ultrasound images. Optimization and fine-tuning techniques have been applied to enhance the performance of UNet, Sharp UNet, and Attention UNet. Building upon this progress, we designed a novel approach by combining Sharp UNet and Attention UNet, known as Sharp Attention UNet. Our analysis yielded the following quantitative evaluation metrics for the Sharp Attention UNet: the Dice coefficient, specificity, sensitivity, and F1 score values obtained were 0.93, 0.99, 0.94, and 0.94, respectively. In addition, McNemar's statistical test was applied to assess significant differences between the approaches. Across a number of measures, our proposed model outperformed all other models, resulting in improved breast lesion segmentation.

Learning active contour models based on self-attention for breast ultrasound image segmentation

Computer-aided diagnosis (CAD) systems based on ultrasound have been developed and widely promoted in breast cancer screening. Due to the characteristics of low contrast and speckle noises, breast ultrasound image segmentation, one of the crucial steps of CAD systems, has always been challenging. Recently, the emerging Transformer-based medical segmentation methods, which have a better ability to model long dependencies than convolutional neural networks (CNNs), have shown significant value for medical image segmentation. However, due to the limited data with the high-quality label, Transformer performs weakly on breast ultrasound image segmentation without pretraining. Thus, we propose the Attention-Gate Medical Transformer (AGMT) for small breast ultrasound datasets, which introduces the attention-gate (AG) module to suppress background information and the average radial derivative increment (ΔARD) loss function to enhance shape information. We evaluate the AGMT on both a private dataset A and a public dataset B. On dataset A, the AGMT outperforms MT on the metrics of true positive ratio, jaccard index (JI) and dice similarity coefficient (DSC) by 6.4%, 2.3% and 1.9%, respectively. Meanwhile, when compared with UNet, the AGMT improves JI and DSC by 5.3% and 4.9%, respectively. The results show performance has significantly improved compared with mainstream models. In addition, we also conduct ablation experiments on the AG module and ΔARD, which prove their effectiveness.

A deep supervised transformer U-shaped full-resolution residual network for the segmentation of breast ultrasound image.

Breast ultrasound (BUS) is an important breast imaging tool. Automatic BUS image segmentation can measure the breast tumor size objectively and reduce doctors' workload. In this article, we proposed a deep supervised transformer U-shaped full-resolution residual network (DSTransUFRRN) to segment BUS images. In the proposed method, a full-resolution residual stream and a deep supervision mechanism were introduced into TransU-Net. The residual stream can keep full resolution features from different levels and enhance features fusion. Then, the deep supervision can suppress gradient dispersion. Moreover, the transformer module can suppress irrelevant features and improve feature extraction process. Two datasets (dataset A and B) were used for training and evaluation. The dataset A included 980 BUS image samples and the dataset B had 163 BUS image samples. Cross-validation was conducted. For the dataset A, the proposed DSTransUFRRN achieved significantly higher Dice (91.04±0.86%) than all compared methods (p<0.05). For the dataset B, the Dice was lower than that for the dataset A due to the small number of samples, but the Dice of DSTransUFRRN (88.15%±2.11%) was significantly higher than that of other compared methods (p<0.05). In this study, we proposed DSTransUFRRN for BUS image segmentation. The proposed methods achieved significantly higher accuracy than the compared previous methods.

SEGMENTATION OF CANCER MASSES ON BREAST ULTRASOUND IMAGES USING MODIFIED U-NET

Breast cancer causes a huge number of women’s deaths every year. The accurate localization of a breast lesion is a crucial stage. The segmentation of breast ultrasound images participates in the improvement of the process of detection of breast anomalies. An automatic approach of segmentation of breast ultrasound images is presented in this paper, the proposed model is a modified u-net called Attention Residual U-net, designed to help radiologists in their clinical examination to determine adequately the limitation of breast tumors. Attention Residual U-net is a combination of existing models (Convolutional Neural Network U-net, the Attention Gate Mechanism and the Residual Neural Network). Public breast ultrasound images dataset of Baheya hospital in Egypt is used in this work. Dice coefficient, Jaccard index and Accuracy are used to evaluate the performance of the proposed model on the test set. Attention residual u-net can significantly give a dice coefficient = 90%, Jaccard index = 76% and Accuracy = 90%. The proposed model is compared with two other breast segmentation methods on the same dataset. The results show that the modified U-net model was able to achieve accurate segmentation of breast lesions in breast ultrasound images.

A dual-stage transformer and MLP-based network for breast ultrasound image segmentation

Automatic segmentation of breast lesions from ultrasound images plays an important role in computer-aided breast cancer diagnosis. Many deep learning methods based on convolutional neural networks (CNNs) have been proposed for breast ultrasound image segmentation. However, breast ultrasound image segmentation is still challenging due to ambiguous lesion boundaries. We propose a novel dual-stage framework based on Transformer and Multi-layer perceptron (MLP) for the segmentation of breast lesions. We combine the Swin Transformer block with an efficient pyramid squeezed attention block in a parallel design and introduce bi-directional interactions across branches, which can efficiently extract multi-scale long-range dependencies to improve the segmentation performance and robustness of the model. Furthermore, we introduce tokenized MLP block in the MLP stage to extract global contextual information while retaining fine-grained information to segment more complex breast lesions. We have conducted extensive experiments with state-of-the-art methods on three breast ultrasound datasets, including BUSI, BUL, and MT_BUS datasets. The dice coefficient reached 0.8127 ± 0.2178, and the intersection over union reached 0.7269 ± 0.2370 on benign lesions when the Hausdorff distance was maintained at 3.75 ± 1.83. The dice coefficient of malignant lesions is improved by 3.09% for BUSI dataset. The segmentation results on the BUL and MT_BUS datasets also show that our proposed model achieves better segmentation results than other methods. Moreover, the external experiments indicate that the proposed model provides better generalization capability for breast lesion segmentation. The dual-stage scheme and the proposed Transformer module achieve the fine-grained local information and long-range dependencies to relieve the burden of radiologists.

Boundary-guided and Region-aware Network with Global Scale-adaptive for Accurate Segmentation of Breast Tumors in Ultrasound Images.

Breast ultrasound (BUS) image segmentation is a critical procedure in the diagnosis and quantitative analysis of breast cancer. Most existing methods for BUS image segmentation do not effectively utilize the prior information extracted from the images. In addition, breast tumors have very blurred boundaries, various sizes and irregular shapes, and the images have a lot of noise. Thus, tumor segmentation remains a challenge. In this paper, we propose a BUS image segmentation method using a boundary-guided and region-aware network with global scale-adaptive (BGRA-GSA). Specifically, we first design a global scale-adaptive module (GSAM) to extract features of tumors of different sizes from multiple perspectives. GSAM encodes the features at the top of the network in both channel and spatial dimensions, which can effectively extract multi-scale context and provide global prior information. Moreover, we develop a boundary-guided module (BGM) for fully mining boundary information. BGM guides the decoder to learn the boundary context by explicitly enhancing the extracted boundary features. Simultaneously, we design a region-aware module (RAM) for realizing the cross-fusion of diverse layers of breast tumor diversity features, which can facilitate the network to improve the learning ability of contextual features of tumor regions. These modules enable our BGRA-GSA to capture and integrate rich global multi-scale context, multi-level fine-grained details, and semantic information to facilitate accurate breast tumor segmentation. Finally, the experimental results on three publicly available datasets show that our model achieves highly effective segmentation of breast tumors even with blurred boundaries, various sizes and shapes, and low contrast.

Noise-robustness test for ultrasound breast nodule neural network models as medical devices.

Deep learning technology has been widely applied to medical image analysis. But due to the limitations of its own imaging principle, ultrasound image has the disadvantages of low resolution and high Speckle Noise density, which not only hinder the diagnosis of patients' conditions but also affect the extraction of ultrasound image features by computer technology. In this study, we investigate the robustness of deep convolutional neural network (CNN) for classification, segmentation, and target detection of breast ultrasound image through random Salt & Pepper Noise and Gaussian Noise. We trained and validated 9 CNN architectures in 8617 breast ultrasound images, but tested the models with noisy test set. Then, we trained and validated 9 CNN architectures with different levels of noise in these breast ultrasound images, and tested the models with noisy test set. Diseases of each breast ultrasound image in our dataset were annotated and voted by three sonographers based on their malignancy suspiciousness. we use evaluation indexes to evaluate the robustness of the neural network algorithm respectively. There is a moderate to high impact (The accuracy of the model decreased by about 5%-40%) on model accuracy when Salt and Pepper Noise, Speckle Noise, or Gaussian Noise is introduced to the images respectively. Consequently, DenseNet, UNet++ and Yolov5 were selected as the most robust model based on the selected index. When any two of these three kinds of noise are introduced into the image at the same time, the accuracy of the model will be greatly affected. Our experimental results reveal new insights: The variation trend of accuracy with the noise level in Each network used for classification tasks and object detection tasks has some unique characteristics. This finding provides us with a method to reveal the black-box architecture of computer-aided diagnosis (CAD) systems. On the other hand, the purpose of this study is to explore the impact of adding noise directly to the image on the performance of neural networks, which is different from the existing articles on robustness in the field of medical image processing. Consequently, it provides a new way to evaluate the robustness of CAD systems in the future.

Paced-curriculum distillation with prediction and label uncertainty for image segmentation.

In curriculum learning, the idea is to train on easier samples first and gradually increase the difficulty, while in self-paced learning, a pacing function defines the speed to adapt the training progress. While both methods heavily rely on the ability to score the difficulty of data samples, an optimal scoring function is still under exploration. Distillation is a knowledge transfer approach where a teacher network guides a student network by feeding a sequence of random samples. We argue that guiding student networks with an efficient curriculum strategy can improve model generalization and robustness. For this purpose, we design an uncertainty-based paced curriculum learning in self-distillation for medical image segmentation. We fuse the prediction uncertainty and annotation boundary uncertainty to develop a novel paced-curriculum distillation (P-CD). We utilize the teacher model to obtain prediction uncertainty and spatially varying label smoothing with Gaussian kernel to generate segmentation boundary uncertainty from the annotation. We also investigate the robustness of our method by applying various types and severity of image perturbation and corruption. The proposed technique is validated on two medical datasets of breast ultrasound image segmentation and robot-assisted surgical scene segmentation and achieved significantly better performance in terms of segmentation and robustness. P-CD improves the performance and obtains better generalization and robustness over the dataset shift. While curriculum learning requires extensive tuning of hyper-parameters for pacing function, the level of performance improvement suppresses this limitation.

A hybrid attentional guidance network for tumors segmentation of breast ultrasound images.

In recent years, breast cancer has become the greatest threat to women. There are many studies dedicated to the precise segmentation of breast tumors, which is indispensable in computer-aided diagnosis. Deep neural networks have achieved accurate segmentation of images. However, convolutional layers are biased to extract local features and tend to lose global and location information as the network deepens, which leads to a decrease in breast tumors segmentation accuracy. For this reason, we propose a hybrid attention-guided network (HAG-Net). We believe that this method will improve the detection rate and segmentation of tumors in breast ultrasound images. The method is equipped with multi-scale guidance block (MSG) for guiding the extraction of low-resolution location information. Short multi-head self-attention (S-MHSA) and convolutional block attention module are used to capture global features and long-range dependencies. Finally, the segmentation results are obtained by fusing multi-scale contextual information. We compare with 7 state-of-the-art methods on two publicly available datasets through five random fivefold cross-validations. The highest dice coefficient, Jaccard Index and detect rate ([Formula: see text]%, [Formula: see text]%, [Formula: see text]% and [Formula: see text]%, [Formula: see text]%, [Formula: see text]%, separately) obtained on two publicly available datasets(BUSI and OASUBD), prove the superiority of our method. HAG-Net can better utilize multi-resolution features to localize the breast tumors. Demonstrating excellent generalizability and applicability for breast tumors segmentation compare to other state-of-the-art methods.

HCTNet: A hybrid CNN-transformer network for breast ultrasound image segmentation

Automatic breast ultrasound image segmentation helps radiologists to improve the accuracy of breast cancer diagnosis. In recent years, the convolutional neural networks (CNNs) have achieved great success in medical image analysis. However, it exhibits limitations in modeling long-range relations, which is unfavorable for ultrasound images with speckle noise and shadows, resulting in decreased accuracy of breast lesion segmentation. Transformer can obtain sufficient global information, but it is deficient in acquiring local details and needs to be pre-trained on large-scale datasets. In this paper, we propose a Hybrid CNN-Transformer network (HCTNet) for boosting the breast lesion segmentation in ultrasound images. In the encoder of HCTNet, Transformer Encoder Blocks (TEBlocks) are designed to learn the global contextual information, which are combined with CNNs to extract features. In the decoder of HCTNet, a Spatial-wise Cross Attention (SCA) module is developed based on the spatial attention mechanism, which reduces the semantic discrepancy with the encoder. Moreover, residual connection is used between decoder blocks to make the generated features more discriminative by aggregating contextual feature maps at different semantic scales. Extensive experiments on three public breast ultrasound datasets demonstrate that HCTNet outperforms other medical image segmentation methods and the recent semantic segmentation methods on breast ultrasound lesion segmentation.

A Regional-Attentive Multi-Task Learning Framework for Breast Ultrasound Image Segmentation and Classification

Breast ultrasound (BUS) imaging is commonly used in the early detection of breast cancer as a portable, valuable, and widely available diagnosis tool. Automated BUS image classification and segmentation can assist radiologists in making accurate and fast decisions. Recent studies illustrate that tumor, peritumoral, and background regions of BUS images provide valuable information for BUS image segmentation or classification. However, few studies have investigated the influence of these three regions on multi-task learning. In this study, we propose an RMTL-Net to simultaneously segment tumor regions and classify tumors in BUS images into benign or malignant categories. To improve both segmentation and classification performance, we design a regional attention (RA) module that employs the predicted probability maps to automatically guide the classifier to learn important category-sensitive information in the tumor, peritumoral, and background regions and seamlessly fuse them to obtain a better feature representation. We conduct detailed ablation experiments of the proposed RA module and comparative experiments with four recent state-of-the-art peer multi-task learning methods, three single-task segmentation methods, and four single-task classification methods on two public BUS datasets. Experimental results show that the proposed RMTL-Net achieves the best overall segmentation and classification accuracy in terms of five segmentation metrics and six classification metrics.