Breast Tumors In Ultrasound Images Research Articles

Leveraging Artificial Neural Networks and Support Vector Machines for Accurate Classification of Breast Tumors in Ultrasound Images

Leveraging Artificial Neural Networks and Support Vector Machines for Accurate Classification of Breast Tumors in Ultrasound Images

Attention based UNet model for breast cancer segmentation using BUSI dataset.

Breast cancer, a prevalent and life-threatening disease, necessitates early detection for the effective intervention and the improved patient health outcomes. This paper focuses on the critical problem of identifying breast cancer using a model called Attention U-Net. The model is utilized on the Breast Ultrasound Image Dataset (BUSI), comprising 780 breast images. The images are categorized into three distinct groups: 437 cases classified as benign, 210 cases classified as malignant, and 133 cases classified as normal. The proposed model leverages the attention-driven U-Net's encoder blocks to capture hierarchical features effectively. The model comprises four decoder blocks which is a pivotal component in the U-Net architecture, responsible for expanding the encoded feature representation obtained from the encoder block and for reconstructing spatial information. Four attention gates are incorporated strategically to enhance feature localization during decoding, showcasing a sophisticated design that facilitates accurate segmentation of breast tumors in ultrasound images. It displays its efficacy in accurately delineating and segregating tumor borders. The experimental findings demonstrate outstanding performance, achieving an overall accuracy of 0.98, precision of 0.97, recall of 0.90, and a dice score of 0.92. It demonstrates its effectiveness in precisely defining and separating tumor boundaries. This research aims to make automated breast cancer segmentation algorithms by emphasizing the importance of early detection in boosting diagnostic capabilities and enabling prompt and targeted medical interventions.

Breast lesions segmentation from ultrasound images using DeepLabV3 + model with channel and spatial attention mechanism

Segmenting breast tumors in ultrasound images is pivotal for computer-aided diagnosis (CAD) systems aimed at detecting breast cancer. This research contributes directly to the Sustainable Development Goal (SDG), good health and well-being by leveraging innovative deep learning technique to boost the accuracy of breast tumor segmentation. Achieving accurate segmentation is crucial for precise tumor size, shape, and location determination, which in turn facilitates subsequent tumor quantification and classification. However, this task presents challenges, particularly when dealing with small tumors. The complexity arises from factors such as speckle noise, variations in tumor shapes and sizes across patients, and the presence of tumor-like regions in the images. Although deep learning-based methods have shown remarkable success in biomedical image analysis, the current state-of-the-art approaches still struggle to achieve satisfactory performance in segmenting small breast tumors. In this paper, BUS (Breast Ultrasound) dataset was initially applied on state of-the art DeepLabV3 + architecture for the segmentation of breast lesions in ultrasound images. After that to focus on more informative features, a modified DeepLabV3 + has been proposed by integrating a CBAM module in encoder as well as decoder. At the end, comparative analysis of state-of-art DeepLabV3 + model and proposed modified DeepLabV3 + has also been presented in terms of performance metrics like Dice Coefficient, Intersection over Union, Precision, Recall and Specificity. The proposed model demonstrates outstanding performance across different evaluation metrics achieving precision, recall, specificity, dice coefficient and Intersection over Union (IoU) values of 0.974, 0.933, 0.997, 0.951, and 0.933. respectively.

Multi-task approach based on combined CNN-transformer for efficient segmentation and classification of breast tumors in ultrasound images

Nowadays, inspired by the great success of Transformers in Natural Language Processing, many applications of Vision Transformers (ViTs) have been investigated in the field of medical image analysis including breast ultrasound (BUS) image segmentation and classification. In this paper, we propose an efficient multi-task framework to segment and classify tumors in BUS images using hybrid convolutional neural networks (CNNs)-ViTs architecture and Multi-Perceptron (MLP)-Mixer. The proposed method uses a two-encoder architecture with EfficientNetV2 backbone and an adapted ViT encoder to extract tumor regions in BUS images. The self-attention (SA) mechanism in the Transformer encoder allows capturing a wide range of high-level and complex features while the EfficientNetV2 encoder preserves local information in image. To fusion the extracted features, a Channel Attention Fusion (CAF) module is introduced. The CAF module selectively emphasizes important features from both encoders, improving the integration of high-level and local information. The resulting feature maps are reconstructed to obtain the segmentation maps using a decoder. Then, our method classifies the segmented tumor regions into benign and malignant using a simple and efficient classifier based on MLP-Mixer, that is applied for the first time, to the best of our knowledge, for the task of lesion classification in BUS images. Experimental results illustrate the outperformance of our framework compared to recent works for the task of segmentation by producing 83.42% in terms of Dice coefficient as well as for the classification with 86% in terms of accuracy.

Boundary-guided and Region-aware Network with Global Scale-adaptive for Accurate Segmentation of Breast Tumors in Ultrasound Images.

Breast ultrasound (BUS) image segmentation is a critical procedure in the diagnosis and quantitative analysis of breast cancer. Most existing methods for BUS image segmentation do not effectively utilize the prior information extracted from the images. In addition, breast tumors have very blurred boundaries, various sizes and irregular shapes, and the images have a lot of noise. Thus, tumor segmentation remains a challenge. In this paper, we propose a BUS image segmentation method using a boundary-guided and region-aware network with global scale-adaptive (BGRA-GSA). Specifically, we first design a global scale-adaptive module (GSAM) to extract features of tumors of different sizes from multiple perspectives. GSAM encodes the features at the top of the network in both channel and spatial dimensions, which can effectively extract multi-scale context and provide global prior information. Moreover, we develop a boundary-guided module (BGM) for fully mining boundary information. BGM guides the decoder to learn the boundary context by explicitly enhancing the extracted boundary features. Simultaneously, we design a region-aware module (RAM) for realizing the cross-fusion of diverse layers of breast tumor diversity features, which can facilitate the network to improve the learning ability of contextual features of tumor regions. These modules enable our BGRA-GSA to capture and integrate rich global multi-scale context, multi-level fine-grained details, and semantic information to facilitate accurate breast tumor segmentation. Finally, the experimental results on three publicly available datasets show that our model achieves highly effective segmentation of breast tumors even with blurred boundaries, various sizes and shapes, and low contrast.

Accurate segmentation of breast tumor in ultrasound images through joint training and refined segmentation

Objective. This paper proposes an automatic breast tumor segmentation method for two-dimensional (2D) ultrasound images, which is significantly more accurate, robust, and adaptable than common deep learning models on small datasets. Approach. A generalized joint training and refined segmentation framework (JR) was established, involving a joint training module (J module ) and a refined segmentation module (R module ). In J module , two segmentation networks are trained simultaneously, under the guidance of the proposed Jocor for Segmentation (JFS) algorithm. In R module , the output of J module is refined by the proposed area first (AF) algorithm, and marked watershed (MW) algorithm. The AF mainly reduces false positives, which arise easily from the inherent features of breast ultrasound images, in the light of the area, distance, average radical derivative (ARD) and radical gradient index (RGI) of candidate contours. Meanwhile, the MW avoids over-segmentation, and refines segmentation results. To verify its performance, the JR framework was evaluated on three breast ultrasound image datasets. Image dataset A contains 1036 images from local hospitals. Image datasets B and C are two public datasets, containing 562 images and 163 images, respectively. The evaluation was followed by related ablation experiments. Main results. The JR outperformed the other state-of-the-art (SOTA) methods on the three image datasets, especially on image dataset B. Compared with the SOTA methods, the JR improved true positive ratio (TPR) and Jaccard index (JI) by 1.5% and 3.2%, respectively, and reduces (false positive ratio) FPR by 3.7% on image dataset B. The results of the ablation experiments show that each component of the JR matters, and contributes to the segmentation accuracy, particularly in the reduction of false positives. Significance. This study successfully combines traditional segmentation methods with deep learning models. The proposed method can segment small-scale breast ultrasound image datasets efficiently and effectively, with excellent generalization performance.

A Breast Cancer Contour Detection With Level Sets and SVM Model

Level sets have been widely used to isolate features of breast tumors in ultrasound images. However, region-based methods always produce multiple contours. Since tumors are regularly undefined from the shadows and muscular regions in breast ultrasound images, computerized tumors location and arrangement is significantly difficult. Therefore, the authors introduce a breast cancer contour detection model using support vector machine (SVM) as a binary classification. Features of the binary SVM model were extracted from level sets and FM method (the fusion of ultrasound, elasticity, and Doppler images). The model was accurately able to predict a correct breast tumor contour from false contours which were segmented by region-based level sets. The proposed method was evaluated on 60 datasets collected by professional radiologists at the Thammasat University Hospital of Thailand. From the experimental results, the breast cancer contours were detected correctly with high accuracy. The percentage of correct detection was 93%.

MfdcModel: A Novel Classification Model for Classification of Benign and Malignant Breast Tumors in Ultrasound Images

Automatic classification of benign and malignant breast ultrasound images is an important and challenging task to improve the efficiency and accuracy of clinical diagnosis of breast tumors and reduce the rate of missed and misdiagnosis. The task often requires a large amount of data to train. However, it is difficult to obtain medical images, which contradicts the large amount of data needed to obtain good diagnostic models for training. In this paper, a novel classification model for the classification of breast tumors is proposed to improve the performance of diagnosis models trained by small datasets. The method integrates three features from medical features extracted from segmented images, features selected from the pre-trained ResNet101 output by principal component analysis (PCA), and texture features. Among the medical features that are used to train the naive Bayes (NB) classifier, and the PCA-selected features are used to train the support vector machine (SVM) classifier. Subsequently, the final results of boosting are obtained by weighting the classifiers. A five-fold cross-validation experiment yields an average accuracy of 89.17%, an average precision of 90.00%, and an average AUC value of 0.95. According to the experimental results, the proposed method has better classification accuracy compared to the accuracy obtained by other models trained on only small datasets. This approach can serve as a reliable second opinion for radiologists, and it can also provide useful advice for junior radiologists who do not have sufficient clinical experience.

Fus2Net: a novel Convolutional Neural Network for classification of benign and malignant breast tumor in ultrasound images

BackgroundThe rapid development of artificial intelligence technology has improved the capability of automatic breast cancer diagnosis, compared to traditional machine learning methods. Convolutional Neural Network (CNN) can automatically select high efficiency features, which helps to improve the level of computer-aided diagnosis (CAD). It can improve the performance of distinguishing benign and malignant breast ultrasound (BUS) tumor images, making rapid breast tumor screening possible.ResultsThe classification model was evaluated with a different dataset of 100 BUS tumor images (50 benign cases and 50 malignant cases), which was not used in network training. Evaluation indicators include accuracy, sensitivity, specificity, and area under curve (AUC) value. The results in the Fus2Net model had an accuracy of 92%, the sensitivity reached 95.65%, the specificity reached 88.89%, and the AUC value reached 0.97 for classifying BUS tumor images.ConclusionsThe experiment compared the existing CNN-categorized architecture, and the Fus2Net architecture we customed has more advantages in a comprehensive performance. The obtained results demonstrated that the Fus2Net classification method we proposed can better assist radiologists in the diagnosis of benign and malignant BUS tumor images.MethodsThe existing public datasets are small and the amount of data suffer from the balance issue. In this paper, we provide a relatively larger dataset with a total of 1052 ultrasound images, including 696 benign images and 356 malignant images, which were collected from a local hospital. We proposed a novel CNN named Fus2Net for the benign and malignant classification of BUS tumor images and it contains two self-designed feature extraction modules. To evaluate how the classifier generalizes on the experimental dataset, we employed the training set (646 benign cases and 306 malignant cases) for tenfold cross-validation. Meanwhile, to solve the balance of the dataset, the training data were augmented before being fed into the Fus2Net. In the experiment, we used hyperparameter fine-tuning and regularization technology to make the Fus2Net convergence.

Accurate segmentation of breast tumors using AE U-net with HDC model in ultrasound images

Breast cancer poses a great threat on women health due to its high malignant rate. In China, ultrasound screening is the commonly-used method for breast cancer diagnosis, and the localization and segmentation of the lesions in ultrasound images are helpful for breast cancer detection. In this paper, an Attention Enhanced U-net with hybrid dilated convolution (AE U-net with HDC) model was proposed and employed to segment the breast tumors in ultrasound images. First, based on Attention U-net, we added a new loss function to update the weight matrix in the AGs module, in order to enhance the weight of the lesion area. Combined with fine-tuning training method, the precision of breast ultrasound image lesion region segmentation was improved from 82.38% to 86.28% and the M-IOU was improved from 76.27% to 81.81%. Second, three groups of HDC with expansion rates of [1,2,5] were integrated into AE U-net to replace the four convolution operations. HDC module brought larger receptive field and reduced the loss of spatial information. The experimental results proved that HDC module was helpful to improve the Acc of image segmentation results from 94.18% to 95.81% and the Recall from 78.69% to 80.48%. Combined with U-net, the F1 score, AUC, Acc and M-IOU of the network proposed in this paper had significantly improved. It proved that AE U-net with HDC model would have very important research value and application prospect for modern medicine.

Gaussian Dropout Based Stacked Ensemble CNN for Classification of Breast Tumor in Ultrasound Images

ObjectiveBreast cancer and breast tumors have been considered to be the most pervasive form of cancer in medical practice. Breast tumors are life-threatening to women, and their early detection could save lives with the proper treatment. Physical methods for detection of Breast Cancer are time-consuming and often prone to a misdiagnosis at classifying tumors. Recent trends in radiological imaging have significantly improved the efficiency and veracity of breast tumor classification. Artificial intelligence techniques could be used as an automated detection and classification system. Materials and methodsIn this research, we propose a novel configuration of a Stacking Ensemble with custom Convolutional Neural Network architectures to classify breast tumors from ultrasound images into ‘Normal’, ‘Benign’, and ‘Malignant’ categories. ResultsAfter thorough experimentation, our ensemble has performed with an accuracy, f1-score, precision, and recall of 92.15%, 92.21%, 92.26%, 92.17% respectively. ConclusionThe presented ensemble leverages three Stacked Feature Extractors coupled with a characteristic meta-learner to provide an overall balanced classification performance, with better accuracy and lower false positives. The architecture works in association with gaussian dropout layers to improve the computation and an alternative pooling scheme to retain essential features.

Automatic semantic segmentation of breast tumors in ultrasound images based on combining fuzzy logic and deep learning-A feasibility study.

Computer aided diagnosis (CAD) of biomedical images assists physicians for a fast facilitated tissue characterization. A scheme based on combining fuzzy logic (FL) and deep learning (DL) for automatic semantic segmentation (SS) of tumors in breast ultrasound (BUS) images is proposed. The proposed scheme consists of two steps: the first is a FL based preprocessing, and the second is a Convolutional neural network (CNN) based SS. Eight well-known CNN based SS models have been utilized in the study. Studying the scheme was by a dataset of 400 cancerous BUS images and their corresponding 400 ground truth images. SS process has been applied in two modes: batch and one by one image processing. Three quantitative performance evaluation metrics have been utilized: global accuracy (GA), mean Jaccard Index (mean intersection over union (IoU)), and mean BF (Boundary F1) Score. In the batch processing mode: quantitative metrics' average results over the eight utilized CNNs based SS models over the 400 cancerous BUS images were: 95.45% GA instead of 86.08% without applying fuzzy preprocessing step, 78.70% mean IoU instead of 49.61%, and 68.08% mean BF score instead of 42.63%. Moreover, the resulted segmented images could show tumors' regions more accurate than with only CNN based SS. While, in one by one image processing mode: there has been no enhancement neither qualitatively nor quantitatively. So, only when a batch processing is needed, utilizing the proposed scheme may be helpful in enhancing automatic ss of tumors in BUS images. Otherwise applying the proposed approach on a one-by-one image mode will disrupt segmentation's efficiency. The proposed batch processing scheme may be generalized for an enhanced CNN based SS of a targeted region of interest (ROI) in any batch of digital images. A modified small dataset is available: https://www.kaggle.com/mohammedtgadallah/mt-small-dataset (S1 Data).

Improved U-net MALF model for lesion segmentation in breast ultrasound images

Breast cancer poses a great threat on women health due to its high malignant rate. In China, ultrasound screening is the commonly-used method for breast cancer diagnosis, and the localization and segmentation of the lesions in ultrasound images are helpful for breast cancer detection. In this paper, an Improved U-net based on Mixed Attention Loss Function (Improved U-net MALF) model was proposed and employed to segment the breast tumors in ultrasound images. Firstly, based on the attention U-net network framework, the residual convolution module and extended residual convolution module were used to replace the convolution module of the coding path, so as to extract more detailed features of ultrasound breast tumor without increasing the computational cost. Secondly, on the basis of the traditional cross entropy loss function, four attention loss functions were integrated. The proportion of the four attention function loss values in the total loss value was measured by the texture consistency index of the feature map to further highlight the tumor target. The combined application of residual block and hybrid attention loss function improves the segmentation accuracy of breast tumor greatly. The results show that our model presented excellent segmentation results as compared to other Deep Networks when tested breast ultrasound images. A variety of quantitative indicators provided performances above 80 % including accuracy, specificity and sensitivity. It illustrates that the proposed Improved U-net MALF model can accurately segment breast lesions and thus has a good prospect for clinical diagnosis.

Multitask Classification Method Based on Label Correction for Breast Tumor Ultrasound Images

To enable deep learning-based computer-aided diagnosis to achieve excellent performance in differentiating benign and malignant breast tumors in ultrasound images, a large number of labeled training samples must be collected. However, it is difficult to acquire sufficient samples due to the high costs of data collection and labeling. Fortunately, breast ultrasound images have two labels from different sources of domain knowledge: the biopsy results are “clean” labels, and the Breast Imaging Reporting and Data System (BI-RADS) score functions as a “noisy” label. Based on these two label types, we propose a multitask classification method based on label distribution correction (MTLC-Net). In our method, we propose different tasks to address the noisy and clean labels. Specifically, we propose a label distribution correction task for noisy labels that includes jointly training the network parameters and soft labels. The model is generalizable and robust by correcting the noisy label distribution based on the BI-RADS score, and it extracts knowledge from the noisy label task to improve the learning in the clean-label task. We conducted extensive comparisons with existing methods. Our method achieved a classification accuracy of 75.8%, a precision of 73.0%, a recall of 80.1% and an F1 score of 0.764—results that are significantly better than those of the existing state-of-the-art methods for differentiating benign and malignant breast tumors in ultrasound images.

A comparative study of pre-trained convolutional neural networks for semantic segmentation of breast tumors in ultrasound

The automatic segmentation of breast tumors in ultrasound (BUS) has recently been addressed using convolutional neural networks (CNN). These CNN-based approaches generally modify a previously proposed CNN architecture or they design a new architecture using CNN ensembles. Although these methods have reported satisfactory results, the trained CNN architectures are often unavailable for reproducibility purposes. Moreover, these methods commonly learn from small BUS datasets with particular properties, which limits generalization in new cases. This paper evaluates four public CNN-based semantic segmentation models that were developed by the computer vision community, as follows: (1) Fully Convolutional Network (FCN) with AlexNet network, (2) U-Net network, (3) SegNet using VGG16 and VGG19 networks, and (4) DeepLabV3+ using ResNet18, ResNet50, MobileNet-V2, and Xception networks. By transfer learning, these CNNs are fine-tuned to segment BUS images in normal and tumoral pixels. The goal is to select a potential CNN-based segmentation model to be further used in computer-aided diagnosis (CAD) systems. The main significance of this study is the comparison of eight well-established CNN architectures using a more extensive BUS dataset than those used by approaches that are currently found in the literature. More than 3000 BUS images acquired from seven US machine models are used for training and validation. The F1-score (F1s) and the Intersection over Union (IoU) quantify the segmentation performance. The segmentation models based on SegNet and DeepLabV3+ obtain the best results with F1s>0.90 and IoU>0.81. In the case of U-Net, the segmentation performance is F1s=0.89 and IoU=0.80, whereas FCN-AlexNet attains the lowest results with F1s=0.84 and IoU=0.73. In particular, ResNet18 obtains F1s=0.905 and IoU=0.827 and requires less training time among SegNet and DeepLabV3+ networks. Hence, ResNet18 is a potential candidate for implementing fully automated end-to-end CAD systems. The CNN models generated in this study are available to researchers at https://github.com/wgomezf/CNN-BUS-segment, which attempts to impact the fair comparison with other CNN-based segmentation approaches for BUS images.

Breast tumors recognition based on edge feature extraction using support vector machine

Nowadays, it is important for the detection of ultrasound images of breast tumors. In this paper, a new ultrasonic image feature extraction algorithm combining edge-based features and morphologic feature information is proposed, which has good effect on benign and malignant identification of breast tumors. This paper mainly studies three features (Sum of maximum curvature, Sum of maximum curvature and peak, Sum of maximum curvature and standard deviation) according to the shape histogram of ultrasound breast tumors from a local perspective. Based on the results of SVM classifier, it was found that the edge-based features have higher classification accuracy. The recognition system would perform better when morphologic features (Roughness, Regularity, Aspect ratio, Ellipticity, Roundness) were incorporated, compared with the control group whose input only with morphologic features. The results show that edge-based features can well describe breast tumors in ultrasound images, and have the potential to be used in breast ultrasound computer-aided design.

Breast tumor classification through learning from noisy labeled ultrasound images.

To train deep learning models to differentiate benign and malignant breast tumors in ultrasound images, we need to collect many training samples with clear labels. In general, biopsy results can be used as benign/malignant labels. However, most clinical samples generally do not have biopsy results. Previous works have proposed generating benign/malignant labels according to Breast Imaging, Reporting and Data System (BI-RADS) ratings. However, this approach will cause noisy labels, which means that the benign/malignant labels produced from BI-RADS diagnoses may be inconsistent with the ground truths. Consequently, deep models will overfit the noisy labels and hence obtain poor generalization performance. In this work, we mainly focus on how to reduce the negative effect of noisy labels when they are used to train breast tumor classification models. We propose an effective approach called noise filter network (NF-Net) to address the problem of noisy labels when training breast tumor classification models. Specifically, to prevent deep models from overfitting the noisy labels, we propose incorporating two softmax layers for classification. Additionally, to strengthen the effect of clean labels, we design a teacher-student module for distilling the knowledge of clean labels. We conduct extensive comparisons with the existing works on addressing noisy labels. Our method achieves a classification accuracy of 73%, with a precision of 69%, recall of 80%, and F1-score of 0.74. This result is significantly better than those of the existing state-of-the-art works on addressing noisy labels. This work provides a means to overcome the label shortage problem in training breast tumor classification models. Specifically, we can generate benign/malignant labels according to the BI-RADS ratings. Although this approach will cause noisy labels, the design of NF-Net can effectively reduce the negative effect of such labels.

Particle method for segmentation of breast tumors in ultrasound images

We propose a new segmentation method based on multiple walking particles (WP) bouncing from the image edges. The particles are able to segment objects characterized by deep concavities as narrow as one pixel and handle single or multiple objects characterized by a noisy background and broken boundaries (“weak edge”, “boundary leakage”). The particles are designed to segment the image by permanently staying inside the object and repairing the boundaries where necessary. The proposed WP combine the advantages of the continuous diffusion models with the principles of multi-agent systems.WP have been tested against recent active contours and the distance regularized level set method on a set of complex-shaped synthetic images and ultrasound (US) images of breast cancer (http://onlinemedicalimages.com). The method has also been compared with localizing region-based active contours, the fuzzy C-mean level set method, and morphological active contours. The WP are faster and more accurate for images characterized by low contrast, noise, broken boundaries, or boundary leakage. However, for good quality, simple shaped objects the WP work similarly to the conventional methods. There is still an important difference even in this case: the WP do not require initialization.A video demo of the algorithm is at https://drive.google.com/drive/folders/1SlTphINKtdUwvdjjxiakFrwI2dDlIUAU

Salient object detection method for breast tumor in ultrasound images based on absorbing Markov chain.

Automatic detection of tumor in breast ultrasound (BUS) images is important for the subsequent image processing and has been researched for decades. However, there still lacks a robust method due to poor quality of BUS images. To propose and test a salient object detection method for BUS images. BUS image is preprocessed by an adaptively selective replacement and speckle reducing anisotropic diffusion (SRAD) algorithm. Then, the preprocessed image is segmented into super pixels by a simple linear iterative clustering (SLIC) algorithm to form a graph model, and the saliency of the nodes in the graph is calculated by using the absorbed time of absorbing Markov chain (AMC). Finally, the initial saliency map is optimized by the recurrent time of ergodic Markov chain (EMC) and a distance weighting formula. Results of the proposed method were compared both qualitatively and quantitatively with two saliency detection models. It was observed that the proposed method outperformed the comparison models and yielded the highest Accuracy value (97.49% vs. 86.63% and 90.33%) using a dataset of 1000 BUS images. After the adaptively selective replacement, AMC can effectively distinguish tumors from background by random walks.

Two-stage CNNs for computerized BI-RADS categorization in breast ultrasound images

BackgroundQuantizing the Breast Imaging Reporting and Data System (BI-RADS) criteria into different categories with the single ultrasound modality has always been a challenge. To achieve this, we proposed a two-stage grading system to automatically evaluate breast tumors from ultrasound images into five categories based on convolutional neural networks (CNNs).MethodsThis new developed automatic grading system was consisted of two stages, including the tumor identification and the tumor grading. The constructed network for tumor identification, denoted as ROI-CNN, can identify the region contained the tumor from the original breast ultrasound images. The following tumor categorization network, denoted as G-CNN, can generate effective features for differentiating the identified regions of interest (ROIs) into five categories: Category “3”, Category “4A”, Category “4B”, Category “4C”, and Category “5”. Particularly, to promote the predictions identified by the ROI-CNN better tailor to the tumor, refinement procedure based on Level-set was leveraged as a joint between the stage and grading stage.ResultsWe tested the proposed two-stage grading system against 2238 cases with breast tumors in ultrasound images. With the accuracy as an indicator, our automatic computerized evaluation for grading breast tumors exhibited a performance comparable to that of subjective categories determined by physicians. Experimental results show that our two-stage framework can achieve the accuracy of 0.998 on Category “3”, 0.940 on Category “4A”, 0.734 on Category “4B”, 0.922 on Category “4C”, and 0.876 on Category “5”.ConclusionThe proposed scheme can extract effective features from the breast ultrasound images for the final classification of breast tumors by decoupling the identification features and classification features with different CNNs. Besides, the proposed scheme can extend the diagnosing of breast tumors in ultrasound images to five sub-categories according to BI-RADS rather than merely distinguishing the breast tumor malignant from benign.