A Temporal Sequence Dual-Branch Network for Classifying Hybrid Ultrasound Data of Breast Cancer

Ziqi Yang,Ying Guo,Xun Gong,Wenbin Liu

doi:10.1109/access.2020.2990683

Abstract

In clinical medicine, the contrast-enhanced ultrasound (CEUS) has been a commonly used imaging modality for diagnosis of breast tumor. However, most researchers in computer vision field only focus on B-mode ultrasound image which does not get good results. To improve the accuracy of classification, first, we propose a novel method, i.e., a Temporal Sequence Dual-Branch Network (TSDBN) which, for the first time, can use B-mode ultrasound data and CEUS data simultaneously. Second, we designed a new Gram matrix to model the temporal sequence, and then proposed a Temporal Sequence Regression Mechanism (TSRM), which is a novel method to extract the enhancement features from CEUS video based on the matrix. For B-mode ultrasound branch, we use the traditional ResNeXt network for feature extraction. While CEUS branch uses ResNeXt + R(2 + 1) D network as the backbone network. We propose a TSRM to learning temporal sequence relationship among frames, and design a Shuffle Temporal Sequence Mechanism (STSM) to shuffle temporal sequences, the purpose of which is to further enhance temporal information among frames. Experimental results show that the proposed TSRM could use temporal information effectively and the accuracy of TSDBN is higher than that of state-of-art approaches in breast cancer classification by nearly 4%.

Highlights

Breast cancer is the most common cancer of women and the second leading cause of cancer death [1]
In order to assess the role of contrast-enhanced ultrasound (CEUS) video in different methods, three experiments are carried out: the first experiment only uses B-mode ultrasound image; the second only uses CEUS video; the third uses both data to classify breast tumor
Combining B-mode ultrasound image and CEUS video, our method can reach to the best Acc of 90.2%

Summary

Introduction

Breast cancer is the most common cancer of women and the second leading cause of cancer death [1]. Detection of breast cancer has been shown to significantly improve survival rate of patients [2], [3]. Correct diagnosis at early stage received widespread attention. Ultrasound has been widely used in the detection of early breast cancer because of its safety, low cost and high versatility [4]. Its diagnostic accuracy depends on the special skills of the ultrasonic physicians—it says that the diagnosis difference could be larger than 30% among physicians of different levels [5]. With the excellent performance of deep learning in image recognition, it has been widely used in

Methods

Results

Conclusion