Automated diagnosis of multi-plane breast ultrasonography images using deep neural networks

Abstract

Breast ultrasonography is currently considered the first-line examination in the diagnosis of breast lesions or other abnormalities. Many automated breast cancer diagnosis methods have been developed for ultrasonography images; however, most previous automated methods used only a single breast ultrasonography image. This is inconsistent with the real-world situation, as breast cancer is heterogeneous, and it can lead to high false negative rates. Generally, sonographers diagnose a lesion by reviewing multiple planes. In this paper, we formulate the diagnosis of breast cancer on ultrasonography images as a Multiple Instance Learning (MIL) problem, diagnosing a breast nodule by jointly analyzing the nodule on multiple planes. An attention-augmented deep neural network is then developed to solve this problem. To the best of our knowledge, this is the first implementation of a MIL framework on such data. A large breast ultrasonography image dataset was constructed to train and evaluate the model; this contained 10,464 images from 3700 lesions labeled as benign or malignant, which originated from 2568 patients. The high classification accuracy achieved demonstrates the effectiveness of the proposed architecture for the diagnosis of breast cancer. The MIL based method obtained superior performance to single instance methods in this breast cancer diagnosis task. Notably, the proposed attention-augmented network allowed us to find key instances, which can be provided the region of interest (ROI) in the final diagnosis given to a doctor. Furthermore, we empirically demonstrate that our approach achieves better performance than other state-of-the-art MIL methods.