Transferred Semantic Scores for Scalable Retrieval of Histopathological Breast Cancer Images

Abstract

Content-based medical image retrieval (CBMIR) is an active field of research and a complementary decision support tool for the diagnosis of breast cancer. Current CBMIR systems employ hand-engineered image descriptors which are not effective enough at retrieval phase. Besides this drawback, the so-called semantic gap in the CBMIR is not still addressed leaving the room for further improvements. To fill in the two mentioned existing gaps, we proposed a new retrieval method which exploited a deep pre-trained convolutional neural network model to extract class-specific and patient-specific tumorous descriptor to firstly train a binary breast cancer classifier and then a multi-patient classifier aiming for reducing dimensions of the raw deeply transferred features and obtaining semantic scores which significantly enhanced the performance in terms of mean average precision. We evaluated the method on scalable BreakHis dataset of histopathological breast cancer images. After conducting five sets of experiments, results demonstrated the superior effectiveness of the proposed semantic-driven retrieval methods by means of increased mean average precision and decreased dimensionality and retrieval time. In overall, an improvement of 29.03% was obtained by the proposed class-driven semantic retrieval method.