Content Based Medical Image Retrieval Based on Salient Regions Combined with Deep Learning

Abstract

In traditional text based medical image retrieval system, it is hard to find visually similar images in large medical image database. Content-based image retrieval is developed to retrieve similar images and it is based on visual attributes describing the content of an image. Developing a method for content based medical image retrieval is a challenging task. This paper proposed a new method for content-based medical image retrieval based on salient regions and deep learning. The proposed method includes two stages: an offline task to extract local object features and an online task for content-based image retrieval in database. In first stage, we extract local object features of medical image depending on shape, texture and intensity, and features extracted by deep learning applied in saliency of decomposition. Secondly, we make online task for content-based image retrieval in database. The user gives query image as an input and the system will retrieve n top most similar images by similarity comparison with bag of code words feature values obtained in the first stage. Evaluation of the proposed method is based on Precision and Recall values. Our dataset includes 5 groups of medical images with their quality varying from low to high. With the best medical image quality group, the accuray can be 91.61% for Precision and 89.61% for Recall. Comparing the average values with others methods, the results of the proposed method are more than 2 to 5% better.